INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50198 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:16:31,864 - lava_callback - INFO - Received callback for node 69bd72c10e93e2314723e28d 2026-03-22 16:16:31,870 - lava_callback - INFO - Starting async job submit for node 69bd72c10e93e2314723e28d INFO: 172.20.0.1:53190 - "POST /node/69bd72c10e93e2314723e28d HTTP/1.0" 202 Accepted 2026-03-22 16:16:31,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:16:31,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd72c10e93e2314723e28d HTTP/1.1" 200 None 2026-03-22 16:16:31,900 - lava_callback - DEBUG - Processing node 69bd72c10e93e2314723e28d: baseline-arm-clabbe 2026-03-22 16:16:31,900 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd72c10e93e2314723e28d 2026-03-22 16:16:31,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:16:31,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd72c10e93e2314723e28d HTTP/1.1" 200 None 2026-03-22 16:16:31,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:16:31,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:16:31,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:16:31,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd72c10e93e2314723e28d HTTP/1.1" 200 None 2026-03-22 16:16:31,960 - lava_callback - ERROR - Error processing callback for node 69bd72c10e93e2314723e28d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46118 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:18:26,972 - lava_callback - INFO - Received callback for node 69bd73430e93e2314723e2c6 INFO: 172.20.0.1:46120 - "POST /node/69bd73430e93e2314723e2c6 HTTP/1.0" 202 Accepted 2026-03-22 16:18:26,981 - lava_callback - INFO - Starting async job submit for node 69bd73430e93e2314723e2c6 2026-03-22 16:18:26,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:18:27,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73430e93e2314723e2c6 HTTP/1.1" 200 None 2026-03-22 16:18:27,032 - lava_callback - DEBUG - Processing node 69bd73430e93e2314723e2c6: baseline-arm64-baylibre 2026-03-22 16:18:27,033 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd73430e93e2314723e2c6 2026-03-22 16:18:27,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:18:27,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73430e93e2314723e2c6 HTTP/1.1" 200 None 2026-03-22 16:18:27,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:18:27,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 16:18:27,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:18:27,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd73430e93e2314723e2c6 HTTP/1.1" 200 None 2026-03-22 16:18:27,150 - lava_callback - ERROR - Error processing callback for node 69bd73430e93e2314723e2c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40422 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:21:06,886 - lava_callback - INFO - Received callback for node 69bd73e00e93e2314723e32d INFO: 172.20.0.1:40436 - "POST /node/69bd73e00e93e2314723e32d HTTP/1.0" 202 Accepted 2026-03-22 16:21:06,891 - lava_callback - INFO - Starting async job submit for node 69bd73e00e93e2314723e32d 2026-03-22 16:21:06,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:21:06,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73e00e93e2314723e32d HTTP/1.1" 200 None 2026-03-22 16:21:06,908 - lava_callback - DEBUG - Processing node 69bd73e00e93e2314723e32d: baseline-arm64-baylibre 2026-03-22 16:21:06,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd73e00e93e2314723e32d 2026-03-22 16:21:06,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:21:06,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73e00e93e2314723e32d HTTP/1.1" 200 None 2026-03-22 16:21:06,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:21:06,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 16:21:06,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:21:06,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd73e00e93e2314723e32d HTTP/1.1" 200 None 2026-03-22 16:21:06,965 - lava_callback - ERROR - Error processing callback for node 69bd73e00e93e2314723e32d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49430 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:22:32,011 - lava_callback - INFO - Received callback for node 69bd74350e93e2314723e34b INFO: 172.20.0.1:43538 - "POST /node/69bd74350e93e2314723e34b HTTP/1.0" 202 Accepted 2026-03-22 16:22:32,017 - lava_callback - INFO - Starting async job submit for node 69bd74350e93e2314723e34b 2026-03-22 16:22:32,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:22:32,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd74350e93e2314723e34b HTTP/1.1" 200 None 2026-03-22 16:22:32,033 - lava_callback - DEBUG - Processing node 69bd74350e93e2314723e34b: baseline-arm64-clabbe 2026-03-22 16:22:32,033 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd74350e93e2314723e34b 2026-03-22 16:22:32,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:22:32,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd74350e93e2314723e34b HTTP/1.1" 200 None 2026-03-22 16:22:32,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:22:32,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 16:22:32,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:22:32,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd74350e93e2314723e34b HTTP/1.1" 200 None 2026-03-22 16:22:32,099 - lava_callback - ERROR - Error processing callback for node 69bd74350e93e2314723e34b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41632 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:23:52,072 - lava_callback - INFO - Received callback for node 69bd74860e93e2314723e35d INFO: 172.20.0.1:55688 - "POST /node/69bd74860e93e2314723e35d HTTP/1.0" 202 Accepted 2026-03-22 16:23:52,078 - lava_callback - INFO - Starting async job submit for node 69bd74860e93e2314723e35d 2026-03-22 16:23:52,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:23:52,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd74860e93e2314723e35d HTTP/1.1" 200 None 2026-03-22 16:23:52,102 - lava_callback - DEBUG - Processing node 69bd74860e93e2314723e35d: baseline-arm64-clabbe 2026-03-22 16:23:52,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd74860e93e2314723e35d 2026-03-22 16:23:52,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:23:52,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd74860e93e2314723e35d HTTP/1.1" 200 None 2026-03-22 16:23:52,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:23:52,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 16:23:52,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:23:52,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd74860e93e2314723e35d HTTP/1.1" 200 None 2026-03-22 16:23:52,159 - lava_callback - ERROR - Error processing callback for node 69bd74860e93e2314723e35d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:25:12,136 - lava_callback - INFO - Received callback for node 69bd74d90e93e2314723e374 INFO: 172.20.0.1:40044 - "POST /node/69bd74d90e93e2314723e374 HTTP/1.0" 202 Accepted 2026-03-22 16:25:12,142 - lava_callback - INFO - Starting async job submit for node 69bd74d90e93e2314723e374 2026-03-22 16:25:12,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:25:12,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd74d90e93e2314723e374 HTTP/1.1" 200 None 2026-03-22 16:25:12,163 - lava_callback - DEBUG - Processing node 69bd74d90e93e2314723e374: baseline-arm64-clabbe 2026-03-22 16:25:12,163 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd74d90e93e2314723e374 2026-03-22 16:25:12,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:25:12,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd74d90e93e2314723e374 HTTP/1.1" 200 None 2026-03-22 16:25:12,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:25:12,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 16:25:12,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:25:12,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd74d90e93e2314723e374 HTTP/1.1" 200 None 2026-03-22 16:25:12,214 - lava_callback - ERROR - Error processing callback for node 69bd74d90e93e2314723e374: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47186 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:26:32,185 - lava_callback - INFO - Received callback for node 69bd75290e93e2314723e39c INFO: 172.20.0.1:48664 - "POST /node/69bd75290e93e2314723e39c HTTP/1.0" 202 Accepted 2026-03-22 16:26:32,191 - lava_callback - INFO - Starting async job submit for node 69bd75290e93e2314723e39c 2026-03-22 16:26:32,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:26:32,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd75290e93e2314723e39c HTTP/1.1" 200 None 2026-03-22 16:26:32,208 - lava_callback - DEBUG - Processing node 69bd75290e93e2314723e39c: baseline-arm64-clabbe 2026-03-22 16:26:32,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd75290e93e2314723e39c 2026-03-22 16:26:32,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:26:32,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd75290e93e2314723e39c HTTP/1.1" 200 None 2026-03-22 16:26:32,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:26:32,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 16:26:32,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:26:32,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd75290e93e2314723e39c HTTP/1.1" 200 None 2026-03-22 16:26:32,266 - lava_callback - ERROR - Error processing callback for node 69bd75290e93e2314723e39c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49500 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:27:52,492 - lava_callback - INFO - Received callback for node 69bd757a0e93e2314723e3bb INFO: 172.20.0.1:34740 - "POST /node/69bd757a0e93e2314723e3bb HTTP/1.0" 202 Accepted 2026-03-22 16:27:52,497 - lava_callback - INFO - Starting async job submit for node 69bd757a0e93e2314723e3bb 2026-03-22 16:27:52,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:27:52,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd757a0e93e2314723e3bb HTTP/1.1" 200 None 2026-03-22 16:27:52,517 - lava_callback - DEBUG - Processing node 69bd757a0e93e2314723e3bb: baseline-arm64-clabbe 2026-03-22 16:27:52,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd757a0e93e2314723e3bb 2026-03-22 16:27:52,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:27:52,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd757a0e93e2314723e3bb HTTP/1.1" 200 None 2026-03-22 16:27:52,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:27:52,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 16:27:52,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:27:52,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd757a0e93e2314723e3bb HTTP/1.1" 200 None 2026-03-22 16:27:52,575 - lava_callback - ERROR - Error processing callback for node 69bd757a0e93e2314723e3bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56294 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:29:12,318 - lava_callback - INFO - Received callback for node 69bd75ca0e93e2314723e3ce INFO: 172.20.0.1:42834 - "POST /node/69bd75ca0e93e2314723e3ce HTTP/1.0" 202 Accepted 2026-03-22 16:29:12,326 - lava_callback - INFO - Starting async job submit for node 69bd75ca0e93e2314723e3ce 2026-03-22 16:29:12,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:29:12,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd75ca0e93e2314723e3ce HTTP/1.1" 200 None 2026-03-22 16:29:12,342 - lava_callback - DEBUG - Processing node 69bd75ca0e93e2314723e3ce: baseline-arm64-clabbe 2026-03-22 16:29:12,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd75ca0e93e2314723e3ce 2026-03-22 16:29:12,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:29:12,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd75ca0e93e2314723e3ce HTTP/1.1" 200 None 2026-03-22 16:29:12,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:29:12,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 16:29:12,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:29:12,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd75ca0e93e2314723e3ce HTTP/1.1" 200 None 2026-03-22 16:29:12,408 - lava_callback - ERROR - Error processing callback for node 69bd75ca0e93e2314723e3ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41870 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:30:47,484 - lava_callback - INFO - Received callback for node 69bd76310e93e2314723e3f1 INFO: 172.20.0.1:41872 - "POST /node/69bd76310e93e2314723e3f1 HTTP/1.0" 202 Accepted 2026-03-22 16:30:47,491 - lava_callback - INFO - Starting async job submit for node 69bd76310e93e2314723e3f1 2026-03-22 16:30:47,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:30:47,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd76310e93e2314723e3f1 HTTP/1.1" 200 None 2026-03-22 16:30:47,512 - lava_callback - DEBUG - Processing node 69bd76310e93e2314723e3f1: baseline-arm-baylibre 2026-03-22 16:30:47,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd76310e93e2314723e3f1 2026-03-22 16:30:47,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:30:47,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd76310e93e2314723e3f1 HTTP/1.1" 200 None 2026-03-22 16:30:47,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:30:47,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:30:47,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:30:47,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd76310e93e2314723e3f1 HTTP/1.1" 200 None 2026-03-22 16:30:47,597 - lava_callback - ERROR - Error processing callback for node 69bd76310e93e2314723e3f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50584 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:31:47,388 - lava_callback - INFO - Received callback for node 69bd766e0e93e2314723e400 INFO: 172.20.0.1:50598 - "POST /node/69bd766e0e93e2314723e400 HTTP/1.0" 202 Accepted 2026-03-22 16:31:47,396 - lava_callback - INFO - Starting async job submit for node 69bd766e0e93e2314723e400 2026-03-22 16:31:47,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:31:47,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd766e0e93e2314723e400 HTTP/1.1" 200 None 2026-03-22 16:31:47,413 - lava_callback - DEBUG - Processing node 69bd766e0e93e2314723e400: baseline-arm-baylibre 2026-03-22 16:31:47,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd766e0e93e2314723e400 2026-03-22 16:31:47,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:31:47,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd766e0e93e2314723e400 HTTP/1.1" 200 None 2026-03-22 16:31:47,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:31:47,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:31:47,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:31:47,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd766e0e93e2314723e400 HTTP/1.1" 200 None 2026-03-22 16:31:47,478 - lava_callback - ERROR - Error processing callback for node 69bd766e0e93e2314723e400: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:32:27,552 - lava_callback - INFO - Received callback for node 69bd768b0e93e2314723e408 INFO: 172.20.0.1:38038 - "POST /node/69bd768b0e93e2314723e408 HTTP/1.0" 202 Accepted 2026-03-22 16:32:27,558 - lava_callback - INFO - Starting async job submit for node 69bd768b0e93e2314723e408 2026-03-22 16:32:27,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:32:27,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd768b0e93e2314723e408 HTTP/1.1" 200 None 2026-03-22 16:32:27,580 - lava_callback - DEBUG - Processing node 69bd768b0e93e2314723e408: baseline-arm-baylibre 2026-03-22 16:32:27,580 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd768b0e93e2314723e408 2026-03-22 16:32:27,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:32:27,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd768b0e93e2314723e408 HTTP/1.1" 200 None 2026-03-22 16:32:27,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:32:27,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:32:27,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:32:27,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd768b0e93e2314723e408 HTTP/1.1" 200 None 2026-03-22 16:32:27,640 - lava_callback - ERROR - Error processing callback for node 69bd768b0e93e2314723e408: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35892 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:32:47,480 - lava_callback - INFO - Received callback for node 69bd76ab0e93e2314723e417 INFO: 172.20.0.1:35898 - "POST /node/69bd76ab0e93e2314723e417 HTTP/1.0" 202 Accepted 2026-03-22 16:32:47,485 - lava_callback - INFO - Starting async job submit for node 69bd76ab0e93e2314723e417 2026-03-22 16:32:47,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:32:47,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd76ab0e93e2314723e417 HTTP/1.1" 200 None 2026-03-22 16:32:47,503 - lava_callback - DEBUG - Processing node 69bd76ab0e93e2314723e417: baseline-arm-baylibre 2026-03-22 16:32:47,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd76ab0e93e2314723e417 2026-03-22 16:32:47,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:32:47,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd76ab0e93e2314723e417 HTTP/1.1" 200 None 2026-03-22 16:32:47,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:32:47,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:32:47,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:32:47,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd76ab0e93e2314723e417 HTTP/1.1" 200 None 2026-03-22 16:32:47,576 - lava_callback - ERROR - Error processing callback for node 69bd76ab0e93e2314723e417: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44712 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:33:47,564 - lava_callback - INFO - Received callback for node 69bd76e80e93e2314723e43b INFO: 172.20.0.1:44722 - "POST /node/69bd76e80e93e2314723e43b HTTP/1.0" 202 Accepted 2026-03-22 16:33:47,569 - lava_callback - INFO - Starting async job submit for node 69bd76e80e93e2314723e43b 2026-03-22 16:33:47,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:33:47,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd76e80e93e2314723e43b HTTP/1.1" 200 None 2026-03-22 16:33:47,586 - lava_callback - DEBUG - Processing node 69bd76e80e93e2314723e43b: baseline-arm-baylibre 2026-03-22 16:33:47,586 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd76e80e93e2314723e43b 2026-03-22 16:33:47,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:33:47,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd76e80e93e2314723e43b HTTP/1.1" 200 None 2026-03-22 16:33:47,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:33:47,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:33:47,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:33:47,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd76e80e93e2314723e43b HTTP/1.1" 200 None 2026-03-22 16:33:47,642 - lava_callback - ERROR - Error processing callback for node 69bd76e80e93e2314723e43b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:34:27,540 - lava_callback - INFO - Received callback for node 69bd77040e93e2314723e43e INFO: 172.20.0.1:49220 - "POST /node/69bd77040e93e2314723e43e HTTP/1.0" 202 Accepted 2026-03-22 16:34:27,546 - lava_callback - INFO - Starting async job submit for node 69bd77040e93e2314723e43e 2026-03-22 16:34:27,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:34:27,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77040e93e2314723e43e HTTP/1.1" 200 None 2026-03-22 16:34:27,563 - lava_callback - DEBUG - Processing node 69bd77040e93e2314723e43e: baseline-arm-baylibre 2026-03-22 16:34:27,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd77040e93e2314723e43e 2026-03-22 16:34:27,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:34:27,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77040e93e2314723e43e HTTP/1.1" 200 None 2026-03-22 16:34:27,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:34:27,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:34:27,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:34:27,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd77040e93e2314723e43e HTTP/1.1" 200 None 2026-03-22 16:34:27,624 - lava_callback - ERROR - Error processing callback for node 69bd77040e93e2314723e43e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37744 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:34:52,611 - lava_callback - INFO - Received callback for node 69bd77270e93e2314723e466 INFO: 172.20.0.1:44430 - "POST /node/69bd77270e93e2314723e466 HTTP/1.0" 202 Accepted 2026-03-22 16:34:52,616 - lava_callback - INFO - Starting async job submit for node 69bd77270e93e2314723e466 2026-03-22 16:34:52,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:34:52,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77270e93e2314723e466 HTTP/1.1" 200 None 2026-03-22 16:34:52,633 - lava_callback - DEBUG - Processing node 69bd77270e93e2314723e466: baseline-arm-clabbe 2026-03-22 16:34:52,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd77270e93e2314723e466 2026-03-22 16:34:52,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:34:52,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77270e93e2314723e466 HTTP/1.1" 200 None 2026-03-22 16:34:52,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:34:52,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:34:52,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:34:52,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd77270e93e2314723e466 HTTP/1.1" 200 None 2026-03-22 16:34:52,690 - lava_callback - ERROR - Error processing callback for node 69bd77270e93e2314723e466: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46808 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:35:17,470 - lava_callback - INFO - Received callback for node 69c019c4c3d73288b0d07a59 INFO: 172.20.0.1:46810 - "POST /node/69c019c4c3d73288b0d07a59 HTTP/1.0" 202 Accepted 2026-03-22 16:35:17,476 - lava_callback - INFO - Starting async job submit for node 69c019c4c3d73288b0d07a59 2026-03-22 16:35:17,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:35:17,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c019c4c3d73288b0d07a59 HTTP/1.1" 200 None 2026-03-22 16:35:17,516 - lava_callback - DEBUG - Processing node 69c019c4c3d73288b0d07a59: baseline-arm 2026-03-22 16:35:17,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:35:17,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:35:17,846 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c019c4c3d73288b0d07a59/log.txt.gz 2026-03-22 16:35:17,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:35:18,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:35:18,200 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c019c4c3d73288b0d07a59/lava-logs.yaml 2026-03-22 16:35:18,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:35:18,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:35:18,352 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c019c4c3d73288b0d07a59/lava_callback.json.gz 2026-03-22 16:35:18,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:35:18,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c019c4c3d73288b0d07a59 HTTP/1.1" 200 None 2026-03-22 16:35:18,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:35:18,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01637c3d73288b0d052f6 HTTP/1.1" 200 None 2026-03-22 16:35:18,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:35:18,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c019c4c3d73288b0d07a59 HTTP/1.1" 200 None 2026-03-22 16:35:18,513 - lava_callback - INFO - Completed processing callback for node 69c019c4c3d73288b0d07a59 INFO: 172.20.0.1:51018 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:35:31,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:35:31,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:35:52,611 - lava_callback - INFO - Received callback for node 69bd77630e93e2314723e47f INFO: 172.20.0.1:36462 - "POST /node/69bd77630e93e2314723e47f HTTP/1.0" 202 Accepted 2026-03-22 16:35:52,616 - lava_callback - INFO - Starting async job submit for node 69bd77630e93e2314723e47f 2026-03-22 16:35:52,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:35:52,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77630e93e2314723e47f HTTP/1.1" 200 None 2026-03-22 16:35:52,633 - lava_callback - DEBUG - Processing node 69bd77630e93e2314723e47f: baseline-arm-clabbe 2026-03-22 16:35:52,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd77630e93e2314723e47f 2026-03-22 16:35:52,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:35:52,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77630e93e2314723e47f HTTP/1.1" 200 None 2026-03-22 16:35:52,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:35:52,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:35:52,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:35:52,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd77630e93e2314723e47f HTTP/1.1" 200 None 2026-03-22 16:35:52,722 - lava_callback - ERROR - Error processing callback for node 69bd77630e93e2314723e47f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47832 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:36:32,575 - lava_callback - INFO - Received callback for node 69bd777f0e93e2314723e496 INFO: 172.20.0.1:38194 - "POST /node/69bd777f0e93e2314723e496 HTTP/1.0" 202 Accepted 2026-03-22 16:36:32,584 - lava_callback - INFO - Starting async job submit for node 69bd777f0e93e2314723e496 2026-03-22 16:36:32,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:36:32,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd777f0e93e2314723e496 HTTP/1.1" 200 None 2026-03-22 16:36:32,620 - lava_callback - DEBUG - Processing node 69bd777f0e93e2314723e496: baseline-arm-clabbe 2026-03-22 16:36:32,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd777f0e93e2314723e496 2026-03-22 16:36:32,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:36:32,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd777f0e93e2314723e496 HTTP/1.1" 200 None 2026-03-22 16:36:32,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:36:32,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:36:32,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:36:32,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd777f0e93e2314723e496 HTTP/1.1" 200 None 2026-03-22 16:36:32,733 - lava_callback - ERROR - Error processing callback for node 69bd777f0e93e2314723e496: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45896 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:36:52,630 - lava_callback - INFO - Received callback for node 69bd779f0e93e2314723e4ae INFO: 172.20.0.1:55558 - "POST /node/69bd779f0e93e2314723e4ae HTTP/1.0" 202 Accepted 2026-03-22 16:36:52,635 - lava_callback - INFO - Starting async job submit for node 69bd779f0e93e2314723e4ae 2026-03-22 16:36:52,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:36:52,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd779f0e93e2314723e4ae HTTP/1.1" 200 None 2026-03-22 16:36:52,653 - lava_callback - DEBUG - Processing node 69bd779f0e93e2314723e4ae: baseline-arm-clabbe 2026-03-22 16:36:52,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd779f0e93e2314723e4ae 2026-03-22 16:36:52,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:36:52,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd779f0e93e2314723e4ae HTTP/1.1" 200 None 2026-03-22 16:36:52,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:36:52,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:36:52,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:36:52,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd779f0e93e2314723e4ae HTTP/1.1" 200 None 2026-03-22 16:36:52,708 - lava_callback - ERROR - Error processing callback for node 69bd779f0e93e2314723e4ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44062 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:37:27,711 - lava_callback - INFO - Received callback for node 69bd77b90e93e2314723e4bf INFO: 172.20.0.1:44076 - "POST /node/69bd77b90e93e2314723e4bf HTTP/1.0" 202 Accepted 2026-03-22 16:37:27,717 - lava_callback - INFO - Starting async job submit for node 69bd77b90e93e2314723e4bf 2026-03-22 16:37:27,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:37:27,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77b90e93e2314723e4bf HTTP/1.1" 200 None 2026-03-22 16:37:27,745 - lava_callback - DEBUG - Processing node 69bd77b90e93e2314723e4bf: baseline-arm-baylibre 2026-03-22 16:37:27,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd77b90e93e2314723e4bf 2026-03-22 16:37:27,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:37:27,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77b90e93e2314723e4bf HTTP/1.1" 200 None 2026-03-22 16:37:27,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:37:27,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:37:27,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:37:27,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd77b90e93e2314723e4bf HTTP/1.1" 200 None 2026-03-22 16:37:27,818 - lava_callback - ERROR - Error processing callback for node 69bd77b90e93e2314723e4bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57374 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:37:52,672 - lava_callback - INFO - Received callback for node 69bd77db0e93e2314723e4e1 INFO: 172.20.0.1:38516 - "POST /node/69bd77db0e93e2314723e4e1 HTTP/1.0" 202 Accepted 2026-03-22 16:37:52,681 - lava_callback - INFO - Starting async job submit for node 69bd77db0e93e2314723e4e1 2026-03-22 16:37:52,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:37:52,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77db0e93e2314723e4e1 HTTP/1.1" 200 None 2026-03-22 16:37:52,711 - lava_callback - DEBUG - Processing node 69bd77db0e93e2314723e4e1: baseline-arm-clabbe 2026-03-22 16:37:52,711 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd77db0e93e2314723e4e1 2026-03-22 16:37:52,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:37:52,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77db0e93e2314723e4e1 HTTP/1.1" 200 None 2026-03-22 16:37:52,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:37:52,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-22 16:37:52,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:37:52,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd77db0e93e2314723e4e1 HTTP/1.1" 200 None 2026-03-22 16:37:52,802 - lava_callback - ERROR - Error processing callback for node 69bd77db0e93e2314723e4e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34546 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:38:27,720 - lava_callback - INFO - Received callback for node 69bd77f20e93e2314723e4fd INFO: 172.20.0.1:34562 - "POST /node/69bd77f20e93e2314723e4fd HTTP/1.0" 202 Accepted 2026-03-22 16:38:27,729 - lava_callback - INFO - Starting async job submit for node 69bd77f20e93e2314723e4fd 2026-03-22 16:38:27,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:38:27,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77f20e93e2314723e4fd HTTP/1.1" 200 None 2026-03-22 16:38:27,753 - lava_callback - DEBUG - Processing node 69bd77f20e93e2314723e4fd: baseline-arm-baylibre 2026-03-22 16:38:27,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd77f20e93e2314723e4fd 2026-03-22 16:38:27,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:38:27,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd77f20e93e2314723e4fd HTTP/1.1" 200 None 2026-03-22 16:38:27,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:38:27,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:38:27,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:38:27,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd77f20e93e2314723e4fd HTTP/1.1" 200 None 2026-03-22 16:38:27,834 - lava_callback - ERROR - Error processing callback for node 69bd77f20e93e2314723e4fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46478 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:38:47,930 - lava_callback - INFO - Received callback for node 69bd780e0e93e2314723e525 INFO: 172.20.0.1:46488 - "POST /node/69bd780e0e93e2314723e525 HTTP/1.0" 202 Accepted 2026-03-22 16:38:47,939 - lava_callback - INFO - Starting async job submit for node 69bd780e0e93e2314723e525 2026-03-22 16:38:47,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:38:47,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd780e0e93e2314723e525 HTTP/1.1" 200 None 2026-03-22 16:38:47,963 - lava_callback - DEBUG - Processing node 69bd780e0e93e2314723e525: baseline-arm-baylibre 2026-03-22 16:38:47,964 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd780e0e93e2314723e525 2026-03-22 16:38:47,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:38:47,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd780e0e93e2314723e525 HTTP/1.1" 200 None 2026-03-22 16:38:47,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:38:48,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:38:48,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:38:48,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd780e0e93e2314723e525 HTTP/1.1" 200 None 2026-03-22 16:38:48,065 - lava_callback - ERROR - Error processing callback for node 69bd780e0e93e2314723e525: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53612 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:39:27,780 - lava_callback - INFO - Received callback for node 69bd782c0e93e2314723e53c INFO: 172.20.0.1:53622 - "POST /node/69bd782c0e93e2314723e53c HTTP/1.0" 202 Accepted 2026-03-22 16:39:27,788 - lava_callback - INFO - Starting async job submit for node 69bd782c0e93e2314723e53c 2026-03-22 16:39:27,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:39:27,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd782c0e93e2314723e53c HTTP/1.1" 200 None 2026-03-22 16:39:27,810 - lava_callback - DEBUG - Processing node 69bd782c0e93e2314723e53c: baseline-arm-baylibre 2026-03-22 16:39:27,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd782c0e93e2314723e53c 2026-03-22 16:39:27,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:39:27,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd782c0e93e2314723e53c HTTP/1.1" 200 None 2026-03-22 16:39:27,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:39:27,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:39:27,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:39:27,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd782c0e93e2314723e53c HTTP/1.1" 200 None 2026-03-22 16:39:27,889 - lava_callback - ERROR - Error processing callback for node 69bd782c0e93e2314723e53c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51384 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:39:47,804 - lava_callback - INFO - Received callback for node 69bd784f0e93e2314723e552 INFO: 172.20.0.1:51400 - "POST /node/69bd784f0e93e2314723e552 HTTP/1.0" 202 Accepted 2026-03-22 16:39:47,810 - lava_callback - INFO - Starting async job submit for node 69bd784f0e93e2314723e552 2026-03-22 16:39:47,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:39:47,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd784f0e93e2314723e552 HTTP/1.1" 200 None 2026-03-22 16:39:47,835 - lava_callback - DEBUG - Processing node 69bd784f0e93e2314723e552: baseline-arm-baylibre 2026-03-22 16:39:47,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd784f0e93e2314723e552 2026-03-22 16:39:47,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:39:47,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd784f0e93e2314723e552 HTTP/1.1" 200 None 2026-03-22 16:39:47,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:39:47,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:39:47,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:39:47,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd784f0e93e2314723e552 HTTP/1.1" 200 None 2026-03-22 16:39:47,890 - lava_callback - ERROR - Error processing callback for node 69bd784f0e93e2314723e552: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59952 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:40:27,840 - lava_callback - INFO - Received callback for node 69bd786c0e93e2314723e564 INFO: 172.20.0.1:59964 - "POST /node/69bd786c0e93e2314723e564 HTTP/1.0" 202 Accepted 2026-03-22 16:40:27,845 - lava_callback - INFO - Starting async job submit for node 69bd786c0e93e2314723e564 2026-03-22 16:40:27,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:40:27,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd786c0e93e2314723e564 HTTP/1.1" 200 None 2026-03-22 16:40:27,861 - lava_callback - DEBUG - Processing node 69bd786c0e93e2314723e564: baseline-arm-baylibre 2026-03-22 16:40:27,861 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd786c0e93e2314723e564 2026-03-22 16:40:27,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:40:27,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd786c0e93e2314723e564 HTTP/1.1" 200 None 2026-03-22 16:40:27,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:40:27,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:40:27,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:40:27,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd786c0e93e2314723e564 HTTP/1.1" 200 None 2026-03-22 16:40:27,923 - lava_callback - ERROR - Error processing callback for node 69bd786c0e93e2314723e564: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50468 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:40:47,836 - lava_callback - INFO - Received callback for node 69bd788c0e93e2314723e576 INFO: 172.20.0.1:50474 - "POST /node/69bd788c0e93e2314723e576 HTTP/1.0" 202 Accepted 2026-03-22 16:40:47,841 - lava_callback - INFO - Starting async job submit for node 69bd788c0e93e2314723e576 2026-03-22 16:40:47,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:40:47,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd788c0e93e2314723e576 HTTP/1.1" 200 None 2026-03-22 16:40:47,867 - lava_callback - DEBUG - Processing node 69bd788c0e93e2314723e576: baseline-arm-baylibre 2026-03-22 16:40:47,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd788c0e93e2314723e576 2026-03-22 16:40:47,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:40:47,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd788c0e93e2314723e576 HTTP/1.1" 200 None 2026-03-22 16:40:47,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:40:47,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:40:47,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:40:47,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd788c0e93e2314723e576 HTTP/1.1" 200 None 2026-03-22 16:40:47,918 - lava_callback - ERROR - Error processing callback for node 69bd788c0e93e2314723e576: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48514 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:41:27,889 - lava_callback - INFO - Received callback for node 69bd78b20e93e2314723e585 INFO: 172.20.0.1:48518 - "POST /node/69bd78b20e93e2314723e585 HTTP/1.0" 202 Accepted 2026-03-22 16:41:27,895 - lava_callback - INFO - Starting async job submit for node 69bd78b20e93e2314723e585 2026-03-22 16:41:27,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:41:27,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd78b20e93e2314723e585 HTTP/1.1" 200 None 2026-03-22 16:41:27,911 - lava_callback - DEBUG - Processing node 69bd78b20e93e2314723e585: baseline-arm-baylibre 2026-03-22 16:41:27,911 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd78b20e93e2314723e585 2026-03-22 16:41:27,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:41:27,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd78b20e93e2314723e585 HTTP/1.1" 200 None 2026-03-22 16:41:27,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:41:27,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:41:27,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:41:27,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd78b20e93e2314723e585 HTTP/1.1" 200 None 2026-03-22 16:41:27,963 - lava_callback - ERROR - Error processing callback for node 69bd78b20e93e2314723e585: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35416 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:42:12,856 - lava_callback - INFO - Received callback for node 69bd78d10e93e2314723e593 INFO: 172.20.0.1:43750 - "POST /node/69bd78d10e93e2314723e593 HTTP/1.0" 202 Accepted 2026-03-22 16:42:12,862 - lava_callback - INFO - Starting async job submit for node 69bd78d10e93e2314723e593 2026-03-22 16:42:12,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:12,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd78d10e93e2314723e593 HTTP/1.1" 200 None 2026-03-22 16:42:12,877 - lava_callback - DEBUG - Processing node 69bd78d10e93e2314723e593: baseline-arm-clabbe 2026-03-22 16:42:12,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd78d10e93e2314723e593 2026-03-22 16:42:12,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:12,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd78d10e93e2314723e593 HTTP/1.1" 200 None 2026-03-22 16:42:12,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:12,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:42:12,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:12,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd78d10e93e2314723e593 HTTP/1.1" 200 None 2026-03-22 16:42:12,929 - lava_callback - ERROR - Error processing callback for node 69bd78d10e93e2314723e593: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43730 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:42:32,799 - lava_callback - INFO - Received callback for node 69bd78ee0e93e2314723e5a4 INFO: 172.20.0.1:48510 - "POST /node/69bd78ee0e93e2314723e5a4 HTTP/1.0" 202 Accepted 2026-03-22 16:42:32,805 - lava_callback - INFO - Starting async job submit for node 69bd78ee0e93e2314723e5a4 2026-03-22 16:42:32,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:32,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd78ee0e93e2314723e5a4 HTTP/1.1" 200 None 2026-03-22 16:42:32,821 - lava_callback - DEBUG - Processing node 69bd78ee0e93e2314723e5a4: baseline-arm-clabbe 2026-03-22 16:42:32,821 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd78ee0e93e2314723e5a4 2026-03-22 16:42:32,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:32,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd78ee0e93e2314723e5a4 HTTP/1.1" 200 None 2026-03-22 16:42:32,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:32,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:42:32,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:32,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd78ee0e93e2314723e5a4 HTTP/1.1" 200 None 2026-03-22 16:42:32,876 - lava_callback - ERROR - Error processing callback for node 69bd78ee0e93e2314723e5a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39242 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:42:52,806 - lava_callback - INFO - Received callback for node 69bd79080e93e2314723e5b8 INFO: 172.20.0.1:60624 - "POST /node/69bd79080e93e2314723e5b8 HTTP/1.0" 202 Accepted 2026-03-22 16:42:52,812 - lava_callback - INFO - Starting async job submit for node 69bd79080e93e2314723e5b8 2026-03-22 16:42:52,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:52,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79080e93e2314723e5b8 HTTP/1.1" 200 None 2026-03-22 16:42:52,827 - lava_callback - DEBUG - Processing node 69bd79080e93e2314723e5b8: baseline-arm-clabbe 2026-03-22 16:42:52,827 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd79080e93e2314723e5b8 2026-03-22 16:42:52,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:52,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79080e93e2314723e5b8 HTTP/1.1" 200 None 2026-03-22 16:42:52,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:52,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:42:52,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:42:52,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd79080e93e2314723e5b8 HTTP/1.1" 200 None 2026-03-22 16:42:52,899 - lava_callback - ERROR - Error processing callback for node 69bd79080e93e2314723e5b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48688 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:43:32,871 - lava_callback - INFO - Received callback for node 69bd79210e93e2314723e5ce INFO: 172.20.0.1:37890 - "POST /node/69bd79210e93e2314723e5ce HTTP/1.0" 202 Accepted 2026-03-22 16:43:32,876 - lava_callback - INFO - Starting async job submit for node 69bd79210e93e2314723e5ce 2026-03-22 16:43:32,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:32,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79210e93e2314723e5ce HTTP/1.1" 200 None 2026-03-22 16:43:32,897 - lava_callback - DEBUG - Processing node 69bd79210e93e2314723e5ce: baseline-arm-clabbe 2026-03-22 16:43:32,897 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd79210e93e2314723e5ce 2026-03-22 16:43:32,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:32,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79210e93e2314723e5ce HTTP/1.1" 200 None 2026-03-22 16:43:32,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:32,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:43:32,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:32,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd79210e93e2314723e5ce HTTP/1.1" 200 None 2026-03-22 16:43:32,953 - lava_callback - ERROR - Error processing callback for node 69bd79210e93e2314723e5ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 16:43:34,221 - lava_callback - INFO - Received callback for node 69c01c1ac3d73288b0d07aec INFO: 172.20.0.1:37900 - "POST /node/69c01c1ac3d73288b0d07aec HTTP/1.0" 202 Accepted 2026-03-22 16:43:34,227 - lava_callback - INFO - Starting async job submit for node 69c01c1ac3d73288b0d07aec 2026-03-22 16:43:34,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:34,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c1ac3d73288b0d07aec HTTP/1.1" 200 None 2026-03-22 16:43:34,262 - lava_callback - DEBUG - Processing node 69c01c1ac3d73288b0d07aec: baseline-x86 2026-03-22 16:43:34,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:43:34,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:43:34,851 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1ac3d73288b0d07aec/log.txt.gz 2026-03-22 16:43:34,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:43:35,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:43:35,441 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1ac3d73288b0d07aec/lava-logs.yaml 2026-03-22 16:43:35,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:43:35,582 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:43:35,583 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1ac3d73288b0d07aec/lava_callback.json.gz 2026-03-22 16:43:35,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:35,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c1ac3d73288b0d07aec HTTP/1.1" 200 None 2026-03-22 16:43:35,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:35,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0164fc3d73288b0d05318 HTTP/1.1" 200 None 2026-03-22 16:43:35,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:35,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c1ac3d73288b0d07aec HTTP/1.1" 200 None 2026-03-22 16:43:35,667 - lava_callback - INFO - Completed processing callback for node 69c01c1ac3d73288b0d07aec INFO: 172.20.0.1:37914 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:43:39,870 - lava_callback - INFO - Received callback for node 69c01c1cc3d73288b0d07af5 INFO: 172.20.0.1:47786 - "POST /node/69c01c1cc3d73288b0d07af5 HTTP/1.0" 202 Accepted 2026-03-22 16:43:39,879 - lava_callback - INFO - Starting async job submit for node 69c01c1cc3d73288b0d07af5 2026-03-22 16:43:39,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:39,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c1cc3d73288b0d07af5 HTTP/1.1" 200 None 2026-03-22 16:43:39,950 - lava_callback - DEBUG - Processing node 69c01c1cc3d73288b0d07af5: baseline-x86 INFO: 172.20.0.1:47790 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:43:40,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:43:40,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:43:40,322 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1cc3d73288b0d07af5/log.txt.gz 2026-03-22 16:43:40,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:43:40,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:43:40,859 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1cc3d73288b0d07af5/lava-logs.yaml 2026-03-22 16:43:40,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:43:41,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:43:41,017 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1cc3d73288b0d07af5/lava_callback.json.gz 2026-03-22 16:43:41,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:41,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c1cc3d73288b0d07af5 HTTP/1.1" 200 None 2026-03-22 16:43:41,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:41,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0164fc3d73288b0d05318 HTTP/1.1" 200 None 2026-03-22 16:43:41,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:41,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c1cc3d73288b0d07af5 HTTP/1.1" 200 None 2026-03-22 16:43:41,139 - lava_callback - INFO - Completed processing callback for node 69c01c1cc3d73288b0d07af5 INFO: 172.20.0.1:47792 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:43:53,037 - lava_callback - INFO - Received callback for node 69bd793c0e93e2314723e5d3 INFO: 172.20.0.1:57374 - "POST /node/69bd793c0e93e2314723e5d3 HTTP/1.0" 202 Accepted 2026-03-22 16:43:53,046 - lava_callback - INFO - Starting async job submit for node 69bd793c0e93e2314723e5d3 2026-03-22 16:43:53,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:53,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd793c0e93e2314723e5d3 HTTP/1.1" 200 None 2026-03-22 16:43:53,077 - lava_callback - DEBUG - Processing node 69bd793c0e93e2314723e5d3: baseline-arm-clabbe 2026-03-22 16:43:53,077 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd793c0e93e2314723e5d3 2026-03-22 16:43:53,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:53,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd793c0e93e2314723e5d3 HTTP/1.1" 200 None 2026-03-22 16:43:53,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:53,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 16:43:53,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:43:53,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd793c0e93e2314723e5d3 HTTP/1.1" 200 None 2026-03-22 16:43:53,169 - lava_callback - ERROR - Error processing callback for node 69bd793c0e93e2314723e5d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57376 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:44:01,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:02,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43080 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:44:08,752 - lava_callback - INFO - Received callback for node 69bd79540e93e2314723e5e7 INFO: 172.20.0.1:43090 - "POST /node/69bd79540e93e2314723e5e7 HTTP/1.0" 202 Accepted 2026-03-22 16:44:08,758 - lava_callback - INFO - Starting async job submit for node 69bd79540e93e2314723e5e7 2026-03-22 16:44:08,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:08,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79540e93e2314723e5e7 HTTP/1.1" 200 None 2026-03-22 16:44:08,774 - lava_callback - DEBUG - Processing node 69bd79540e93e2314723e5e7: baseline-arm-baylibre 2026-03-22 16:44:08,774 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd79540e93e2314723e5e7 2026-03-22 16:44:08,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:08,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79540e93e2314723e5e7 HTTP/1.1" 200 None 2026-03-22 16:44:08,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:08,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:44:08,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:08,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd79540e93e2314723e5e7 HTTP/1.1" 200 None 2026-03-22 16:44:08,843 - lava_callback - ERROR - Error processing callback for node 69bd79540e93e2314723e5e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53666 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:44:25,940 - lava_callback - INFO - Received callback for node 69c01c1bc3d73288b0d07aef INFO: 172.20.0.1:47136 - "POST /node/69c01c1bc3d73288b0d07aef HTTP/1.0" 202 Accepted 2026-03-22 16:44:25,947 - lava_callback - INFO - Starting async job submit for node 69c01c1bc3d73288b0d07aef 2026-03-22 16:44:25,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:26,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c1bc3d73288b0d07aef HTTP/1.1" 200 None 2026-03-22 16:44:26,003 - lava_callback - DEBUG - Processing node 69c01c1bc3d73288b0d07aef: baseline-x86 2026-03-22 16:44:26,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:47152 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:44:26,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:44:26,627 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1bc3d73288b0d07aef/log.txt.gz 2026-03-22 16:44:26,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:44:27,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:44:27,242 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1bc3d73288b0d07aef/lava-logs.yaml 2026-03-22 16:44:27,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:44:27,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:44:27,369 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1bc3d73288b0d07aef/lava_callback.json.gz 2026-03-22 16:44:27,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:27,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c1bc3d73288b0d07aef HTTP/1.1" 200 None 2026-03-22 16:44:27,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:27,415 - lava_callback - INFO - Received callback for node 69c01c1bc3d73288b0d07af2 2026-03-22 16:44:27,423 - lava_callback - INFO - Starting async job submit for node 69c01c1bc3d73288b0d07af2 INFO: 172.20.0.1:47158 - "POST /node/69c01c1bc3d73288b0d07af2 HTTP/1.0" 202 Accepted 2026-03-22 16:44:27,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:27,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0164fc3d73288b0d05318 HTTP/1.1" 200 None 2026-03-22 16:44:27,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:27,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c1bc3d73288b0d07af2 HTTP/1.1" 200 None 2026-03-22 16:44:27,482 - lava_callback - DEBUG - Processing node 69c01c1bc3d73288b0d07af2: baseline-x86 2026-03-22 16:44:27,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c1bc3d73288b0d07aef HTTP/1.1" 200 None 2026-03-22 16:44:27,560 - lava_callback - INFO - Completed processing callback for node 69c01c1bc3d73288b0d07aef 2026-03-22 16:44:28,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:44:28,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:44:28,179 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1bc3d73288b0d07af2/log.txt.gz 2026-03-22 16:44:28,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:44:28,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:44:28,899 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1bc3d73288b0d07af2/lava-logs.yaml 2026-03-22 16:44:28,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:44:29,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:44:29,045 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1bc3d73288b0d07af2/lava_callback.json.gz 2026-03-22 16:44:29,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:29,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c1bc3d73288b0d07af2 HTTP/1.1" 200 None 2026-03-22 16:44:29,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:29,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0164fc3d73288b0d05318 HTTP/1.1" 200 None 2026-03-22 16:44:29,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:29,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c1bc3d73288b0d07af2 HTTP/1.1" 200 None 2026-03-22 16:44:29,158 - lava_callback - INFO - Completed processing callback for node 69c01c1bc3d73288b0d07af2 2026-03-22 16:44:29,945 - lava_callback - INFO - Received callback for node 69c01c1dc3d73288b0d07af8 INFO: 172.20.0.1:55166 - "POST /node/69c01c1dc3d73288b0d07af8 HTTP/1.0" 202 Accepted 2026-03-22 16:44:29,954 - lava_callback - INFO - Starting async job submit for node 69c01c1dc3d73288b0d07af8 2026-03-22 16:44:29,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:30,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c1dc3d73288b0d07af8 HTTP/1.1" 200 None 2026-03-22 16:44:30,023 - lava_callback - DEBUG - Processing node 69c01c1dc3d73288b0d07af8: baseline-x86 2026-03-22 16:44:30,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:44:30,662 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:44:30,664 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1dc3d73288b0d07af8/log.txt.gz 2026-03-22 16:44:30,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:44:31,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:44:31,254 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1dc3d73288b0d07af8/lava-logs.yaml 2026-03-22 16:44:31,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:44:31,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:44:31,407 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c01c1dc3d73288b0d07af8/lava_callback.json.gz 2026-03-22 16:44:31,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:31,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c1dc3d73288b0d07af8 HTTP/1.1" 200 None 2026-03-22 16:44:31,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:31,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0164fc3d73288b0d05318 HTTP/1.1" 200 None 2026-03-22 16:44:31,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:31,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c1dc3d73288b0d07af8 HTTP/1.1" 200 None 2026-03-22 16:44:31,525 - lava_callback - INFO - Completed processing callback for node 69c01c1dc3d73288b0d07af8 2026-03-22 16:44:32,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:32,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53792 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:44:48,048 - lava_callback - INFO - Received callback for node 69bd79700e93e2314723e5f8 INFO: 172.20.0.1:53802 - "POST /node/69bd79700e93e2314723e5f8 HTTP/1.0" 202 Accepted 2026-03-22 16:44:48,054 - lava_callback - INFO - Starting async job submit for node 69bd79700e93e2314723e5f8 2026-03-22 16:44:48,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:48,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79700e93e2314723e5f8 HTTP/1.1" 200 None 2026-03-22 16:44:48,072 - lava_callback - DEBUG - Processing node 69bd79700e93e2314723e5f8: baseline-arm-baylibre 2026-03-22 16:44:48,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd79700e93e2314723e5f8 2026-03-22 16:44:48,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:48,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79700e93e2314723e5f8 HTTP/1.1" 200 None 2026-03-22 16:44:48,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:48,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:44:48,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:44:48,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd79700e93e2314723e5f8 HTTP/1.1" 200 None 2026-03-22 16:44:48,144 - lava_callback - ERROR - Error processing callback for node 69bd79700e93e2314723e5f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42568 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:45:01,812 - lava_callback - INFO - Received callback for node 69c01c87c3d73288b0d07b50 INFO: 172.20.0.1:49824 - "POST /node/69c01c87c3d73288b0d07b50 HTTP/1.0" 202 Accepted 2026-03-22 16:45:01,818 - lava_callback - INFO - Starting async job submit for node 69c01c87c3d73288b0d07b50 2026-03-22 16:45:01,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:01,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c87c3d73288b0d07b50 HTTP/1.1" 200 None 2026-03-22 16:45:01,838 - lava_callback - DEBUG - Processing node 69c01c87c3d73288b0d07b50: baseline-arm 2026-03-22 16:45:01,843 - lava_callback - WARNING - Failed to upload log 2026-03-22 16:45:01,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:45:01,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:45:01,991 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c87c3d73288b0d07b50/lava-logs.yaml 2026-03-22 16:45:01,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:45:02,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:45:02,116 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c87c3d73288b0d07b50/lava_callback.json.gz 2026-03-22 16:45:02,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:02,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c87c3d73288b0d07b50 HTTP/1.1" 200 None 2026-03-22 16:45:02,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:02,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:45:02,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:02,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c87c3d73288b0d07b50 HTTP/1.1" 200 None 2026-03-22 16:45:02,231 - lava_callback - INFO - Completed processing callback for node 69c01c87c3d73288b0d07b50 INFO: 172.20.0.1:49826 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:45:08,028 - lava_callback - INFO - Received callback for node 69bd798d0e93e2314723e608 INFO: 172.20.0.1:49832 - "POST /node/69bd798d0e93e2314723e608 HTTP/1.0" 202 Accepted 2026-03-22 16:45:08,034 - lava_callback - INFO - Starting async job submit for node 69bd798d0e93e2314723e608 2026-03-22 16:45:08,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:08,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd798d0e93e2314723e608 HTTP/1.1" 200 None 2026-03-22 16:45:08,050 - lava_callback - DEBUG - Processing node 69bd798d0e93e2314723e608: baseline-arm-baylibre 2026-03-22 16:45:08,050 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd798d0e93e2314723e608 2026-03-22 16:45:08,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:08,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd798d0e93e2314723e608 HTTP/1.1" 200 None 2026-03-22 16:45:08,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:08,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:45:08,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:08,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd798d0e93e2314723e608 HTTP/1.1" 200 None 2026-03-22 16:45:08,113 - lava_callback - ERROR - Error processing callback for node 69bd798d0e93e2314723e608: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50490 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:45:32,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:32,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33928 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:45:46,907 - lava_callback - INFO - Received callback for node 69c01c88c3d73288b0d07b53 INFO: 172.20.0.1:33934 - "POST /node/69c01c88c3d73288b0d07b53 HTTP/1.0" 202 Accepted 2026-03-22 16:45:46,953 - lava_callback - INFO - Starting async job submit for node 69c01c88c3d73288b0d07b53 2026-03-22 16:45:46,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:46,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c88c3d73288b0d07b53 HTTP/1.1" 200 None 2026-03-22 16:45:46,986 - lava_callback - DEBUG - Processing node 69c01c88c3d73288b0d07b53: baseline-arm 2026-03-22 16:45:47,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:45:47,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:45:47,230 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c88c3d73288b0d07b53/log.txt.gz 2026-03-22 16:45:47,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:45:47,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:45:47,497 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c88c3d73288b0d07b53/lava-logs.yaml 2026-03-22 16:45:47,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:45:47,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:45:47,668 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c88c3d73288b0d07b53/lava_callback.json.gz 2026-03-22 16:45:47,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:47,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c88c3d73288b0d07b53 HTTP/1.1" 200 None 2026-03-22 16:45:47,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:47,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:45:47,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:47,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c88c3d73288b0d07b53 HTTP/1.1" 200 None 2026-03-22 16:45:47,752 - lava_callback - INFO - Completed processing callback for node 69c01c88c3d73288b0d07b53 2026-03-22 16:45:48,144 - lava_callback - INFO - Received callback for node 69bd79aa0e93e2314723e61c INFO: 172.20.0.1:33944 - "POST /node/69bd79aa0e93e2314723e61c HTTP/1.0" 202 Accepted 2026-03-22 16:45:48,150 - lava_callback - INFO - Starting async job submit for node 69bd79aa0e93e2314723e61c 2026-03-22 16:45:48,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:48,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79aa0e93e2314723e61c HTTP/1.1" 200 None 2026-03-22 16:45:48,170 - lava_callback - DEBUG - Processing node 69bd79aa0e93e2314723e61c: baseline-arm-baylibre 2026-03-22 16:45:48,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd79aa0e93e2314723e61c 2026-03-22 16:45:48,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:48,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79aa0e93e2314723e61c HTTP/1.1" 200 None 2026-03-22 16:45:48,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:48,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:45:48,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:48,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd79aa0e93e2314723e61c HTTP/1.1" 200 None 2026-03-22 16:45:48,231 - lava_callback - ERROR - Error processing callback for node 69bd79aa0e93e2314723e61c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 16:45:51,690 - lava_callback - INFO - Received callback for node 69c01c8ac3d73288b0d07b5c INFO: 172.20.0.1:59188 - "POST /node/69c01c8ac3d73288b0d07b5c HTTP/1.0" 202 Accepted 2026-03-22 16:45:51,737 - lava_callback - INFO - Starting async job submit for node 69c01c8ac3d73288b0d07b5c 2026-03-22 16:45:51,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:51,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c8ac3d73288b0d07b5c HTTP/1.1" 200 None 2026-03-22 16:45:51,774 - lava_callback - DEBUG - Processing node 69c01c8ac3d73288b0d07b5c: baseline-arm 2026-03-22 16:45:51,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:45:52,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:45:52,055 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c8ac3d73288b0d07b5c/log.txt.gz 2026-03-22 16:45:52,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:45:52,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:45:52,415 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c8ac3d73288b0d07b5c/lava-logs.yaml 2026-03-22 16:45:52,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:45:52,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:45:52,544 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c8ac3d73288b0d07b5c/lava_callback.json.gz 2026-03-22 16:45:52,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:52,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c8ac3d73288b0d07b5c HTTP/1.1" 200 None 2026-03-22 16:45:52,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:52,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:45:52,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:45:52,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c8ac3d73288b0d07b5c HTTP/1.1" 200 None 2026-03-22 16:45:52,619 - lava_callback - INFO - Completed processing callback for node 69c01c8ac3d73288b0d07b5c INFO: 172.20.0.1:59194 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:46:02,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:02,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-22 16:46:05,248 - lava_callback - INFO - Received callback for node 69c01c89c3d73288b0d07b59 INFO: 172.20.0.1:45562 - "POST /node/69c01c89c3d73288b0d07b59 HTTP/1.0" 202 Accepted 2026-03-22 16:46:05,254 - lava_callback - INFO - Starting async job submit for node 69c01c89c3d73288b0d07b59 2026-03-22 16:46:05,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:05,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c89c3d73288b0d07b59 HTTP/1.1" 200 None 2026-03-22 16:46:05,293 - lava_callback - DEBUG - Processing node 69c01c89c3d73288b0d07b59: baseline-arm 2026-03-22 16:46:05,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:05,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:05,705 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c89c3d73288b0d07b59/log.txt.gz 2026-03-22 16:46:05,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:06,049 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:06,051 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c89c3d73288b0d07b59/lava-logs.yaml 2026-03-22 16:46:06,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:06,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:06,230 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c89c3d73288b0d07b59/lava_callback.json.gz 2026-03-22 16:46:06,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:06,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c89c3d73288b0d07b59 HTTP/1.1" 200 None 2026-03-22 16:46:06,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:06,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:46:06,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:06,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c89c3d73288b0d07b59 HTTP/1.1" 200 None 2026-03-22 16:46:06,314 - lava_callback - INFO - Completed processing callback for node 69c01c89c3d73288b0d07b59 INFO: 172.20.0.1:45574 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:46:12,865 - lava_callback - INFO - Received callback for node 69bd79c70e93e2314723e62d INFO: 172.20.0.1:33822 - "POST /node/69bd79c70e93e2314723e62d HTTP/1.0" 202 Accepted 2026-03-22 16:46:12,873 - lava_callback - INFO - Starting async job submit for node 69bd79c70e93e2314723e62d 2026-03-22 16:46:12,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:12,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79c70e93e2314723e62d HTTP/1.1" 200 None 2026-03-22 16:46:12,894 - lava_callback - DEBUG - Processing node 69bd79c70e93e2314723e62d: baseline-arm-clabbe 2026-03-22 16:46:12,895 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd79c70e93e2314723e62d 2026-03-22 16:46:12,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:12,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79c70e93e2314723e62d HTTP/1.1" 200 None 2026-03-22 16:46:12,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:12,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:46:12,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:12,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd79c70e93e2314723e62d HTTP/1.1" 200 None 2026-03-22 16:46:12,962 - lava_callback - ERROR - Error processing callback for node 69bd79c70e93e2314723e62d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33842 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:46:17,540 - lava_callback - INFO - Received callback for node 69c01c86c3d73288b0d07b4d INFO: 172.20.0.1:33858 - "POST /node/69c01c86c3d73288b0d07b4d HTTP/1.0" 202 Accepted 2026-03-22 16:46:17,546 - lava_callback - INFO - Starting async job submit for node 69c01c86c3d73288b0d07b4d 2026-03-22 16:46:17,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:17,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c86c3d73288b0d07b4d HTTP/1.1" 200 None 2026-03-22 16:46:17,583 - lava_callback - DEBUG - Processing node 69c01c86c3d73288b0d07b4d: baseline-arm 2026-03-22 16:46:17,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:17,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:17,899 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c86c3d73288b0d07b4d/log.txt.gz 2026-03-22 16:46:18,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:18,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:18,224 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c86c3d73288b0d07b4d/lava-logs.yaml 2026-03-22 16:46:18,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:18,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:18,376 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c86c3d73288b0d07b4d/lava_callback.json.gz 2026-03-22 16:46:18,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:18,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c86c3d73288b0d07b4d HTTP/1.1" 200 None 2026-03-22 16:46:18,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:18,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:46:18,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:18,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c86c3d73288b0d07b4d HTTP/1.1" 200 None 2026-03-22 16:46:18,460 - lava_callback - INFO - Completed processing callback for node 69c01c86c3d73288b0d07b4d 2026-03-22 16:46:19,244 - lava_callback - INFO - Received callback for node 69c01ca9c3d73288b0d07bcd INFO: 172.20.0.1:33872 - "POST /node/69c01ca9c3d73288b0d07bcd HTTP/1.0" 202 Accepted 2026-03-22 16:46:19,250 - lava_callback - INFO - Starting async job submit for node 69c01ca9c3d73288b0d07bcd 2026-03-22 16:46:19,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:19,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca9c3d73288b0d07bcd HTTP/1.1" 200 None 2026-03-22 16:46:19,283 - lava_callback - DEBUG - Processing node 69c01ca9c3d73288b0d07bcd: baseline-arm-pengutronix 2026-03-22 16:46:19,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:19,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:19,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01ca9c3d73288b0d07bcd/log.txt.gz 2026-03-22 16:46:19,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:19,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:19,929 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01ca9c3d73288b0d07bcd/lava-logs.yaml 2026-03-22 16:46:19,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:20,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:20,099 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01ca9c3d73288b0d07bcd/lava_callback.json.gz 2026-03-22 16:46:20,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:20,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca9c3d73288b0d07bcd HTTP/1.1" 200 None 2026-03-22 16:46:20,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:20,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:46:20,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:20,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ca9c3d73288b0d07bcd HTTP/1.1" 200 None 2026-03-22 16:46:20,187 - lava_callback - INFO - Completed processing callback for node 69c01ca9c3d73288b0d07bcd 2026-03-22 16:46:20,541 - lava_callback - INFO - Received callback for node 69c01ca8c3d73288b0d07bca INFO: 172.20.0.1:60646 - "POST /node/69c01ca8c3d73288b0d07bca HTTP/1.0" 202 Accepted 2026-03-22 16:46:20,547 - lava_callback - INFO - Starting async job submit for node 69c01ca8c3d73288b0d07bca 2026-03-22 16:46:20,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:20,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca8c3d73288b0d07bca HTTP/1.1" 200 None 2026-03-22 16:46:20,591 - lava_callback - DEBUG - Processing node 69c01ca8c3d73288b0d07bca: baseline-arm-pengutronix 2026-03-22 16:46:20,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:20,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:20,871 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01ca8c3d73288b0d07bca/log.txt.gz 2026-03-22 16:46:20,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:21,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:21,140 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01ca8c3d73288b0d07bca/lava-logs.yaml 2026-03-22 16:46:21,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:21,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:21,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01ca8c3d73288b0d07bca/lava_callback.json.gz 2026-03-22 16:46:21,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:21,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca8c3d73288b0d07bca HTTP/1.1" 200 None 2026-03-22 16:46:21,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:21,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:46:21,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:21,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ca8c3d73288b0d07bca HTTP/1.1" 200 None 2026-03-22 16:46:21,373 - lava_callback - INFO - Completed processing callback for node 69c01ca8c3d73288b0d07bca INFO: 172.20.0.1:60650 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:46:32,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:32,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-22 16:46:32,911 - lava_callback - INFO - Received callback for node 69c01cacc3d73288b0d07bd4 INFO: 172.20.0.1:48726 - "POST /node/69c01cacc3d73288b0d07bd4 HTTP/1.0" 202 Accepted 2026-03-22 16:46:32,957 - lava_callback - INFO - Starting async job submit for node 69c01cacc3d73288b0d07bd4 2026-03-22 16:46:32,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:32,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cacc3d73288b0d07bd4 HTTP/1.1" 200 None 2026-03-22 16:46:32,997 - lava_callback - DEBUG - Processing node 69c01cacc3d73288b0d07bd4: baseline-arm-pengutronix 2026-03-22 16:46:33,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:33,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:33,259 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01cacc3d73288b0d07bd4/log.txt.gz 2026-03-22 16:46:33,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:33,486 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:33,488 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01cacc3d73288b0d07bd4/lava-logs.yaml 2026-03-22 16:46:33,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:33,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:33,622 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01cacc3d73288b0d07bd4/lava_callback.json.gz 2026-03-22 16:46:33,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:33,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cacc3d73288b0d07bd4 HTTP/1.1" 200 None 2026-03-22 16:46:33,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:33,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:46:33,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:33,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cacc3d73288b0d07bd4 HTTP/1.1" 200 None 2026-03-22 16:46:33,719 - lava_callback - INFO - Completed processing callback for node 69c01cacc3d73288b0d07bd4 2026-03-22 16:46:34,665 - lava_callback - INFO - Received callback for node 69bd79e40e93e2314723e63c INFO: 172.20.0.1:48734 - "POST /node/69bd79e40e93e2314723e63c HTTP/1.0" 202 Accepted 2026-03-22 16:46:34,671 - lava_callback - INFO - Starting async job submit for node 69bd79e40e93e2314723e63c 2026-03-22 16:46:34,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:34,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79e40e93e2314723e63c HTTP/1.1" 200 None 2026-03-22 16:46:34,717 - lava_callback - DEBUG - Processing node 69bd79e40e93e2314723e63c: baseline-arm-clabbe 2026-03-22 16:46:34,717 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd79e40e93e2314723e63c 2026-03-22 16:46:34,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:34,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd79e40e93e2314723e63c HTTP/1.1" 200 None 2026-03-22 16:46:34,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:34,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:46:34,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:34,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd79e40e93e2314723e63c HTTP/1.1" 200 None 2026-03-22 16:46:34,774 - lava_callback - ERROR - Error processing callback for node 69bd79e40e93e2314723e63c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35418 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:46:49,040 - lava_callback - INFO - Received callback for node 69c01c9bc3d73288b0d07b99 INFO: 172.20.0.1:35422 - "POST /node/69c01c9bc3d73288b0d07b99 HTTP/1.0" 202 Accepted 2026-03-22 16:46:49,046 - lava_callback - INFO - Starting async job submit for node 69c01c9bc3d73288b0d07b99 2026-03-22 16:46:49,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:49,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9bc3d73288b0d07b99 HTTP/1.1" 200 None 2026-03-22 16:46:49,084 - lava_callback - DEBUG - Processing node 69c01c9bc3d73288b0d07b99: baseline-arm-broonie 2026-03-22 16:46:49,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:49,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:49,451 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c9bc3d73288b0d07b99/log.txt.gz 2026-03-22 16:46:49,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:49,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:49,784 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c9bc3d73288b0d07b99/lava-logs.yaml 2026-03-22 16:46:49,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:46:49,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:46:49,934 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c9bc3d73288b0d07b99/lava_callback.json.gz 2026-03-22 16:46:49,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:49,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9bc3d73288b0d07b99 HTTP/1.1" 200 None 2026-03-22 16:46:49,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:49,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:46:49,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:46:50,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c9bc3d73288b0d07b99 HTTP/1.1" 200 None 2026-03-22 16:46:50,028 - lava_callback - INFO - Completed processing callback for node 69c01c9bc3d73288b0d07b99 INFO: 172.20.0.1:36434 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:47:02,102 - lava_callback - INFO - Received callback for node 69c01c9ec3d73288b0d07ba9 INFO: 172.20.0.1:60254 - "POST /node/69c01c9ec3d73288b0d07ba9 HTTP/1.0" 202 Accepted 2026-03-22 16:47:02,109 - lava_callback - INFO - Starting async job submit for node 69c01c9ec3d73288b0d07ba9 2026-03-22 16:47:02,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:02,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:02,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9ec3d73288b0d07ba9 HTTP/1.1" 200 None 2026-03-22 16:47:02,152 - lava_callback - DEBUG - Processing node 69c01c9ec3d73288b0d07ba9: baseline-arm-clabbe 2026-03-22 16:47:02,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-22 16:47:02,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:02,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:02,503 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01c9ec3d73288b0d07ba9/log.txt.gz 2026-03-22 16:47:02,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:02,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:02,865 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01c9ec3d73288b0d07ba9/lava-logs.yaml 2026-03-22 16:47:02,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:03,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:03,016 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01c9ec3d73288b0d07ba9/lava_callback.json.gz 2026-03-22 16:47:03,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:03,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9ec3d73288b0d07ba9 HTTP/1.1" 200 None 2026-03-22 16:47:03,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:03,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:47:03,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:03,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c9ec3d73288b0d07ba9 HTTP/1.1" 200 None 2026-03-22 16:47:03,110 - lava_callback - INFO - Completed processing callback for node 69c01c9ec3d73288b0d07ba9 INFO: 172.20.0.1:60258 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:47:09,170 - lava_callback - INFO - Received callback for node 69c01c97c3d73288b0d07b8e INFO: 172.20.0.1:60270 - "POST /node/69c01c97c3d73288b0d07b8e HTTP/1.0" 202 Accepted 2026-03-22 16:47:09,176 - lava_callback - INFO - Starting async job submit for node 69c01c97c3d73288b0d07b8e 2026-03-22 16:47:09,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:09,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c97c3d73288b0d07b8e HTTP/1.1" 200 None 2026-03-22 16:47:09,213 - lava_callback - DEBUG - Processing node 69c01c97c3d73288b0d07b8e: baseline-arm-broonie 2026-03-22 16:47:09,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:09,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:09,574 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c97c3d73288b0d07b8e/log.txt.gz 2026-03-22 16:47:09,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:09,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:09,903 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c97c3d73288b0d07b8e/lava-logs.yaml 2026-03-22 16:47:09,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:10,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:10,048 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c97c3d73288b0d07b8e/lava_callback.json.gz 2026-03-22 16:47:10,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:10,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c97c3d73288b0d07b8e HTTP/1.1" 200 None 2026-03-22 16:47:10,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:10,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:47:10,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:10,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c97c3d73288b0d07b8e HTTP/1.1" 200 None 2026-03-22 16:47:10,135 - lava_callback - INFO - Completed processing callback for node 69c01c97c3d73288b0d07b8e 2026-03-22 16:47:13,078 - lava_callback - INFO - Received callback for node 69bd7a020e93e2314723e651 INFO: 172.20.0.1:44508 - "POST /node/69bd7a020e93e2314723e651 HTTP/1.0" 202 Accepted 2026-03-22 16:47:13,087 - lava_callback - INFO - Starting async job submit for node 69bd7a020e93e2314723e651 2026-03-22 16:47:13,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:13,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7a020e93e2314723e651 HTTP/1.1" 200 None 2026-03-22 16:47:13,103 - lava_callback - DEBUG - Processing node 69bd7a020e93e2314723e651: baseline-arm-clabbe 2026-03-22 16:47:13,103 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7a020e93e2314723e651 2026-03-22 16:47:13,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:13,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7a020e93e2314723e651 HTTP/1.1" 200 None 2026-03-22 16:47:13,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:13,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:47:13,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:13,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7a020e93e2314723e651 HTTP/1.1" 200 None 2026-03-22 16:47:13,181 - lava_callback - ERROR - Error processing callback for node 69bd7a020e93e2314723e651: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46592 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:47:32,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:32,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-22 16:47:32,866 - lava_callback - INFO - Received callback for node 69bd7a1f0e93e2314723e663 INFO: 172.20.0.1:53376 - "POST /node/69bd7a1f0e93e2314723e663 HTTP/1.0" 202 Accepted 2026-03-22 16:47:32,872 - lava_callback - INFO - Starting async job submit for node 69bd7a1f0e93e2314723e663 2026-03-22 16:47:32,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:32,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7a1f0e93e2314723e663 HTTP/1.1" 200 None 2026-03-22 16:47:32,888 - lava_callback - DEBUG - Processing node 69bd7a1f0e93e2314723e663: baseline-arm-clabbe 2026-03-22 16:47:32,888 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7a1f0e93e2314723e663 2026-03-22 16:47:32,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:32,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7a1f0e93e2314723e663 HTTP/1.1" 200 None 2026-03-22 16:47:32,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:32,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:47:32,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:32,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7a1f0e93e2314723e663 HTTP/1.1" 200 None 2026-03-22 16:47:32,946 - lava_callback - ERROR - Error processing callback for node 69bd7a1f0e93e2314723e663: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53384 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:47:38,068 - lava_callback - INFO - Received callback for node 69c01c98c3d73288b0d07b91 INFO: 172.20.0.1:53392 - "POST /node/69c01c98c3d73288b0d07b91 HTTP/1.0" 202 Accepted 2026-03-22 16:47:38,074 - lava_callback - INFO - Starting async job submit for node 69c01c98c3d73288b0d07b91 2026-03-22 16:47:38,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:38,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c98c3d73288b0d07b91 HTTP/1.1" 200 None 2026-03-22 16:47:38,111 - lava_callback - DEBUG - Processing node 69c01c98c3d73288b0d07b91: baseline-arm-broonie 2026-03-22 16:47:38,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:38,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:38,468 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c98c3d73288b0d07b91/log.txt.gz 2026-03-22 16:47:38,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:38,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:38,784 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c98c3d73288b0d07b91/lava-logs.yaml 2026-03-22 16:47:38,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:38,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:38,940 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c98c3d73288b0d07b91/lava_callback.json.gz 2026-03-22 16:47:38,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:38,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c98c3d73288b0d07b91 HTTP/1.1" 200 None 2026-03-22 16:47:38,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:38,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:47:38,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:39,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c98c3d73288b0d07b91 HTTP/1.1" 200 None 2026-03-22 16:47:39,037 - lava_callback - INFO - Completed processing callback for node 69c01c98c3d73288b0d07b91 2026-03-22 16:47:39,652 - lava_callback - INFO - Received callback for node 69c01c9ac3d73288b0d07b94 INFO: 172.20.0.1:53398 - "POST /node/69c01c9ac3d73288b0d07b94 HTTP/1.0" 202 Accepted 2026-03-22 16:47:39,659 - lava_callback - INFO - Starting async job submit for node 69c01c9ac3d73288b0d07b94 2026-03-22 16:47:39,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:39,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9ac3d73288b0d07b94 HTTP/1.1" 200 None 2026-03-22 16:47:39,706 - lava_callback - DEBUG - Processing node 69c01c9ac3d73288b0d07b94: baseline-arm-broonie 2026-03-22 16:47:39,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:40,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:40,024 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c9ac3d73288b0d07b94/log.txt.gz 2026-03-22 16:47:40,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33964 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:47:40,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:40,318 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c9ac3d73288b0d07b94/lava-logs.yaml 2026-03-22 16:47:40,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:40,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:40,481 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c01c9ac3d73288b0d07b94/lava_callback.json.gz 2026-03-22 16:47:40,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:40,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9ac3d73288b0d07b94 HTTP/1.1" 200 None 2026-03-22 16:47:40,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:40,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:47:40,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:40,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c9ac3d73288b0d07b94 HTTP/1.1" 200 None 2026-03-22 16:47:40,569 - lava_callback - INFO - Completed processing callback for node 69c01c9ac3d73288b0d07b94 2026-03-22 16:47:43,027 - lava_callback - INFO - Received callback for node 69c01cc3c3d73288b0d07c19 INFO: 172.20.0.1:33978 - "POST /node/69c01cc3c3d73288b0d07c19 HTTP/1.0" 202 Accepted 2026-03-22 16:47:43,034 - lava_callback - INFO - Starting async job submit for node 69c01cc3c3d73288b0d07c19 2026-03-22 16:47:43,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:43,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc3c3d73288b0d07c19 HTTP/1.1" 200 None 2026-03-22 16:47:43,073 - lava_callback - DEBUG - Processing node 69c01cc3c3d73288b0d07c19: ltp-dio 2026-03-22 16:47:43,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:43,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:43,667 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c01cc3c3d73288b0d07c19/log.txt.gz 2026-03-22 16:47:43,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:44,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:44,095 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c01cc3c3d73288b0d07c19/lava-logs.yaml 2026-03-22 16:47:44,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:47:44,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:47:44,254 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c01cc3c3d73288b0d07c19/lava_callback.json.gz 2026-03-22 16:47:44,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:44,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc3c3d73288b0d07c19 HTTP/1.1" 200 None 2026-03-22 16:47:44,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:44,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:47:44,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:47:44,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cc3c3d73288b0d07c19 HTTP/1.1" 200 None 2026-03-22 16:47:44,341 - lava_callback - INFO - Completed processing callback for node 69c01cc3c3d73288b0d07c19 INFO: 172.20.0.1:33992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49562 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:48:02,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:02,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49984 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:48:13,893 - lava_callback - INFO - Received callback for node 69bd7a3c0e93e2314723e676 INFO: 172.20.0.1:57004 - "POST /node/69bd7a3c0e93e2314723e676 HTTP/1.0" 202 Accepted 2026-03-22 16:48:13,899 - lava_callback - INFO - Starting async job submit for node 69bd7a3c0e93e2314723e676 2026-03-22 16:48:13,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:13,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7a3c0e93e2314723e676 HTTP/1.1" 200 None 2026-03-22 16:48:13,914 - lava_callback - DEBUG - Processing node 69bd7a3c0e93e2314723e676: baseline-arm-clabbe 2026-03-22 16:48:13,914 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7a3c0e93e2314723e676 2026-03-22 16:48:13,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:13,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7a3c0e93e2314723e676 HTTP/1.1" 200 None 2026-03-22 16:48:13,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:13,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 16:48:13,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:13,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7a3c0e93e2314723e676 HTTP/1.1" 200 None 2026-03-22 16:48:13,982 - lava_callback - ERROR - Error processing callback for node 69bd7a3c0e93e2314723e676: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46488 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:48:27,640 - lava_callback - INFO - Received callback for node 69c01d50c3d73288b0d07cd0 INFO: 172.20.0.1:46490 - "POST /node/69c01d50c3d73288b0d07cd0 HTTP/1.0" 202 Accepted 2026-03-22 16:48:27,649 - lava_callback - INFO - Starting async job submit for node 69c01d50c3d73288b0d07cd0 2026-03-22 16:48:27,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:27,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01d50c3d73288b0d07cd0 HTTP/1.1" 200 None 2026-03-22 16:48:27,685 - lava_callback - DEBUG - Processing node 69c01d50c3d73288b0d07cd0: baseline-arm 2026-03-22 16:48:27,694 - lava_callback - WARNING - Failed to upload log 2026-03-22 16:48:27,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:48:27,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:48:27,968 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c01d50c3d73288b0d07cd0/lava-logs.yaml 2026-03-22 16:48:27,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:48:28,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:48:28,284 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c01d50c3d73288b0d07cd0/lava_callback.json.gz 2026-03-22 16:48:28,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:28,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01d50c3d73288b0d07cd0 HTTP/1.1" 200 None 2026-03-22 16:48:28,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:28,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:48:28,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:28,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01d50c3d73288b0d07cd0 HTTP/1.1" 200 None 2026-03-22 16:48:28,376 - lava_callback - INFO - Completed processing callback for node 69c01d50c3d73288b0d07cd0 2026-03-22 16:48:32,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:32,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34018 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:48:53,194 - lava_callback - INFO - Received callback for node 69c01c9fc3d73288b0d07bad INFO: 172.20.0.1:55722 - "POST /node/69c01c9fc3d73288b0d07bad HTTP/1.0" 202 Accepted 2026-03-22 16:48:53,204 - lava_callback - INFO - Starting async job submit for node 69c01c9fc3d73288b0d07bad 2026-03-22 16:48:53,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:53,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9fc3d73288b0d07bad HTTP/1.1" 200 None 2026-03-22 16:48:53,273 - lava_callback - DEBUG - Processing node 69c01c9fc3d73288b0d07bad: baseline-arm-clabbe 2026-03-22 16:48:53,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:48:53,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:48:53,990 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01c9fc3d73288b0d07bad/log.txt.gz 2026-03-22 16:48:54,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:48:54,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:48:54,715 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01c9fc3d73288b0d07bad/lava-logs.yaml 2026-03-22 16:48:54,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:48:55,218 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:48:55,221 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01c9fc3d73288b0d07bad/lava_callback.json.gz 2026-03-22 16:48:55,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:55,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c9fc3d73288b0d07bad HTTP/1.1" 200 None 2026-03-22 16:48:55,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:55,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:48:55,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:48:55,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c9fc3d73288b0d07bad HTTP/1.1" 200 None 2026-03-22 16:48:55,438 - lava_callback - INFO - Completed processing callback for node 69c01c9fc3d73288b0d07bad INFO: 172.20.0.1:55732 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:49:02,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:49:02,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38494 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:49:49,319 - lava_callback - INFO - Received callback for node 69c01cbcc3d73288b0d07c06 INFO: 172.20.0.1:38502 - "POST /node/69c01cbcc3d73288b0d07c06 HTTP/1.0" 202 Accepted 2026-03-22 16:49:49,326 - lava_callback - INFO - Starting async job submit for node 69c01cbcc3d73288b0d07c06 2026-03-22 16:49:49,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:49:49,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cbcc3d73288b0d07c06 HTTP/1.1" 200 None 2026-03-22 16:49:49,366 - lava_callback - DEBUG - Processing node 69c01cbcc3d73288b0d07c06: ltp-cap_bounds 2026-03-22 16:49:49,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:49:49,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:49:49,856 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c01cbcc3d73288b0d07c06/log.txt.gz 2026-03-22 16:49:50,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:49:50,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:49:50,264 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c01cbcc3d73288b0d07c06/lava-logs.yaml 2026-03-22 16:49:50,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:49:50,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:49:50,415 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c01cbcc3d73288b0d07c06/lava_callback.json.gz 2026-03-22 16:49:50,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:49:50,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cbcc3d73288b0d07c06 HTTP/1.1" 200 None 2026-03-22 16:49:50,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:49:50,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:49:50,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:49:50,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cbcc3d73288b0d07c06 HTTP/1.1" 200 None 2026-03-22 16:49:50,510 - lava_callback - INFO - Completed processing callback for node 69c01cbcc3d73288b0d07c06 INFO: 172.20.0.1:58246 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:49:58,945 - lava_callback - INFO - Received callback for node 69c01cbfc3d73288b0d07c0c INFO: 172.20.0.1:58248 - "POST /node/69c01cbfc3d73288b0d07c0c HTTP/1.0" 202 Accepted 2026-03-22 16:49:58,958 - lava_callback - INFO - Starting async job submit for node 69c01cbfc3d73288b0d07c0c 2026-03-22 16:49:59,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:49:59,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cbfc3d73288b0d07c0c HTTP/1.1" 200 None 2026-03-22 16:49:59,074 - lava_callback - DEBUG - Processing node 69c01cbfc3d73288b0d07c0c: ltp-controllers 2026-03-22 16:49:59,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:00,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:00,432 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c01cbfc3d73288b0d07c0c/log.txt.gz 2026-03-22 16:50:00,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:01,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:01,485 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c01cbfc3d73288b0d07c0c/lava-logs.yaml 2026-03-22 16:50:01,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:02,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:02,038 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c01cbfc3d73288b0d07c0c/lava_callback.json.gz 2026-03-22 16:50:02,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:02,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cbfc3d73288b0d07c0c HTTP/1.1" 200 None 2026-03-22 16:50:02,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:02,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:50:02,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:02,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cbfc3d73288b0d07c0c HTTP/1.1" 200 None 2026-03-22 16:50:02,239 - lava_callback - INFO - Completed processing callback for node 69c01cbfc3d73288b0d07c0c 2026-03-22 16:50:02,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:02,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53474 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:50:09,580 - lava_callback - INFO - Received callback for node 69c01ccec3d73288b0d07c4c INFO: 172.20.0.1:53478 - "POST /node/69c01ccec3d73288b0d07c4c HTTP/1.0" 202 Accepted 2026-03-22 16:50:09,592 - lava_callback - INFO - Starting async job submit for node 69c01ccec3d73288b0d07c4c 2026-03-22 16:50:09,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:09,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ccec3d73288b0d07c4c HTTP/1.1" 200 None 2026-03-22 16:50:09,663 - lava_callback - DEBUG - Processing node 69c01ccec3d73288b0d07c4c: ltp-syscalls 2026-03-22 16:50:11,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:11,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:11,532 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c01ccec3d73288b0d07c4c/log.txt.gz 2026-03-22 16:50:12,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:12,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:12,880 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c01ccec3d73288b0d07c4c/lava-logs.yaml 2026-03-22 16:50:12,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:13,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:13,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c01ccec3d73288b0d07c4c/lava_callback.json.gz 2026-03-22 16:50:13,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:13,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ccec3d73288b0d07c4c HTTP/1.1" 200 None 2026-03-22 16:50:13,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:13,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:50:13,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:13,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ccec3d73288b0d07c4c HTTP/1.1" 200 None 2026-03-22 16:50:13,578 - lava_callback - INFO - Completed processing callback for node 69c01ccec3d73288b0d07c4c INFO: 172.20.0.1:43600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44086 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:50:29,320 - lava_callback - INFO - Received callback for node 69c01cc1c3d73288b0d07c15 INFO: 172.20.0.1:44100 - "POST /node/69c01cc1c3d73288b0d07c15 HTTP/1.0" 202 Accepted 2026-03-22 16:50:29,333 - lava_callback - INFO - Starting async job submit for node 69c01cc1c3d73288b0d07c15 2026-03-22 16:50:29,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:29,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc1c3d73288b0d07c15 HTTP/1.1" 200 None 2026-03-22 16:50:29,411 - lava_callback - DEBUG - Processing node 69c01cc1c3d73288b0d07c15: ltp-cve 2026-03-22 16:50:30,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:30,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:30,545 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c01cc1c3d73288b0d07c15/log.txt.gz 2026-03-22 16:50:30,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:31,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:31,475 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c01cc1c3d73288b0d07c15/lava-logs.yaml 2026-03-22 16:50:31,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:31,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:31,842 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c01cc1c3d73288b0d07c15/lava_callback.json.gz 2026-03-22 16:50:31,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:31,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc1c3d73288b0d07c15 HTTP/1.1" 200 None 2026-03-22 16:50:31,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:31,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:50:31,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:31,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cc1c3d73288b0d07c15 HTTP/1.1" 200 None 2026-03-22 16:50:31,988 - lava_callback - INFO - Completed processing callback for node 69c01cc1c3d73288b0d07c15 2026-03-22 16:50:32,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:32,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34480 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:50:48,774 - lava_callback - INFO - Received callback for node 69bd7ad30e93e2314723e725 INFO: 172.20.0.1:34486 - "POST /node/69bd7ad30e93e2314723e725 HTTP/1.0" 202 Accepted 2026-03-22 16:50:48,788 - lava_callback - INFO - Starting async job submit for node 69bd7ad30e93e2314723e725 2026-03-22 16:50:48,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:48,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7ad30e93e2314723e725 HTTP/1.1" 200 None 2026-03-22 16:50:48,824 - lava_callback - DEBUG - Processing node 69bd7ad30e93e2314723e725: baseline-arm64-baylibre 2026-03-22 16:50:48,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7ad30e93e2314723e725 2026-03-22 16:50:48,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:48,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7ad30e93e2314723e725 HTTP/1.1" 200 None 2026-03-22 16:50:48,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:48,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 16:50:48,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:48,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7ad30e93e2314723e725 HTTP/1.1" 200 None 2026-03-22 16:50:48,933 - lava_callback - ERROR - Error processing callback for node 69bd7ad30e93e2314723e725: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 16:50:52,304 - lava_callback - INFO - Received callback for node 69c01ca3c3d73288b0d07bb9 INFO: 172.20.0.1:40192 - "POST /node/69c01ca3c3d73288b0d07bb9 HTTP/1.0" 202 Accepted 2026-03-22 16:50:52,313 - lava_callback - INFO - Starting async job submit for node 69c01ca3c3d73288b0d07bb9 2026-03-22 16:50:52,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:52,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca3c3d73288b0d07bb9 HTTP/1.1" 200 None 2026-03-22 16:50:52,396 - lava_callback - DEBUG - Processing node 69c01ca3c3d73288b0d07bb9: baseline-arm-clabbe 2026-03-22 16:50:52,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:53,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:53,473 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01ca3c3d73288b0d07bb9/log.txt.gz 2026-03-22 16:50:53,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:54,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:54,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01ca3c3d73288b0d07bb9/lava-logs.yaml 2026-03-22 16:50:54,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:50:56,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:50:56,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01ca3c3d73288b0d07bb9/lava_callback.json.gz 2026-03-22 16:50:56,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:56,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca3c3d73288b0d07bb9 HTTP/1.1" 200 None 2026-03-22 16:50:56,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:56,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:50:56,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:50:56,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ca3c3d73288b0d07bb9 HTTP/1.1" 200 None 2026-03-22 16:50:56,289 - lava_callback - INFO - Completed processing callback for node 69c01ca3c3d73288b0d07bb9 INFO: 172.20.0.1:40202 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:51:02,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:02,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35932 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:51:09,344 - lava_callback - INFO - Received callback for node 69c01cc4c3d73288b0d07c1c INFO: 172.20.0.1:35946 - "POST /node/69c01cc4c3d73288b0d07c1c HTTP/1.0" 202 Accepted 2026-03-22 16:51:09,355 - lava_callback - INFO - Starting async job submit for node 69c01cc4c3d73288b0d07c1c 2026-03-22 16:51:09,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:09,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc4c3d73288b0d07c1c HTTP/1.1" 200 None 2026-03-22 16:51:09,428 - lava_callback - DEBUG - Processing node 69c01cc4c3d73288b0d07c1c: ltp-fs_bind 2026-03-22 16:51:10,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:51:10,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:51:10,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c01cc4c3d73288b0d07c1c/log.txt.gz 2026-03-22 16:51:11,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:51:12,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:51:12,493 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c01cc4c3d73288b0d07c1c/lava-logs.yaml 2026-03-22 16:51:12,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:51:13,256 - lava_callback - INFO - Received callback for node 69bd7af30e93e2314723e773 INFO: 172.20.0.1:49478 - "POST /node/69bd7af30e93e2314723e773 HTTP/1.0" 202 Accepted 2026-03-22 16:51:13,265 - lava_callback - INFO - Starting async job submit for node 69bd7af30e93e2314723e773 2026-03-22 16:51:13,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:13,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7af30e93e2314723e773 HTTP/1.1" 200 None 2026-03-22 16:51:13,299 - lava_callback - DEBUG - Processing node 69bd7af30e93e2314723e773: baseline-arm64-clabbe 2026-03-22 16:51:13,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7af30e93e2314723e773 2026-03-22 16:51:13,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:13,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7af30e93e2314723e773 HTTP/1.1" 200 None 2026-03-22 16:51:13,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:13,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 16:51:13,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:13,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7af30e93e2314723e773 HTTP/1.1" 200 None 2026-03-22 16:51:13,398 - lava_callback - ERROR - Error processing callback for node 69bd7af30e93e2314723e773: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 16:51:13,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:51:13,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c01cc4c3d73288b0d07c1c/lava_callback.json.gz 2026-03-22 16:51:13,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:13,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc4c3d73288b0d07c1c HTTP/1.1" 200 None 2026-03-22 16:51:13,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:13,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:51:13,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:13,958 - lava_callback - INFO - Received callback for node 69bd7af60e93e2314723e77e INFO: 172.20.0.1:49484 - "POST /node/69bd7af60e93e2314723e77e HTTP/1.0" 202 Accepted 2026-03-22 16:51:13,968 - lava_callback - INFO - Starting async job submit for node 69bd7af60e93e2314723e77e 2026-03-22 16:51:13,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:14,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7af60e93e2314723e77e HTTP/1.1" 200 None 2026-03-22 16:51:14,029 - lava_callback - DEBUG - Processing node 69bd7af60e93e2314723e77e: baseline-arm64-clabbe 2026-03-22 16:51:14,029 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7af60e93e2314723e77e 2026-03-22 16:51:14,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:14,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cc4c3d73288b0d07c1c HTTP/1.1" 200 None 2026-03-22 16:51:14,040 - lava_callback - INFO - Completed processing callback for node 69c01cc4c3d73288b0d07c1c 2026-03-22 16:51:14,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7af60e93e2314723e77e HTTP/1.1" 200 None 2026-03-22 16:51:14,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:14,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 16:51:14,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:14,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7af60e93e2314723e77e HTTP/1.1" 200 None 2026-03-22 16:51:14,212 - lava_callback - ERROR - Error processing callback for node 69bd7af60e93e2314723e77e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 16:51:14,650 - lava_callback - INFO - Received callback for node 69bd7af90e93e2314723e781 INFO: 172.20.0.1:49494 - "POST /node/69bd7af90e93e2314723e781 HTTP/1.0" 202 Accepted 2026-03-22 16:51:14,672 - lava_callback - INFO - Starting async job submit for node 69bd7af90e93e2314723e781 2026-03-22 16:51:14,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:14,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7af90e93e2314723e781 HTTP/1.1" 200 None 2026-03-22 16:51:14,700 - lava_callback - DEBUG - Processing node 69bd7af90e93e2314723e781: baseline-arm64-clabbe 2026-03-22 16:51:14,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7af90e93e2314723e781 2026-03-22 16:51:14,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:14,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7af90e93e2314723e781 HTTP/1.1" 200 None 2026-03-22 16:51:14,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:14,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 16:51:14,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:14,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7af90e93e2314723e781 HTTP/1.1" 200 None 2026-03-22 16:51:14,819 - lava_callback - ERROR - Error processing callback for node 69bd7af90e93e2314723e781: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49510 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-22 16:51:15,347 - lava_callback - INFO - Received callback for node 69bd7afb0e93e2314723e784 INFO: 172.20.0.1:49522 - "POST /node/69bd7afb0e93e2314723e784 HTTP/1.0" 202 Accepted 2026-03-22 16:51:15,357 - lava_callback - INFO - Starting async job submit for node 69bd7afb0e93e2314723e784 2026-03-22 16:51:15,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:15,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7afb0e93e2314723e784 HTTP/1.1" 200 None 2026-03-22 16:51:15,388 - lava_callback - DEBUG - Processing node 69bd7afb0e93e2314723e784: baseline-arm64-clabbe 2026-03-22 16:51:15,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7afb0e93e2314723e784 2026-03-22 16:51:15,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:15,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7afb0e93e2314723e784 HTTP/1.1" 200 None 2026-03-22 16:51:15,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:15,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 16:51:15,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:15,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7afb0e93e2314723e784 HTTP/1.1" 200 None 2026-03-22 16:51:15,480 - lava_callback - ERROR - Error processing callback for node 69bd7afb0e93e2314723e784: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58066 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:51:32,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:32,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-22 16:51:33,202 - lava_callback - INFO - Received callback for node 69bd7b000e93e2314723e79f INFO: 172.20.0.1:44006 - "POST /node/69bd7b000e93e2314723e79f HTTP/1.0" 202 Accepted 2026-03-22 16:51:33,211 - lava_callback - INFO - Starting async job submit for node 69bd7b000e93e2314723e79f 2026-03-22 16:51:33,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:33,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7b000e93e2314723e79f HTTP/1.1" 200 None 2026-03-22 16:51:33,239 - lava_callback - DEBUG - Processing node 69bd7b000e93e2314723e79f: baseline-arm64-clabbe 2026-03-22 16:51:33,239 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7b000e93e2314723e79f 2026-03-22 16:51:33,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:33,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7b000e93e2314723e79f HTTP/1.1" 200 None 2026-03-22 16:51:33,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:33,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 16:51:33,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:33,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7b000e93e2314723e79f HTTP/1.1" 200 None 2026-03-22 16:51:33,339 - lava_callback - ERROR - Error processing callback for node 69bd7b000e93e2314723e79f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 16:51:33,897 - lava_callback - INFO - Received callback for node 69bd7b020e93e2314723e7ac INFO: 172.20.0.1:44008 - "POST /node/69bd7b020e93e2314723e7ac HTTP/1.0" 202 Accepted 2026-03-22 16:51:33,906 - lava_callback - INFO - Starting async job submit for node 69bd7b020e93e2314723e7ac 2026-03-22 16:51:33,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:33,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7b020e93e2314723e7ac HTTP/1.1" 200 None 2026-03-22 16:51:33,937 - lava_callback - DEBUG - Processing node 69bd7b020e93e2314723e7ac: baseline-arm64-clabbe 2026-03-22 16:51:33,937 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7b020e93e2314723e7ac 2026-03-22 16:51:33,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:33,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7b020e93e2314723e7ac HTTP/1.1" 200 None 2026-03-22 16:51:33,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:33,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 16:51:33,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:51:34,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7b020e93e2314723e7ac HTTP/1.1" 200 None 2026-03-22 16:51:34,043 - lava_callback - ERROR - Error processing callback for node 69bd7b020e93e2314723e7ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33332 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:52:51,403 - lava_callback - INFO - Received callback for node 69c01ca4c3d73288b0d07bbc INFO: 172.20.0.1:40592 - "POST /node/69c01ca4c3d73288b0d07bbc HTTP/1.0" 202 Accepted 2026-03-22 16:52:51,410 - lava_callback - INFO - Starting async job submit for node 69c01ca4c3d73288b0d07bbc 2026-03-22 16:52:51,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:52:51,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca4c3d73288b0d07bbc HTTP/1.1" 200 None 2026-03-22 16:52:51,448 - lava_callback - DEBUG - Processing node 69c01ca4c3d73288b0d07bbc: baseline-arm-clabbe 2026-03-22 16:52:51,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:52:52,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:52:52,231 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01ca4c3d73288b0d07bbc/log.txt.gz 2026-03-22 16:52:52,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:52:52,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:52:52,818 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01ca4c3d73288b0d07bbc/lava-logs.yaml 2026-03-22 16:52:52,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:52:53,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:52:53,372 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c01ca4c3d73288b0d07bbc/lava_callback.json.gz 2026-03-22 16:52:53,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:52:53,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ca4c3d73288b0d07bbc HTTP/1.1" 200 None 2026-03-22 16:52:53,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:52:53,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:52:53,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:52:53,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ca4c3d73288b0d07bbc HTTP/1.1" 200 None 2026-03-22 16:52:53,502 - lava_callback - INFO - Completed processing callback for node 69c01ca4c3d73288b0d07bbc INFO: 172.20.0.1:40608 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:52:58,000 - lava_callback - INFO - Received callback for node 69c019c5c3d73288b0d07a5c INFO: 172.20.0.1:40620 - "POST /node/69c019c5c3d73288b0d07a5c HTTP/1.0" 202 Accepted 2026-03-22 16:52:58,011 - lava_callback - INFO - Starting async job submit for node 69c019c5c3d73288b0d07a5c 2026-03-22 16:52:58,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:52:58,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c019c5c3d73288b0d07a5c HTTP/1.1" 200 None 2026-03-22 16:52:58,148 - lava_callback - DEBUG - Processing node 69c019c5c3d73288b0d07a5c: ltp-smoke 2026-03-22 16:52:58,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:52:59,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:52:59,405 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c019c5c3d73288b0d07a5c/log.txt.gz 2026-03-22 16:52:59,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:53:00,215 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:53:00,218 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c019c5c3d73288b0d07a5c/lava-logs.yaml 2026-03-22 16:53:00,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:53:00,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:53:00,466 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c019c5c3d73288b0d07a5c/lava_callback.json.gz 2026-03-22 16:53:00,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:53:00,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c019c5c3d73288b0d07a5c HTTP/1.1" 200 None 2026-03-22 16:53:00,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:53:00,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01637c3d73288b0d052f6 HTTP/1.1" 200 None 2026-03-22 16:53:00,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:53:00,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c019c5c3d73288b0d07a5c HTTP/1.1" 200 None 2026-03-22 16:53:00,911 - lava_callback - INFO - Completed processing callback for node 69c019c5c3d73288b0d07a5c 2026-03-22 16:53:02,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:53:02,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44188 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:53:49,448 - lava_callback - INFO - Received callback for node 69c01cc6c3d73288b0d07c1f INFO: 172.20.0.1:44194 - "POST /node/69c01cc6c3d73288b0d07c1f HTTP/1.0" 202 Accepted 2026-03-22 16:53:49,457 - lava_callback - INFO - Starting async job submit for node 69c01cc6c3d73288b0d07c1f 2026-03-22 16:53:49,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:53:49,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc6c3d73288b0d07c1f HTTP/1.1" 200 None 2026-03-22 16:53:49,516 - lava_callback - DEBUG - Processing node 69c01cc6c3d73288b0d07c1f: ltp-fsx 2026-03-22 16:53:50,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:53:51,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:53:51,371 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c01cc6c3d73288b0d07c1f/log.txt.gz 2026-03-22 16:53:51,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:53:52,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:53:52,894 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c01cc6c3d73288b0d07c1f/lava-logs.yaml 2026-03-22 16:53:52,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:53:54,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:53:54,174 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c01cc6c3d73288b0d07c1f/lava_callback.json.gz 2026-03-22 16:53:54,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:53:54,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc6c3d73288b0d07c1f HTTP/1.1" 200 None 2026-03-22 16:53:54,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:53:54,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:53:54,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:53:54,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cc6c3d73288b0d07c1f HTTP/1.1" 200 None 2026-03-22 16:53:54,271 - lava_callback - INFO - Completed processing callback for node 69c01cc6c3d73288b0d07c1f INFO: 172.20.0.1:49442 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:53:58,281 - lava_callback - INFO - Received callback for node 69c01cc7c3d73288b0d07c22 INFO: 172.20.0.1:49458 - "POST /node/69c01cc7c3d73288b0d07c22 HTTP/1.0" 202 Accepted 2026-03-22 16:53:58,291 - lava_callback - INFO - Starting async job submit for node 69c01cc7c3d73288b0d07c22 2026-03-22 16:53:58,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:53:58,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc7c3d73288b0d07c22 HTTP/1.1" 200 None 2026-03-22 16:53:58,345 - lava_callback - DEBUG - Processing node 69c01cc7c3d73288b0d07c22: ltp-input 2026-03-22 16:53:58,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:54:00,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:54:00,035 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c01cc7c3d73288b0d07c22/log.txt.gz 2026-03-22 16:54:00,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:54:01,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:54:01,235 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c01cc7c3d73288b0d07c22/lava-logs.yaml 2026-03-22 16:54:01,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:54:02,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:54:02,160 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c01cc7c3d73288b0d07c22/lava_callback.json.gz 2026-03-22 16:54:02,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:54:02,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc7c3d73288b0d07c22 HTTP/1.1" 200 None 2026-03-22 16:54:02,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:54:02,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:54:02,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:54:02,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cc7c3d73288b0d07c22 HTTP/1.1" 200 None 2026-03-22 16:54:02,281 - lava_callback - INFO - Completed processing callback for node 69c01cc7c3d73288b0d07c22 2026-03-22 16:54:02,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:54:02,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38802 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:55:09,487 - lava_callback - INFO - Received callback for node 69c01cc9c3d73288b0d07c25 INFO: 172.20.0.1:38804 - "POST /node/69c01cc9c3d73288b0d07c25 HTTP/1.0" 202 Accepted 2026-03-22 16:55:09,496 - lava_callback - INFO - Starting async job submit for node 69c01cc9c3d73288b0d07c25 2026-03-22 16:55:09,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:55:09,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc9c3d73288b0d07c25 HTTP/1.1" 200 None 2026-03-22 16:55:09,553 - lava_callback - DEBUG - Processing node 69c01cc9c3d73288b0d07c25: ltp-ipc 2026-03-22 16:55:10,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:55:10,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:55:10,623 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c01cc9c3d73288b0d07c25/log.txt.gz 2026-03-22 16:55:10,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:55:11,103 - lava_callback - INFO - Received callback for node 69c01ccac3d73288b0d07c2e INFO: 172.20.0.1:40638 - "POST /node/69c01ccac3d73288b0d07c2e HTTP/1.0" 202 Accepted 2026-03-22 16:55:11,113 - lava_callback - INFO - Starting async job submit for node 69c01ccac3d73288b0d07c2e 2026-03-22 16:55:11,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:55:11,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ccac3d73288b0d07c2e HTTP/1.1" 200 None 2026-03-22 16:55:11,179 - lava_callback - DEBUG - Processing node 69c01ccac3d73288b0d07c2e: ltp-pty 2026-03-22 16:55:11,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:55:11,582 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c01cc9c3d73288b0d07c25/lava-logs.yaml 2026-03-22 16:55:11,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:55:11,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:55:12,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:55:12,175 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c01cc9c3d73288b0d07c25/lava_callback.json.gz 2026-03-22 16:55:12,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:55:12,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:55:12,226 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c01ccac3d73288b0d07c2e/log.txt.gz 2026-03-22 16:55:12,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cc9c3d73288b0d07c25 HTTP/1.1" 200 None 2026-03-22 16:55:12,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:55:12,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:55:12,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:55:12,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cc9c3d73288b0d07c25 HTTP/1.1" 200 None 2026-03-22 16:55:12,469 - lava_callback - INFO - Completed processing callback for node 69c01cc9c3d73288b0d07c25 2026-03-22 16:55:12,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:55:13,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:55:13,244 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c01ccac3d73288b0d07c2e/lava-logs.yaml 2026-03-22 16:55:13,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:55:13,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:55:13,759 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c01ccac3d73288b0d07c2e/lava_callback.json.gz 2026-03-22 16:55:13,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:55:13,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ccac3d73288b0d07c2e HTTP/1.1" 200 None 2026-03-22 16:55:13,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:55:13,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:55:13,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:55:13,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ccac3d73288b0d07c2e HTTP/1.1" 200 None 2026-03-22 16:55:13,876 - lava_callback - INFO - Completed processing callback for node 69c01ccac3d73288b0d07c2e INFO: 172.20.0.1:40644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55416 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:55:32,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:55:32,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35732 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:57:33,470 - lava_callback - INFO - Received callback for node 69bd7c720e93e2314723e8e4 INFO: 172.20.0.1:55570 - "POST /node/69bd7c720e93e2314723e8e4 HTTP/1.0" 202 Accepted 2026-03-22 16:57:33,479 - lava_callback - INFO - Starting async job submit for node 69bd7c720e93e2314723e8e4 2026-03-22 16:57:33,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:33,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7c720e93e2314723e8e4 HTTP/1.1" 200 None 2026-03-22 16:57:33,525 - lava_callback - DEBUG - Processing node 69bd7c720e93e2314723e8e4: baseline-arm-clabbe 2026-03-22 16:57:33,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7c720e93e2314723e8e4 2026-03-22 16:57:33,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:33,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7c720e93e2314723e8e4 HTTP/1.1" 200 None 2026-03-22 16:57:33,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:33,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-22 16:57:33,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:33,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7c720e93e2314723e8e4 HTTP/1.1" 200 None 2026-03-22 16:57:33,649 - lava_callback - ERROR - Error processing callback for node 69bd7c720e93e2314723e8e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 16:57:34,151 - lava_callback - INFO - Received callback for node 69bd7c780e93e2314723e8f1 INFO: 172.20.0.1:55584 - "POST /node/69bd7c780e93e2314723e8f1 HTTP/1.0" 202 Accepted 2026-03-22 16:57:34,158 - lava_callback - INFO - Starting async job submit for node 69bd7c780e93e2314723e8f1 2026-03-22 16:57:34,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:34,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7c780e93e2314723e8f1 HTTP/1.1" 200 None 2026-03-22 16:57:34,197 - lava_callback - DEBUG - Processing node 69bd7c780e93e2314723e8f1: baseline-arm-clabbe 2026-03-22 16:57:34,197 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7c780e93e2314723e8f1 2026-03-22 16:57:34,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:34,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7c780e93e2314723e8f1 HTTP/1.1" 200 None 2026-03-22 16:57:34,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:34,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-22 16:57:34,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:34,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7c780e93e2314723e8f1 HTTP/1.1" 200 None 2026-03-22 16:57:34,277 - lava_callback - ERROR - Error processing callback for node 69bd7c780e93e2314723e8f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 16:57:34,841 - lava_callback - INFO - Received callback for node 69bd7c7a0e93e2314723e8f4 INFO: 172.20.0.1:55596 - "POST /node/69bd7c7a0e93e2314723e8f4 HTTP/1.0" 202 Accepted 2026-03-22 16:57:34,847 - lava_callback - INFO - Starting async job submit for node 69bd7c7a0e93e2314723e8f4 2026-03-22 16:57:34,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:34,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7c7a0e93e2314723e8f4 HTTP/1.1" 200 None 2026-03-22 16:57:34,871 - lava_callback - DEBUG - Processing node 69bd7c7a0e93e2314723e8f4: baseline-arm-clabbe 2026-03-22 16:57:34,872 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7c7a0e93e2314723e8f4 2026-03-22 16:57:34,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:34,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7c7a0e93e2314723e8f4 HTTP/1.1" 200 None 2026-03-22 16:57:34,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:34,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-22 16:57:34,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:34,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7c7a0e93e2314723e8f4 HTTP/1.1" 200 None 2026-03-22 16:57:34,959 - lava_callback - ERROR - Error processing callback for node 69bd7c7a0e93e2314723e8f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 16:57:35,536 - lava_callback - INFO - Received callback for node 69bd7c7b0e93e2314723e8f7 INFO: 172.20.0.1:55602 - "POST /node/69bd7c7b0e93e2314723e8f7 HTTP/1.0" 202 Accepted 2026-03-22 16:57:35,546 - lava_callback - INFO - Starting async job submit for node 69bd7c7b0e93e2314723e8f7 2026-03-22 16:57:35,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:35,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7c7b0e93e2314723e8f7 HTTP/1.1" 200 None 2026-03-22 16:57:35,564 - lava_callback - DEBUG - Processing node 69bd7c7b0e93e2314723e8f7: baseline-arm-clabbe 2026-03-22 16:57:35,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7c7b0e93e2314723e8f7 2026-03-22 16:57:35,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:35,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7c7b0e93e2314723e8f7 HTTP/1.1" 200 None 2026-03-22 16:57:35,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:35,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-22 16:57:35,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:35,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7c7b0e93e2314723e8f7 HTTP/1.1" 200 None 2026-03-22 16:57:35,630 - lava_callback - ERROR - Error processing callback for node 69bd7c7b0e93e2314723e8f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36108 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:57:49,510 - lava_callback - INFO - Received callback for node 69c01cccc3d73288b0d07c31 INFO: 172.20.0.1:36124 - "POST /node/69c01cccc3d73288b0d07c31 HTTP/1.0" 202 Accepted 2026-03-22 16:57:49,520 - lava_callback - INFO - Starting async job submit for node 69c01cccc3d73288b0d07c31 2026-03-22 16:57:49,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:49,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cccc3d73288b0d07c31 HTTP/1.1" 200 None 2026-03-22 16:57:49,599 - lava_callback - DEBUG - Processing node 69c01cccc3d73288b0d07c31: ltp-sched 2026-03-22 16:57:50,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:57:50,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:57:50,394 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c01cccc3d73288b0d07c31/log.txt.gz 2026-03-22 16:57:50,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:57:51,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:57:51,115 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c01cccc3d73288b0d07c31/lava-logs.yaml 2026-03-22 16:57:51,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:57:51,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:57:51,307 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c01cccc3d73288b0d07c31/lava_callback.json.gz 2026-03-22 16:57:51,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:51,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cccc3d73288b0d07c31 HTTP/1.1" 200 None 2026-03-22 16:57:51,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:51,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:57:51,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:51,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cccc3d73288b0d07c31 HTTP/1.1" 200 None 2026-03-22 16:57:51,445 - lava_callback - INFO - Completed processing callback for node 69c01cccc3d73288b0d07c31 2026-03-22 16:57:53,435 - lava_callback - INFO - Received callback for node 69bd7c7f0e93e2314723e903 INFO: 172.20.0.1:41638 - "POST /node/69bd7c7f0e93e2314723e903 HTTP/1.0" 202 Accepted 2026-03-22 16:57:53,444 - lava_callback - INFO - Starting async job submit for node 69bd7c7f0e93e2314723e903 2026-03-22 16:57:53,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:53,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7c7f0e93e2314723e903 HTTP/1.1" 200 None 2026-03-22 16:57:53,471 - lava_callback - DEBUG - Processing node 69bd7c7f0e93e2314723e903: baseline-arm-clabbe 2026-03-22 16:57:53,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd7c7f0e93e2314723e903 2026-03-22 16:57:53,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:53,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd7c7f0e93e2314723e903 HTTP/1.1" 200 None 2026-03-22 16:57:53,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:53,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-22 16:57:53,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:57:53,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd7c7f0e93e2314723e903 HTTP/1.1" 200 None 2026-03-22 16:57:53,550 - lava_callback - ERROR - Error processing callback for node 69bd7c7f0e93e2314723e903: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41642 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:58:02,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:58:02,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50046 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:58:21,496 - lava_callback - INFO - Received callback for node 69c01fa6c3d73288b0d07e06 INFO: 172.20.0.1:44084 - "POST /node/69c01fa6c3d73288b0d07e06 HTTP/1.0" 202 Accepted 2026-03-22 16:58:21,505 - lava_callback - INFO - Starting async job submit for node 69c01fa6c3d73288b0d07e06 2026-03-22 16:58:21,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:58:21,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01fa6c3d73288b0d07e06 HTTP/1.1" 200 None 2026-03-22 16:58:21,543 - lava_callback - DEBUG - Processing node 69c01fa6c3d73288b0d07e06: baseline-arm 2026-03-22 16:58:21,551 - lava_callback - WARNING - Failed to upload log 2026-03-22 16:58:21,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:58:21,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:58:21,810 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c01fa6c3d73288b0d07e06/lava-logs.yaml 2026-03-22 16:58:21,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:58:21,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:58:21,960 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c01fa6c3d73288b0d07e06/lava_callback.json.gz 2026-03-22 16:58:21,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:58:21,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01fa6c3d73288b0d07e06 HTTP/1.1" 200 None 2026-03-22 16:58:21,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:58:21,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:58:21,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:58:22,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01fa6c3d73288b0d07e06 HTTP/1.1" 200 None 2026-03-22 16:58:22,047 - lava_callback - INFO - Completed processing callback for node 69c01fa6c3d73288b0d07e06 INFO: 172.20.0.1:44092 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:58:32,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:58:32,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60758 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:59:02,356 - lava_callback - INFO - Received callback for node 69c01c88c3d73288b0d07b56 INFO: 172.20.0.1:59280 - "POST /node/69c01c88c3d73288b0d07b56 HTTP/1.0" 202 Accepted 2026-03-22 16:59:02,362 - lava_callback - INFO - Starting async job submit for node 69c01c88c3d73288b0d07b56 2026-03-22 16:59:02,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:02,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c88c3d73288b0d07b56 HTTP/1.1" 200 None 2026-03-22 16:59:02,441 - lava_callback - DEBUG - Processing node 69c01c88c3d73288b0d07b56: baseline-arm 2026-03-22 16:59:03,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:59:03,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:59:03,452 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c88c3d73288b0d07b56/log.txt.gz 2026-03-22 16:59:03,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:59:04,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:59:04,025 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c88c3d73288b0d07b56/lava-logs.yaml 2026-03-22 16:59:04,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:59:04,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:59:04,204 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c01c88c3d73288b0d07b56/lava_callback.json.gz DEBUG: Setup failed for 69c01c88c3d73288b0d07b56. Transitting job node result: incomplete -> fail 2026-03-22 16:59:04,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:04,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01c88c3d73288b0d07b56 HTTP/1.1" 200 None 2026-03-22 16:59:04,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:04,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:59:04,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:04,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01c88c3d73288b0d07b56 HTTP/1.1" 200 None 2026-03-22 16:59:04,507 - lava_callback - INFO - Completed processing callback for node 69c01c88c3d73288b0d07b56 INFO: 172.20.0.1:59288 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:59:09,615 - lava_callback - INFO - Received callback for node 69c01cd0c3d73288b0d07c4f INFO: 172.20.0.1:59294 - "POST /node/69c01cd0c3d73288b0d07c4f HTTP/1.0" 202 Accepted 2026-03-22 16:59:09,622 - lava_callback - INFO - Starting async job submit for node 69c01cd0c3d73288b0d07c4f 2026-03-22 16:59:09,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:09,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cd0c3d73288b0d07c4f HTTP/1.1" 200 None 2026-03-22 16:59:09,664 - lava_callback - DEBUG - Processing node 69c01cd0c3d73288b0d07c4f: ltp-syscalls-ipc 2026-03-22 16:59:10,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:59:10,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:59:10,199 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c01cd0c3d73288b0d07c4f/log.txt.gz 2026-03-22 16:59:10,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:59:10,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:59:10,619 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c01cd0c3d73288b0d07c4f/lava-logs.yaml 2026-03-22 16:59:10,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:59:10,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:59:10,802 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c01cd0c3d73288b0d07c4f/lava_callback.json.gz 2026-03-22 16:59:10,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:10,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cd0c3d73288b0d07c4f HTTP/1.1" 200 None 2026-03-22 16:59:10,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:10,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:59:10,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:10,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cd0c3d73288b0d07c4f HTTP/1.1" 200 None 2026-03-22 16:59:10,895 - lava_callback - INFO - Completed processing callback for node 69c01cd0c3d73288b0d07c4f INFO: 172.20.0.1:38658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:59:32,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:32,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50546 - "GET / HTTP/1.0" 200 OK 2026-03-22 16:59:49,586 - lava_callback - INFO - Received callback for node 69c01cd1c3d73288b0d07c52 INFO: 172.20.0.1:50554 - "POST /node/69c01cd1c3d73288b0d07c52 HTTP/1.0" 202 Accepted 2026-03-22 16:59:49,592 - lava_callback - INFO - Starting async job submit for node 69c01cd1c3d73288b0d07c52 2026-03-22 16:59:49,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:49,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cd1c3d73288b0d07c52 HTTP/1.1" 200 None 2026-03-22 16:59:49,633 - lava_callback - DEBUG - Processing node 69c01cd1c3d73288b0d07c52: ltp-timers 2026-03-22 16:59:50,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:59:50,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:59:50,146 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c01cd1c3d73288b0d07c52/log.txt.gz 2026-03-22 16:59:50,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:59:50,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:59:50,585 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c01cd1c3d73288b0d07c52/lava-logs.yaml 2026-03-22 16:59:50,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 16:59:50,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 16:59:50,787 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c01cd1c3d73288b0d07c52/lava_callback.json.gz 2026-03-22 16:59:50,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:50,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cd1c3d73288b0d07c52 HTTP/1.1" 200 None 2026-03-22 16:59:50,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:50,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 16:59:50,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 16:59:50,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cd1c3d73288b0d07c52 HTTP/1.1" 200 None 2026-03-22 16:59:50,888 - lava_callback - INFO - Completed processing callback for node 69c01cd1c3d73288b0d07c52 INFO: 172.20.0.1:54980 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:00:02,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:00:02,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:01:18,307 - lava_callback - INFO - Received callback for node 69c01ccdc3d73288b0d07c45 INFO: 172.20.0.1:47852 - "POST /node/69c01ccdc3d73288b0d07c45 HTTP/1.0" 202 Accepted 2026-03-22 17:01:18,314 - lava_callback - INFO - Starting async job submit for node 69c01ccdc3d73288b0d07c45 2026-03-22 17:01:18,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:01:18,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ccdc3d73288b0d07c45 HTTP/1.1" 200 None 2026-03-22 17:01:18,372 - lava_callback - DEBUG - Processing node 69c01ccdc3d73288b0d07c45: ltp-smoke 2026-03-22 17:01:18,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:01:18,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:01:18,924 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c01ccdc3d73288b0d07c45/log.txt.gz 2026-03-22 17:01:19,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:01:19,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:01:19,417 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c01ccdc3d73288b0d07c45/lava-logs.yaml 2026-03-22 17:01:19,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:01:19,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:01:19,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c01ccdc3d73288b0d07c45/lava_callback.json.gz 2026-03-22 17:01:19,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:01:19,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01ccdc3d73288b0d07c45 HTTP/1.1" 200 None 2026-03-22 17:01:19,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:01:19,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 17:01:19,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:01:19,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01ccdc3d73288b0d07c45 HTTP/1.1" 200 None 2026-03-22 17:01:19,698 - lava_callback - INFO - Completed processing callback for node 69c01ccdc3d73288b0d07c45 INFO: 172.20.0.1:59736 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:01:32,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:01:32,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36426 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:01:49,718 - lava_callback - INFO - Received callback for node 69c01cd3c3d73288b0d07c55 INFO: 172.20.0.1:53672 - "POST /node/69c01cd3c3d73288b0d07c55 HTTP/1.0" 202 Accepted 2026-03-22 17:01:49,724 - lava_callback - INFO - Starting async job submit for node 69c01cd3c3d73288b0d07c55 2026-03-22 17:01:49,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:01:49,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cd3c3d73288b0d07c55 HTTP/1.1" 200 None 2026-03-22 17:01:49,764 - lava_callback - DEBUG - Processing node 69c01cd3c3d73288b0d07c55: ltp-watchqueue 2026-03-22 17:01:50,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:01:50,305 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:01:50,306 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c01cd3c3d73288b0d07c55/log.txt.gz 2026-03-22 17:01:50,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:01:50,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:01:50,737 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c01cd3c3d73288b0d07c55/lava-logs.yaml 2026-03-22 17:01:50,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:01:50,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:01:50,904 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c01cd3c3d73288b0d07c55/lava_callback.json.gz 2026-03-22 17:01:50,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:01:50,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cd3c3d73288b0d07c55 HTTP/1.1" 200 None 2026-03-22 17:01:50,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:01:50,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 17:01:50,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:01:51,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cd3c3d73288b0d07c55 HTTP/1.1" 200 None 2026-03-22 17:01:51,009 - lava_callback - INFO - Completed processing callback for node 69c01cd3c3d73288b0d07c55 INFO: 172.20.0.1:53688 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:02:02,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:02:02,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55474 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:04:57,706 - lava_callback - INFO - Received callback for node 69c01cabc3d73288b0d07bd0 INFO: 172.20.0.1:55488 - "POST /node/69c01cabc3d73288b0d07bd0 HTTP/1.0" 202 Accepted 2026-03-22 17:04:57,755 - lava_callback - INFO - Starting async job submit for node 69c01cabc3d73288b0d07bd0 2026-03-22 17:04:57,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:04:57,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cabc3d73288b0d07bd0 HTTP/1.1" 200 None 2026-03-22 17:04:57,808 - lava_callback - DEBUG - Processing node 69c01cabc3d73288b0d07bd0: baseline-arm-pengutronix 2026-03-22 17:04:57,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:04:58,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:04:58,221 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01cabc3d73288b0d07bd0/log.txt.gz 2026-03-22 17:04:58,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:04:58,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:04:58,653 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01cabc3d73288b0d07bd0/lava-logs.yaml 2026-03-22 17:04:58,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:04:58,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:04:58,925 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c01cabc3d73288b0d07bd0/lava_callback.json.gz DEBUG: Setup failed for 69c01cabc3d73288b0d07bd0. Transitting job node result: incomplete -> fail 2026-03-22 17:04:58,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:04:58,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cabc3d73288b0d07bd0 HTTP/1.1" 200 None 2026-03-22 17:04:58,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:04:59,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 17:04:59,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:04:59,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cabc3d73288b0d07bd0 HTTP/1.1" 200 None 2026-03-22 17:04:59,101 - lava_callback - INFO - Completed processing callback for node 69c01cabc3d73288b0d07bd0 2026-03-22 17:05:02,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:05:02,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46216 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:11:29,985 - lava_callback - INFO - Received callback for node 69c01cbdc3d73288b0d07c09 INFO: 172.20.0.1:48822 - "POST /node/69c01cbdc3d73288b0d07c09 HTTP/1.0" 202 Accepted 2026-03-22 17:11:29,991 - lava_callback - INFO - Starting async job submit for node 69c01cbdc3d73288b0d07c09 2026-03-22 17:11:30,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:11:30,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cbdc3d73288b0d07c09 HTTP/1.1" 200 None 2026-03-22 17:11:30,039 - lava_callback - DEBUG - Processing node 69c01cbdc3d73288b0d07c09: ltp-containers 2026-03-22 17:11:30,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:11:30,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:11:30,346 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c01cbdc3d73288b0d07c09/log.txt.gz 2026-03-22 17:11:30,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:11:30,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:11:30,619 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c01cbdc3d73288b0d07c09/lava-logs.yaml 2026-03-22 17:11:30,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:11:30,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:11:30,760 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c01cbdc3d73288b0d07c09/lava_callback.json.gz 2026-03-22 17:11:30,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:11:30,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01cbdc3d73288b0d07c09 HTTP/1.1" 200 None 2026-03-22 17:11:30,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:11:30,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 17:11:30,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:11:30,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c01cbdc3d73288b0d07c09 HTTP/1.1" 200 None 2026-03-22 17:11:30,921 - lava_callback - INFO - Completed processing callback for node 69c01cbdc3d73288b0d07c09 2026-03-22 17:11:32,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:11:32,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44776 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:13:49,820 - lava_callback - INFO - Received callback for node 69bd804a0e93e2314723ec4f INFO: 172.20.0.1:50276 - "POST /node/69bd804a0e93e2314723ec4f HTTP/1.0" 202 Accepted 2026-03-22 17:13:49,826 - lava_callback - INFO - Starting async job submit for node 69bd804a0e93e2314723ec4f 2026-03-22 17:13:49,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:13:49,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd804a0e93e2314723ec4f HTTP/1.1" 200 None 2026-03-22 17:13:49,841 - lava_callback - DEBUG - Processing node 69bd804a0e93e2314723ec4f: baseline-arm64-baylibre 2026-03-22 17:13:49,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd804a0e93e2314723ec4f 2026-03-22 17:13:49,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:13:49,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd804a0e93e2314723ec4f HTTP/1.1" 200 None 2026-03-22 17:13:49,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:13:49,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 17:13:49,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:13:49,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd804a0e93e2314723ec4f HTTP/1.1" 200 None 2026-03-22 17:13:49,899 - lava_callback - ERROR - Error processing callback for node 69bd804a0e93e2314723ec4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60170 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:15:58,601 - lava_callback - INFO - Received callback for node 69c02341c3d73288b0d07ef5 INFO: 172.20.0.1:60182 - "POST /node/69c02341c3d73288b0d07ef5 HTTP/1.0" 202 Accepted 2026-03-22 17:15:58,607 - lava_callback - INFO - Starting async job submit for node 69c02341c3d73288b0d07ef5 2026-03-22 17:15:58,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:15:58,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02341c3d73288b0d07ef5 HTTP/1.1" 200 None 2026-03-22 17:15:58,668 - lava_callback - DEBUG - Processing node 69c02341c3d73288b0d07ef5: ltp-smoke 2026-03-22 17:15:58,789 - lava_callback - WARNING - Failed to upload log 2026-03-22 17:15:58,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:15:59,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:15:59,071 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c02341c3d73288b0d07ef5/lava-logs.yaml 2026-03-22 17:15:59,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:15:59,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:15:59,210 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c02341c3d73288b0d07ef5/lava_callback.json.gz 2026-03-22 17:15:59,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:15:59,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02341c3d73288b0d07ef5 HTTP/1.1" 200 None 2026-03-22 17:15:59,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:15:59,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01637c3d73288b0d052f6 HTTP/1.1" 200 None 2026-03-22 17:15:59,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:15:59,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02341c3d73288b0d07ef5 HTTP/1.1" 200 None 2026-03-22 17:15:59,383 - lava_callback - INFO - Completed processing callback for node 69c02341c3d73288b0d07ef5 2026-03-22 17:16:02,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:16:02,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54246 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:16:49,878 - lava_callback - INFO - Received callback for node 69bd80f00e93e2314723ec6e INFO: 172.20.0.1:52158 - "POST /node/69bd80f00e93e2314723ec6e HTTP/1.0" 202 Accepted 2026-03-22 17:16:49,884 - lava_callback - INFO - Starting async job submit for node 69bd80f00e93e2314723ec6e 2026-03-22 17:16:49,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:16:49,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd80f00e93e2314723ec6e HTTP/1.1" 200 None 2026-03-22 17:16:49,900 - lava_callback - DEBUG - Processing node 69bd80f00e93e2314723ec6e: baseline-arm64-baylibre 2026-03-22 17:16:49,901 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd80f00e93e2314723ec6e 2026-03-22 17:16:49,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:16:49,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd80f00e93e2314723ec6e HTTP/1.1" 200 None 2026-03-22 17:16:49,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:16:49,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 17:16:49,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:16:49,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd80f00e93e2314723ec6e HTTP/1.1" 200 None 2026-03-22 17:16:49,965 - lava_callback - ERROR - Error processing callback for node 69bd80f00e93e2314723ec6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42818 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:18:09,974 - lava_callback - INFO - Received callback for node 69bd81410e93e2314723ec83 INFO: 172.20.0.1:42578 - "POST /node/69bd81410e93e2314723ec83 HTTP/1.0" 202 Accepted 2026-03-22 17:18:09,979 - lava_callback - INFO - Starting async job submit for node 69bd81410e93e2314723ec83 2026-03-22 17:18:09,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:18:09,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd81410e93e2314723ec83 HTTP/1.1" 200 None 2026-03-22 17:18:09,996 - lava_callback - DEBUG - Processing node 69bd81410e93e2314723ec83: baseline-arm64-baylibre 2026-03-22 17:18:09,996 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd81410e93e2314723ec83 2026-03-22 17:18:09,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:18:10,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd81410e93e2314723ec83 HTTP/1.1" 200 None 2026-03-22 17:18:10,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:18:10,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 17:18:10,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:18:10,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd81410e93e2314723ec83 HTTP/1.1" 200 None 2026-03-22 17:18:10,081 - lava_callback - ERROR - Error processing callback for node 69bd81410e93e2314723ec83: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37670 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:20:50,103 - lava_callback - INFO - Received callback for node 69bd81ec0e93e2314723ec9b INFO: 172.20.0.1:39960 - "POST /node/69bd81ec0e93e2314723ec9b HTTP/1.0" 202 Accepted 2026-03-22 17:20:50,109 - lava_callback - INFO - Starting async job submit for node 69bd81ec0e93e2314723ec9b 2026-03-22 17:20:50,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:20:50,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd81ec0e93e2314723ec9b HTTP/1.1" 200 None 2026-03-22 17:20:50,125 - lava_callback - DEBUG - Processing node 69bd81ec0e93e2314723ec9b: baseline-arm64-baylibre 2026-03-22 17:20:50,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd81ec0e93e2314723ec9b 2026-03-22 17:20:50,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:20:50,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd81ec0e93e2314723ec9b HTTP/1.1" 200 None 2026-03-22 17:20:50,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:20:50,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 17:20:50,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:20:50,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd81ec0e93e2314723ec9b HTTP/1.1" 200 None 2026-03-22 17:20:50,186 - lava_callback - ERROR - Error processing callback for node 69bd81ec0e93e2314723ec9b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54140 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:22:03,677 - lava_callback - INFO - Received callback for node 69c02522c3d73288b0d07f57 INFO: 172.20.0.1:56238 - "POST /node/69c02522c3d73288b0d07f57 HTTP/1.0" 202 Accepted 2026-03-22 17:22:03,683 - lava_callback - INFO - Starting async job submit for node 69c02522c3d73288b0d07f57 2026-03-22 17:22:03,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:22:03,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02522c3d73288b0d07f57 HTTP/1.1" 200 None 2026-03-22 17:22:03,701 - lava_callback - DEBUG - Processing node 69c02522c3d73288b0d07f57: baseline-arm 2026-03-22 17:22:03,706 - lava_callback - WARNING - Failed to upload log 2026-03-22 17:22:03,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:22:04,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:22:04,040 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c02522c3d73288b0d07f57/lava-logs.yaml 2026-03-22 17:22:04,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:22:04,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:22:04,209 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c02522c3d73288b0d07f57/lava_callback.json.gz 2026-03-22 17:22:04,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:22:04,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02522c3d73288b0d07f57 HTTP/1.1" 200 None 2026-03-22 17:22:04,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:22:04,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 17:22:04,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:22:04,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02522c3d73288b0d07f57 HTTP/1.1" 200 None 2026-03-22 17:22:04,297 - lava_callback - INFO - Completed processing callback for node 69c02522c3d73288b0d07f57 INFO: 172.20.0.1:56240 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:22:10,165 - lava_callback - INFO - Received callback for node 69bd823e0e93e2314723eca6 INFO: 172.20.0.1:36168 - "POST /node/69bd823e0e93e2314723eca6 HTTP/1.0" 202 Accepted 2026-03-22 17:22:10,171 - lava_callback - INFO - Starting async job submit for node 69bd823e0e93e2314723eca6 2026-03-22 17:22:10,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:22:10,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd823e0e93e2314723eca6 HTTP/1.1" 200 None 2026-03-22 17:22:10,187 - lava_callback - DEBUG - Processing node 69bd823e0e93e2314723eca6: baseline-arm64-baylibre 2026-03-22 17:22:10,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd823e0e93e2314723eca6 2026-03-22 17:22:10,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:22:10,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd823e0e93e2314723eca6 HTTP/1.1" 200 None 2026-03-22 17:22:10,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:22:10,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 17:22:10,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:22:10,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd823e0e93e2314723eca6 HTTP/1.1" 200 None 2026-03-22 17:22:10,282 - lava_callback - ERROR - Error processing callback for node 69bd823e0e93e2314723eca6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49890 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:22:32,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:22:32,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41252 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:23:30,240 - lava_callback - INFO - Received callback for node 69bd828f0e93e2314723ecb9 INFO: 172.20.0.1:35112 - "POST /node/69bd828f0e93e2314723ecb9 HTTP/1.0" 202 Accepted 2026-03-22 17:23:30,245 - lava_callback - INFO - Starting async job submit for node 69bd828f0e93e2314723ecb9 2026-03-22 17:23:30,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:23:30,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd828f0e93e2314723ecb9 HTTP/1.1" 200 None 2026-03-22 17:23:30,261 - lava_callback - DEBUG - Processing node 69bd828f0e93e2314723ecb9: baseline-arm64-baylibre 2026-03-22 17:23:30,262 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd828f0e93e2314723ecb9 2026-03-22 17:23:30,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:23:30,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd828f0e93e2314723ecb9 HTTP/1.1" 200 None 2026-03-22 17:23:30,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:23:30,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 17:23:30,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:23:30,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd828f0e93e2314723ecb9 HTTP/1.1" 200 None 2026-03-22 17:23:30,332 - lava_callback - ERROR - Error processing callback for node 69bd828f0e93e2314723ecb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51062 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:25:10,367 - lava_callback - INFO - Received callback for node 69bd82e10e93e2314723ecc4 INFO: 172.20.0.1:43784 - "POST /node/69bd82e10e93e2314723ecc4 HTTP/1.0" 202 Accepted 2026-03-22 17:25:10,372 - lava_callback - INFO - Starting async job submit for node 69bd82e10e93e2314723ecc4 2026-03-22 17:25:10,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:25:10,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd82e10e93e2314723ecc4 HTTP/1.1" 200 None 2026-03-22 17:25:10,400 - lava_callback - DEBUG - Processing node 69bd82e10e93e2314723ecc4: baseline-arm64-baylibre 2026-03-22 17:25:10,400 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd82e10e93e2314723ecc4 2026-03-22 17:25:10,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:25:10,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd82e10e93e2314723ecc4 HTTP/1.1" 200 None 2026-03-22 17:25:10,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:25:10,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:25:10,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:25:10,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd82e10e93e2314723ecc4 HTTP/1.1" 200 None 2026-03-22 17:25:10,461 - lava_callback - ERROR - Error processing callback for node 69bd82e10e93e2314723ecc4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43426 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:26:30,404 - lava_callback - INFO - Received callback for node 69bd83390e93e2314723eccd INFO: 172.20.0.1:51630 - "POST /node/69bd83390e93e2314723eccd HTTP/1.0" 202 Accepted 2026-03-22 17:26:30,410 - lava_callback - INFO - Starting async job submit for node 69bd83390e93e2314723eccd 2026-03-22 17:26:30,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:26:30,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd83390e93e2314723eccd HTTP/1.1" 200 None 2026-03-22 17:26:30,431 - lava_callback - DEBUG - Processing node 69bd83390e93e2314723eccd: baseline-arm64-baylibre 2026-03-22 17:26:30,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd83390e93e2314723eccd 2026-03-22 17:26:30,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:26:30,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd83390e93e2314723eccd HTTP/1.1" 200 None 2026-03-22 17:26:30,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:26:30,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:26:30,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:26:30,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd83390e93e2314723eccd HTTP/1.1" 200 None 2026-03-22 17:26:30,495 - lava_callback - ERROR - Error processing callback for node 69bd83390e93e2314723eccd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51984 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:28:10,622 - lava_callback - INFO - Received callback for node 69bd83990e93e2314723ecd8 INFO: 172.20.0.1:49548 - "POST /node/69bd83990e93e2314723ecd8 HTTP/1.0" 202 Accepted 2026-03-22 17:28:10,627 - lava_callback - INFO - Starting async job submit for node 69bd83990e93e2314723ecd8 2026-03-22 17:28:10,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:28:10,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd83990e93e2314723ecd8 HTTP/1.1" 200 None 2026-03-22 17:28:10,650 - lava_callback - DEBUG - Processing node 69bd83990e93e2314723ecd8: baseline-arm64-baylibre 2026-03-22 17:28:10,650 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd83990e93e2314723ecd8 2026-03-22 17:28:10,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:28:10,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd83990e93e2314723ecd8 HTTP/1.1" 200 None 2026-03-22 17:28:10,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:28:10,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:28:10,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:28:10,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd83990e93e2314723ecd8 HTTP/1.1" 200 None 2026-03-22 17:28:10,708 - lava_callback - ERROR - Error processing callback for node 69bd83990e93e2314723ecd8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38278 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:29:30,584 - lava_callback - INFO - Received callback for node 69bd83f30e93e2314723ecea INFO: 172.20.0.1:35818 - "POST /node/69bd83f30e93e2314723ecea HTTP/1.0" 202 Accepted 2026-03-22 17:29:30,590 - lava_callback - INFO - Starting async job submit for node 69bd83f30e93e2314723ecea 2026-03-22 17:29:30,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:29:30,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd83f30e93e2314723ecea HTTP/1.1" 200 None 2026-03-22 17:29:30,607 - lava_callback - DEBUG - Processing node 69bd83f30e93e2314723ecea: baseline-arm64-baylibre 2026-03-22 17:29:30,607 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd83f30e93e2314723ecea 2026-03-22 17:29:30,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:29:30,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd83f30e93e2314723ecea HTTP/1.1" 200 None 2026-03-22 17:29:30,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:29:30,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:29:30,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:29:30,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd83f30e93e2314723ecea HTTP/1.1" 200 None 2026-03-22 17:29:30,662 - lava_callback - ERROR - Error processing callback for node 69bd83f30e93e2314723ecea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41604 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:30:50,643 - lava_callback - INFO - Received callback for node 69bd84470e93e2314723ecf6 INFO: 172.20.0.1:55918 - "POST /node/69bd84470e93e2314723ecf6 HTTP/1.0" 202 Accepted 2026-03-22 17:30:50,649 - lava_callback - INFO - Starting async job submit for node 69bd84470e93e2314723ecf6 2026-03-22 17:30:50,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:30:50,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd84470e93e2314723ecf6 HTTP/1.1" 200 None 2026-03-22 17:30:50,665 - lava_callback - DEBUG - Processing node 69bd84470e93e2314723ecf6: baseline-arm64-baylibre 2026-03-22 17:30:50,665 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd84470e93e2314723ecf6 2026-03-22 17:30:50,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:30:50,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd84470e93e2314723ecf6 HTTP/1.1" 200 None 2026-03-22 17:30:50,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:30:50,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:30:50,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:30:50,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd84470e93e2314723ecf6 HTTP/1.1" 200 None 2026-03-22 17:30:50,732 - lava_callback - ERROR - Error processing callback for node 69bd84470e93e2314723ecf6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 17:30:50,753 - lava_callback - INFO - Received callback for node 69c02654c3d73288b0d07fb1 INFO: 172.20.0.1:55928 - "POST /node/69c02654c3d73288b0d07fb1 HTTP/1.0" 202 Accepted 2026-03-22 17:30:50,772 - lava_callback - INFO - Starting async job submit for node 69c02654c3d73288b0d07fb1 2026-03-22 17:30:50,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:30:50,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02654c3d73288b0d07fb1 HTTP/1.1" 200 None 2026-03-22 17:30:50,848 - lava_callback - DEBUG - Processing node 69c02654c3d73288b0d07fb1: ltp-containers 2026-03-22 17:30:51,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:30:51,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:30:51,404 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c02654c3d73288b0d07fb1/log.txt.gz 2026-03-22 17:30:51,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:30:51,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:30:51,844 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c02654c3d73288b0d07fb1/lava-logs.yaml 2026-03-22 17:30:51,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:30:52,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:30:52,034 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c02654c3d73288b0d07fb1/lava_callback.json.gz 2026-03-22 17:30:52,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:30:52,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02654c3d73288b0d07fb1 HTTP/1.1" 200 None 2026-03-22 17:30:52,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:30:52,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 17:30:52,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:30:52,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02654c3d73288b0d07fb1 HTTP/1.1" 200 None 2026-03-22 17:30:52,124 - lava_callback - INFO - Completed processing callback for node 69c02654c3d73288b0d07fb1 INFO: 172.20.0.1:55942 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:31:02,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:31:02,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57870 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:32:10,661 - lava_callback - INFO - Received callback for node 69bd84970e93e2314723ed05 INFO: 172.20.0.1:37144 - "POST /node/69bd84970e93e2314723ed05 HTTP/1.0" 202 Accepted 2026-03-22 17:32:10,707 - lava_callback - INFO - Starting async job submit for node 69bd84970e93e2314723ed05 2026-03-22 17:32:10,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:32:10,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd84970e93e2314723ed05 HTTP/1.1" 200 None 2026-03-22 17:32:10,735 - lava_callback - DEBUG - Processing node 69bd84970e93e2314723ed05: baseline-arm64-baylibre 2026-03-22 17:32:10,735 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd84970e93e2314723ed05 2026-03-22 17:32:10,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:32:10,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd84970e93e2314723ed05 HTTP/1.1" 200 None 2026-03-22 17:32:10,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:32:10,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:32:10,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:32:10,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd84970e93e2314723ed05 HTTP/1.1" 200 None 2026-03-22 17:32:10,805 - lava_callback - ERROR - Error processing callback for node 69bd84970e93e2314723ed05: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55502 - "GET /robots.txt HTTP/1.0" 404 Not Found INFO: 172.20.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43254 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:33:54,962 - lava_callback - INFO - Received callback for node 69bd84ee0e93e2314723ed13 INFO: 172.20.0.1:51592 - "POST /node/69bd84ee0e93e2314723ed13 HTTP/1.0" 202 Accepted 2026-03-22 17:33:54,969 - lava_callback - INFO - Starting async job submit for node 69bd84ee0e93e2314723ed13 2026-03-22 17:33:54,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:33:54,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd84ee0e93e2314723ed13 HTTP/1.1" 200 None 2026-03-22 17:33:54,987 - lava_callback - DEBUG - Processing node 69bd84ee0e93e2314723ed13: baseline-arm64-clabbe 2026-03-22 17:33:54,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd84ee0e93e2314723ed13 2026-03-22 17:33:54,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:33:55,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd84ee0e93e2314723ed13 HTTP/1.1" 200 None 2026-03-22 17:33:55,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:33:55,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:33:55,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:33:55,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd84ee0e93e2314723ed13 HTTP/1.1" 200 None 2026-03-22 17:33:55,072 - lava_callback - ERROR - Error processing callback for node 69bd84ee0e93e2314723ed13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46792 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:35:14,902 - lava_callback - INFO - Received callback for node 69bd85460e93e2314723ed1f INFO: 172.20.0.1:37874 - "POST /node/69bd85460e93e2314723ed1f HTTP/1.0" 202 Accepted 2026-03-22 17:35:14,908 - lava_callback - INFO - Starting async job submit for node 69bd85460e93e2314723ed1f 2026-03-22 17:35:14,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:35:14,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd85460e93e2314723ed1f HTTP/1.1" 200 None 2026-03-22 17:35:14,926 - lava_callback - DEBUG - Processing node 69bd85460e93e2314723ed1f: baseline-arm64-clabbe 2026-03-22 17:35:14,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd85460e93e2314723ed1f 2026-03-22 17:35:14,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:35:14,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd85460e93e2314723ed1f HTTP/1.1" 200 None 2026-03-22 17:35:14,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:35:14,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:35:14,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:35:14,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd85460e93e2314723ed1f HTTP/1.1" 200 None 2026-03-22 17:35:14,977 - lava_callback - ERROR - Error processing callback for node 69bd85460e93e2314723ed1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59684 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:35:54,916 - lava_callback - INFO - Received callback for node 69c02869c3d73288b0d08090 INFO: 172.20.0.1:41500 - "POST /node/69c02869c3d73288b0d08090 HTTP/1.0" 202 Accepted 2026-03-22 17:35:54,926 - lava_callback - INFO - Starting async job submit for node 69c02869c3d73288b0d08090 2026-03-22 17:35:54,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:35:54,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02869c3d73288b0d08090 HTTP/1.1" 200 None 2026-03-22 17:35:54,954 - lava_callback - DEBUG - Processing node 69c02869c3d73288b0d08090: baseline-nfs-arm64 2026-03-22 17:35:54,957 - lava_callback - WARNING - Failed to upload log 2026-03-22 17:35:54,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:35:55,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:35:55,097 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02869c3d73288b0d08090/lava-logs.yaml 2026-03-22 17:35:55,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:35:55,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:35:55,215 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02869c3d73288b0d08090/lava_callback.json.gz 2026-03-22 17:35:55,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:35:55,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02869c3d73288b0d08090 HTTP/1.1" 200 None 2026-03-22 17:35:55,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:35:55,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:35:55,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:35:55,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02869c3d73288b0d08090 HTTP/1.1" 200 None 2026-03-22 17:35:55,283 - lava_callback - INFO - Completed processing callback for node 69c02869c3d73288b0d08090 INFO: 172.20.0.1:41502 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:36:00,562 - lava_callback - INFO - Received callback for node 69c0284dc3d73288b0d08023 INFO: 172.20.0.1:51320 - "POST /node/69c0284dc3d73288b0d08023 HTTP/1.0" 202 Accepted 2026-03-22 17:36:00,569 - lava_callback - INFO - Starting async job submit for node 69c0284dc3d73288b0d08023 2026-03-22 17:36:00,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:00,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0284dc3d73288b0d08023 HTTP/1.1" 200 None 2026-03-22 17:36:00,603 - lava_callback - DEBUG - Processing node 69c0284dc3d73288b0d08023: baseline-arm64 2026-03-22 17:36:00,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:00,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:00,951 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284dc3d73288b0d08023/log.txt.gz 2026-03-22 17:36:01,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:01,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:01,223 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284dc3d73288b0d08023/lava-logs.yaml 2026-03-22 17:36:01,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:01,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:01,380 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284dc3d73288b0d08023/lava_callback.json.gz 2026-03-22 17:36:01,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:01,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0284dc3d73288b0d08023 HTTP/1.1" 200 None 2026-03-22 17:36:01,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:01,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:01,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:01,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0284dc3d73288b0d08023 HTTP/1.1" 200 None 2026-03-22 17:36:01,470 - lava_callback - INFO - Completed processing callback for node 69c0284dc3d73288b0d08023 2026-03-22 17:36:02,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:02,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51322 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:36:10,740 - lava_callback - INFO - Received callback for node 69c02859c3d73288b0d08047 INFO: 172.20.0.1:36570 - "POST /node/69c02859c3d73288b0d08047 HTTP/1.0" 202 Accepted 2026-03-22 17:36:10,751 - lava_callback - INFO - Starting async job submit for node 69c02859c3d73288b0d08047 2026-03-22 17:36:10,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:10,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02859c3d73288b0d08047 HTTP/1.1" 200 None 2026-03-22 17:36:10,884 - lava_callback - DEBUG - Processing node 69c02859c3d73288b0d08047: baseline-arm64 2026-03-22 17:36:11,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:11,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:11,335 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02859c3d73288b0d08047/log.txt.gz 2026-03-22 17:36:11,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:11,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:11,628 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02859c3d73288b0d08047/lava-logs.yaml 2026-03-22 17:36:11,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:11,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:11,814 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02859c3d73288b0d08047/lava_callback.json.gz 2026-03-22 17:36:11,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:11,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02859c3d73288b0d08047 HTTP/1.1" 200 None 2026-03-22 17:36:11,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:11,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:11,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:11,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02859c3d73288b0d08047 HTTP/1.1" 200 None 2026-03-22 17:36:11,891 - lava_callback - INFO - Completed processing callback for node 69c02859c3d73288b0d08047 2026-03-22 17:36:13,322 - lava_callback - INFO - Received callback for node 69c0284fc3d73288b0d0802a INFO: 172.20.0.1:36582 - "POST /node/69c0284fc3d73288b0d0802a HTTP/1.0" 202 Accepted 2026-03-22 17:36:13,328 - lava_callback - INFO - Starting async job submit for node 69c0284fc3d73288b0d0802a 2026-03-22 17:36:13,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:13,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0284fc3d73288b0d0802a HTTP/1.1" 200 None 2026-03-22 17:36:13,366 - lava_callback - DEBUG - Processing node 69c0284fc3d73288b0d0802a: baseline-arm64 2026-03-22 17:36:13,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:13,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:13,831 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284fc3d73288b0d0802a/log.txt.gz 2026-03-22 17:36:14,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:14,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:14,342 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284fc3d73288b0d0802a/lava-logs.yaml 2026-03-22 17:36:14,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:14,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:14,494 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284fc3d73288b0d0802a/lava_callback.json.gz 2026-03-22 17:36:14,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:14,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0284fc3d73288b0d0802a HTTP/1.1" 200 None 2026-03-22 17:36:14,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:14,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:14,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:14,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0284fc3d73288b0d0802a HTTP/1.1" 200 None 2026-03-22 17:36:14,586 - lava_callback - INFO - Completed processing callback for node 69c0284fc3d73288b0d0802a INFO: 172.20.0.1:36592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36598 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:36:23,821 - lava_callback - INFO - Received callback for node 69c02854c3d73288b0d0803b INFO: 172.20.0.1:48254 - "POST /node/69c02854c3d73288b0d0803b HTTP/1.0" 202 Accepted 2026-03-22 17:36:23,830 - lava_callback - INFO - Starting async job submit for node 69c02854c3d73288b0d0803b 2026-03-22 17:36:23,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:23,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02854c3d73288b0d0803b HTTP/1.1" 200 None 2026-03-22 17:36:23,894 - lava_callback - DEBUG - Processing node 69c02854c3d73288b0d0803b: baseline-arm64 2026-03-22 17:36:24,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:24,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:24,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02854c3d73288b0d0803b/log.txt.gz 2026-03-22 17:36:24,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:24,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:24,816 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02854c3d73288b0d0803b/lava-logs.yaml 2026-03-22 17:36:24,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:24,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:24,965 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02854c3d73288b0d0803b/lava_callback.json.gz 2026-03-22 17:36:24,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:25,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02854c3d73288b0d0803b HTTP/1.1" 200 None 2026-03-22 17:36:25,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:25,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:25,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:25,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02854c3d73288b0d0803b HTTP/1.1" 200 None 2026-03-22 17:36:25,057 - lava_callback - INFO - Completed processing callback for node 69c02854c3d73288b0d0803b 2026-03-22 17:36:26,224 - lava_callback - INFO - Received callback for node 69c02856c3d73288b0d0803e INFO: 172.20.0.1:48264 - "POST /node/69c02856c3d73288b0d0803e HTTP/1.0" 202 Accepted 2026-03-22 17:36:26,230 - lava_callback - INFO - Starting async job submit for node 69c02856c3d73288b0d0803e 2026-03-22 17:36:26,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:26,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02856c3d73288b0d0803e HTTP/1.1" 200 None 2026-03-22 17:36:26,268 - lava_callback - DEBUG - Processing node 69c02856c3d73288b0d0803e: baseline-arm64 INFO: 172.20.0.1:48276 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:36:26,675 - lava_callback - INFO - Received callback for node 69c02851c3d73288b0d08030 INFO: 172.20.0.1:48290 - "POST /node/69c02851c3d73288b0d08030 HTTP/1.0" 202 Accepted 2026-03-22 17:36:26,697 - lava_callback - INFO - Starting async job submit for node 69c02851c3d73288b0d08030 2026-03-22 17:36:26,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:26,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:26,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02851c3d73288b0d08030 HTTP/1.1" 200 None 2026-03-22 17:36:26,781 - lava_callback - DEBUG - Processing node 69c02851c3d73288b0d08030: baseline-arm64 2026-03-22 17:36:26,943 - lava_callback - WARNING - Failed to upload log 2026-03-22 17:36:26,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:27,007 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02856c3d73288b0d0803e/log.txt.gz 2026-03-22 17:36:27,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:27,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:27,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:27,347 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02851c3d73288b0d08030/lava-logs.yaml 2026-03-22 17:36:27,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:27,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:27,464 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02856c3d73288b0d0803e/lava-logs.yaml 2026-03-22 17:36:27,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:27,467 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02851c3d73288b0d08030/lava_callback.json.gz 2026-03-22 17:36:27,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:27,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:27,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02851c3d73288b0d08030 HTTP/1.1" 200 None 2026-03-22 17:36:27,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:27,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:27,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:27,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02851c3d73288b0d08030 HTTP/1.1" 200 None 2026-03-22 17:36:27,636 - lava_callback - INFO - Completed processing callback for node 69c02851c3d73288b0d08030 2026-03-22 17:36:27,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:27,695 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02856c3d73288b0d0803e/lava_callback.json.gz 2026-03-22 17:36:27,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:27,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02856c3d73288b0d0803e HTTP/1.1" 200 None 2026-03-22 17:36:27,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:27,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:27,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:27,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02856c3d73288b0d0803e HTTP/1.1" 200 None 2026-03-22 17:36:27,827 - lava_callback - INFO - Completed processing callback for node 69c02856c3d73288b0d0803e 2026-03-22 17:36:30,378 - lava_callback - INFO - Received callback for node 69c0284fc3d73288b0d0802d INFO: 172.20.0.1:49734 - "POST /node/69c0284fc3d73288b0d0802d HTTP/1.0" 202 Accepted 2026-03-22 17:36:30,384 - lava_callback - INFO - Starting async job submit for node 69c0284fc3d73288b0d0802d 2026-03-22 17:36:30,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:30,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0284fc3d73288b0d0802d HTTP/1.1" 200 None 2026-03-22 17:36:30,421 - lava_callback - DEBUG - Processing node 69c0284fc3d73288b0d0802d: baseline-arm64 2026-03-22 17:36:30,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:30,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:30,805 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284fc3d73288b0d0802d/log.txt.gz 2026-03-22 17:36:30,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:31,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:31,140 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284fc3d73288b0d0802d/lava-logs.yaml 2026-03-22 17:36:31,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:31,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:31,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284fc3d73288b0d0802d/lava_callback.json.gz 2026-03-22 17:36:31,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:31,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0284fc3d73288b0d0802d HTTP/1.1" 200 None 2026-03-22 17:36:31,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:31,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:31,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:31,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0284fc3d73288b0d0802d HTTP/1.1" 200 None 2026-03-22 17:36:31,378 - lava_callback - INFO - Completed processing callback for node 69c0284fc3d73288b0d0802d 2026-03-22 17:36:32,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:32,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-22 17:36:33,116 - lava_callback - INFO - Received callback for node 69c02858c3d73288b0d08044 INFO: 172.20.0.1:49748 - "POST /node/69c02858c3d73288b0d08044 HTTP/1.0" 202 Accepted 2026-03-22 17:36:33,123 - lava_callback - INFO - Starting async job submit for node 69c02858c3d73288b0d08044 2026-03-22 17:36:33,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:33,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02858c3d73288b0d08044 HTTP/1.1" 200 None 2026-03-22 17:36:33,161 - lava_callback - DEBUG - Processing node 69c02858c3d73288b0d08044: baseline-arm64 2026-03-22 17:36:33,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:33,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:33,717 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02858c3d73288b0d08044/log.txt.gz 2026-03-22 17:36:33,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:34,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:34,181 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02858c3d73288b0d08044/lava-logs.yaml 2026-03-22 17:36:34,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:34,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:34,366 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02858c3d73288b0d08044/lava_callback.json.gz 2026-03-22 17:36:34,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:34,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02858c3d73288b0d08044 HTTP/1.1" 200 None 2026-03-22 17:36:34,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:34,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:34,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:34,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02858c3d73288b0d08044 HTTP/1.1" 200 None 2026-03-22 17:36:34,446 - lava_callback - INFO - Completed processing callback for node 69c02858c3d73288b0d08044 2026-03-22 17:36:35,120 - lava_callback - INFO - Received callback for node 69bd85970e93e2314723ed2f INFO: 172.20.0.1:49760 - "POST /node/69bd85970e93e2314723ed2f HTTP/1.0" 202 Accepted 2026-03-22 17:36:35,126 - lava_callback - INFO - Starting async job submit for node 69bd85970e93e2314723ed2f 2026-03-22 17:36:35,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:35,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd85970e93e2314723ed2f HTTP/1.1" 200 None 2026-03-22 17:36:35,142 - lava_callback - DEBUG - Processing node 69bd85970e93e2314723ed2f: baseline-arm64-clabbe 2026-03-22 17:36:35,143 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd85970e93e2314723ed2f 2026-03-22 17:36:35,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:35,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd85970e93e2314723ed2f HTTP/1.1" 200 None 2026-03-22 17:36:35,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:35,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:36:35,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:35,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd85970e93e2314723ed2f HTTP/1.1" 200 None 2026-03-22 17:36:35,198 - lava_callback - ERROR - Error processing callback for node 69bd85970e93e2314723ed2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 17:36:35,636 - lava_callback - INFO - Received callback for node 69c02857c3d73288b0d08041 INFO: 172.20.0.1:49762 - "POST /node/69c02857c3d73288b0d08041 HTTP/1.0" 202 Accepted 2026-03-22 17:36:35,642 - lava_callback - INFO - Starting async job submit for node 69c02857c3d73288b0d08041 2026-03-22 17:36:35,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:35,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02857c3d73288b0d08041 HTTP/1.1" 200 None 2026-03-22 17:36:35,676 - lava_callback - DEBUG - Processing node 69c02857c3d73288b0d08041: baseline-arm64 2026-03-22 17:36:36,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:36,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:36,173 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02857c3d73288b0d08041/log.txt.gz 2026-03-22 17:36:36,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:49774 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:36:36,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:36,599 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02857c3d73288b0d08041/lava-logs.yaml 2026-03-22 17:36:36,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:36,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:36,749 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02857c3d73288b0d08041/lava_callback.json.gz 2026-03-22 17:36:36,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:36,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02857c3d73288b0d08041 HTTP/1.1" 200 None 2026-03-22 17:36:36,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:36,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:36,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:36,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02857c3d73288b0d08041 HTTP/1.1" 200 None 2026-03-22 17:36:36,828 - lava_callback - INFO - Completed processing callback for node 69c02857c3d73288b0d08041 2026-03-22 17:36:38,776 - lava_callback - INFO - Received callback for node 69c0285ec3d73288b0d08054 INFO: 172.20.0.1:49778 - "POST /node/69c0285ec3d73288b0d08054 HTTP/1.0" 202 Accepted 2026-03-22 17:36:38,782 - lava_callback - INFO - Starting async job submit for node 69c0285ec3d73288b0d08054 2026-03-22 17:36:38,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:38,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0285ec3d73288b0d08054 HTTP/1.1" 200 None 2026-03-22 17:36:38,821 - lava_callback - DEBUG - Processing node 69c0285ec3d73288b0d08054: baseline-arm64 2026-03-22 17:36:39,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:39,190 - lava_callback - INFO - Received callback for node 69c0285ac3d73288b0d0804a INFO: 172.20.0.1:49788 - "POST /node/69c0285ac3d73288b0d0804a HTTP/1.0" 202 Accepted 2026-03-22 17:36:39,199 - lava_callback - INFO - Starting async job submit for node 69c0285ac3d73288b0d0804a 2026-03-22 17:36:39,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:39,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:39,224 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285ec3d73288b0d08054/log.txt.gz 2026-03-22 17:36:39,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0285ac3d73288b0d0804a HTTP/1.1" 200 None 2026-03-22 17:36:39,313 - lava_callback - DEBUG - Processing node 69c0285ac3d73288b0d0804a: baseline-arm64 2026-03-22 17:36:39,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:39,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:39,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:39,772 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285ac3d73288b0d0804a/log.txt.gz 2026-03-22 17:36:39,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:39,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285ec3d73288b0d08054/lava-logs.yaml 2026-03-22 17:36:39,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:39,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:40,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:40,077 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285ec3d73288b0d08054/lava_callback.json.gz 2026-03-22 17:36:40,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:40,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0285ec3d73288b0d08054 HTTP/1.1" 200 None 2026-03-22 17:36:40,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:40,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:40,131 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285ac3d73288b0d0804a/lava-logs.yaml 2026-03-22 17:36:40,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:40,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:40,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:40,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0285ec3d73288b0d08054 HTTP/1.1" 200 None 2026-03-22 17:36:40,189 - lava_callback - INFO - Completed processing callback for node 69c0285ec3d73288b0d08054 2026-03-22 17:36:40,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:40,361 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285ac3d73288b0d0804a/lava_callback.json.gz 2026-03-22 17:36:40,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:40,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0285ac3d73288b0d0804a HTTP/1.1" 200 None 2026-03-22 17:36:40,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:40,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:40,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:40,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0285ac3d73288b0d0804a HTTP/1.1" 200 None 2026-03-22 17:36:40,481 - lava_callback - INFO - Completed processing callback for node 69c0285ac3d73288b0d0804a INFO: 172.20.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36180 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:36:46,614 - lava_callback - INFO - Received callback for node 69c0284ec3d73288b0d08026 INFO: 172.20.0.1:36184 - "POST /node/69c0284ec3d73288b0d08026 HTTP/1.0" 202 Accepted 2026-03-22 17:36:46,620 - lava_callback - INFO - Starting async job submit for node 69c0284ec3d73288b0d08026 2026-03-22 17:36:46,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:46,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0284ec3d73288b0d08026 HTTP/1.1" 200 None 2026-03-22 17:36:46,656 - lava_callback - DEBUG - Processing node 69c0284ec3d73288b0d08026: baseline-arm64 2026-03-22 17:36:46,846 - lava_callback - INFO - Received callback for node 69c0285dc3d73288b0d08051 INFO: 172.20.0.1:36198 - "POST /node/69c0285dc3d73288b0d08051 HTTP/1.0" 202 Accepted 2026-03-22 17:36:46,872 - lava_callback - INFO - Starting async job submit for node 69c0285dc3d73288b0d08051 2026-03-22 17:36:46,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:46,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:46,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0285dc3d73288b0d08051 HTTP/1.1" 200 None 2026-03-22 17:36:46,930 - lava_callback - DEBUG - Processing node 69c0285dc3d73288b0d08051: baseline-arm64 2026-03-22 17:36:47,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:47,135 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284ec3d73288b0d08026/log.txt.gz 2026-03-22 17:36:47,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:47,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:47,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:47,552 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285dc3d73288b0d08051/log.txt.gz 2026-03-22 17:36:47,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:47,698 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284ec3d73288b0d08026/lava-logs.yaml 2026-03-22 17:36:47,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:47,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:47,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:47,901 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0284ec3d73288b0d08026/lava_callback.json.gz 2026-03-22 17:36:47,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:47,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0284ec3d73288b0d08026 HTTP/1.1" 200 None 2026-03-22 17:36:47,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:47,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:47,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:47,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:47,963 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285dc3d73288b0d08051/lava-logs.yaml 2026-03-22 17:36:47,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:47,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0284ec3d73288b0d08026 HTTP/1.1" 200 None 2026-03-22 17:36:47,994 - lava_callback - INFO - Completed processing callback for node 69c0284ec3d73288b0d08026 2026-03-22 17:36:48,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:48,184 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285dc3d73288b0d08051/lava_callback.json.gz 2026-03-22 17:36:48,216 - lava_callback - INFO - Received callback for node 69c02852c3d73288b0d08033 INFO: 172.20.0.1:36208 - "POST /node/69c02852c3d73288b0d08033 HTTP/1.0" 202 Accepted 2026-03-22 17:36:48,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:48,225 - lava_callback - INFO - Starting async job submit for node 69c02852c3d73288b0d08033 2026-03-22 17:36:48,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:48,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0285dc3d73288b0d08051 HTTP/1.1" 200 None 2026-03-22 17:36:48,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:48,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02852c3d73288b0d08033 HTTP/1.1" 200 None 2026-03-22 17:36:48,286 - lava_callback - DEBUG - Processing node 69c02852c3d73288b0d08033: baseline-arm64 2026-03-22 17:36:48,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:48,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:48,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0285dc3d73288b0d08051 HTTP/1.1" 200 None 2026-03-22 17:36:48,473 - lava_callback - INFO - Completed processing callback for node 69c0285dc3d73288b0d08051 2026-03-22 17:36:48,585 - lava_callback - WARNING - Failed to upload log 2026-03-22 17:36:48,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:48,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:48,878 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02852c3d73288b0d08033/lava-logs.yaml 2026-03-22 17:36:48,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:36:49,012 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:36:49,014 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02852c3d73288b0d08033/lava_callback.json.gz 2026-03-22 17:36:49,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:49,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02852c3d73288b0d08033 HTTP/1.1" 200 None 2026-03-22 17:36:49,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:49,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:36:49,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:36:49,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02852c3d73288b0d08033 HTTP/1.1" 200 None 2026-03-22 17:36:49,163 - lava_callback - INFO - Completed processing callback for node 69c02852c3d73288b0d08033 INFO: 172.20.0.1:58254 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:37:00,041 - lava_callback - INFO - Received callback for node 69c02867c3d73288b0d0806a INFO: 172.20.0.1:47742 - "POST /node/69c02867c3d73288b0d0806a HTTP/1.0" 202 Accepted 2026-03-22 17:37:00,049 - lava_callback - INFO - Starting async job submit for node 69c02867c3d73288b0d0806a 2026-03-22 17:37:00,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:00,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02867c3d73288b0d0806a HTTP/1.1" 200 None 2026-03-22 17:37:00,088 - lava_callback - DEBUG - Processing node 69c02867c3d73288b0d0806a: baseline-nfs-arm64 2026-03-22 17:37:00,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:00,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:00,753 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02867c3d73288b0d0806a/log.txt.gz 2026-03-22 17:37:01,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:01,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:01,268 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02867c3d73288b0d0806a/lava-logs.yaml 2026-03-22 17:37:01,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:01,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:01,439 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02867c3d73288b0d0806a/lava_callback.json.gz 2026-03-22 17:37:01,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:01,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02867c3d73288b0d0806a HTTP/1.1" 200 None 2026-03-22 17:37:01,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:01,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:01,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:01,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02867c3d73288b0d0806a HTTP/1.1" 200 None 2026-03-22 17:37:01,531 - lava_callback - INFO - Completed processing callback for node 69c02867c3d73288b0d0806a 2026-03-22 17:37:02,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:02,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-22 17:37:04,582 - lava_callback - INFO - Received callback for node 69c0286ac3d73288b0d08093 INFO: 172.20.0.1:47754 - "POST /node/69c0286ac3d73288b0d08093 HTTP/1.0" 202 Accepted 2026-03-22 17:37:04,588 - lava_callback - INFO - Starting async job submit for node 69c0286ac3d73288b0d08093 2026-03-22 17:37:04,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:04,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0286ac3d73288b0d08093 HTTP/1.1" 200 None 2026-03-22 17:37:04,638 - lava_callback - DEBUG - Processing node 69c0286ac3d73288b0d08093: baseline-nfs-arm64 2026-03-22 17:37:04,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:05,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:05,099 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286ac3d73288b0d08093/log.txt.gz 2026-03-22 17:37:05,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:05,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:05,487 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286ac3d73288b0d08093/lava-logs.yaml 2026-03-22 17:37:05,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:05,637 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:05,639 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286ac3d73288b0d08093/lava_callback.json.gz 2026-03-22 17:37:05,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:05,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0286ac3d73288b0d08093 HTTP/1.1" 200 None 2026-03-22 17:37:05,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:05,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:05,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:05,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0286ac3d73288b0d08093 HTTP/1.1" 200 None 2026-03-22 17:37:05,719 - lava_callback - INFO - Completed processing callback for node 69c0286ac3d73288b0d08093 INFO: 172.20.0.1:47766 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:37:09,235 - lava_callback - INFO - Received callback for node 69c0286dc3d73288b0d08099 INFO: 172.20.0.1:47774 - "POST /node/69c0286dc3d73288b0d08099 HTTP/1.0" 202 Accepted 2026-03-22 17:37:09,243 - lava_callback - INFO - Starting async job submit for node 69c0286dc3d73288b0d08099 2026-03-22 17:37:09,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:09,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0286dc3d73288b0d08099 HTTP/1.1" 200 None 2026-03-22 17:37:09,280 - lava_callback - DEBUG - Processing node 69c0286dc3d73288b0d08099: baseline-nfs-arm64 2026-03-22 17:37:09,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:09,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:09,756 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286dc3d73288b0d08099/log.txt.gz 2026-03-22 17:37:09,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:10,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:10,162 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286dc3d73288b0d08099/lava-logs.yaml 2026-03-22 17:37:10,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:10,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:10,352 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286dc3d73288b0d08099/lava_callback.json.gz 2026-03-22 17:37:10,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:10,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0286dc3d73288b0d08099 HTTP/1.1" 200 None 2026-03-22 17:37:10,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:10,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:10,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:10,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0286dc3d73288b0d08099 HTTP/1.1" 200 None 2026-03-22 17:37:10,443 - lava_callback - INFO - Completed processing callback for node 69c0286dc3d73288b0d08099 INFO: 172.20.0.1:46844 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-22 17:37:15,601 - lava_callback - INFO - Received callback for node 69c0286ec3d73288b0d0809c INFO: 172.20.0.1:46852 - "POST /node/69c0286ec3d73288b0d0809c HTTP/1.0" 202 Accepted 2026-03-22 17:37:15,611 - lava_callback - INFO - Starting async job submit for node 69c0286ec3d73288b0d0809c 2026-03-22 17:37:15,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:15,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0286ec3d73288b0d0809c HTTP/1.1" 200 None 2026-03-22 17:37:15,659 - lava_callback - DEBUG - Processing node 69c0286ec3d73288b0d0809c: baseline-nfs-arm64 2026-03-22 17:37:16,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:16,290 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:16,292 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286ec3d73288b0d0809c/log.txt.gz INFO: 172.20.0.1:46854 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:37:16,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:16,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:16,714 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286ec3d73288b0d0809c/lava-logs.yaml 2026-03-22 17:37:16,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:16,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:16,900 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286ec3d73288b0d0809c/lava_callback.json.gz 2026-03-22 17:37:16,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:16,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0286ec3d73288b0d0809c HTTP/1.1" 200 None 2026-03-22 17:37:16,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:16,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:16,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:16,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0286ec3d73288b0d0809c HTTP/1.1" 200 None 2026-03-22 17:37:16,998 - lava_callback - INFO - Completed processing callback for node 69c0286ec3d73288b0d0809c 2026-03-22 17:37:17,046 - lava_callback - INFO - Received callback for node 69c02897c3d73288b0d0811c INFO: 172.20.0.1:46856 - "POST /node/69c02897c3d73288b0d0811c HTTP/1.0" 202 Accepted 2026-03-22 17:37:17,057 - lava_callback - INFO - Starting async job submit for node 69c02897c3d73288b0d0811c 2026-03-22 17:37:17,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:17,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02897c3d73288b0d0811c HTTP/1.1" 200 None 2026-03-22 17:37:17,107 - lava_callback - DEBUG - Processing node 69c02897c3d73288b0d0811c: baseline-arm64-foundriesio 2026-03-22 17:37:17,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:17,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:17,503 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c02897c3d73288b0d0811c/log.txt.gz 2026-03-22 17:37:17,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:17,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:17,812 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c02897c3d73288b0d0811c/lava-logs.yaml 2026-03-22 17:37:17,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:17,974 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:17,976 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c02897c3d73288b0d0811c/lava_callback.json.gz 2026-03-22 17:37:17,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:18,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02897c3d73288b0d0811c HTTP/1.1" 200 None 2026-03-22 17:37:18,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:18,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:18,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:18,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02897c3d73288b0d0811c HTTP/1.1" 200 None 2026-03-22 17:37:18,064 - lava_callback - INFO - Completed processing callback for node 69c02897c3d73288b0d0811c 2026-03-22 17:37:19,498 - lava_callback - INFO - Received callback for node 69c0287cc3d73288b0d080ba INFO: 172.20.0.1:46868 - "POST /node/69c0287cc3d73288b0d080ba HTTP/1.0" 202 Accepted 2026-03-22 17:37:19,504 - lava_callback - INFO - Starting async job submit for node 69c0287cc3d73288b0d080ba 2026-03-22 17:37:19,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:19,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0287cc3d73288b0d080ba HTTP/1.1" 200 None 2026-03-22 17:37:19,542 - lava_callback - DEBUG - Processing node 69c0287cc3d73288b0d080ba: baseline-arm64-broonie 2026-03-22 17:37:19,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:19,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:19,876 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0287cc3d73288b0d080ba/log.txt.gz 2026-03-22 17:37:19,905 - lava_callback - INFO - Received callback for node 69c02868c3d73288b0d0806d INFO: 172.20.0.1:51616 - "POST /node/69c02868c3d73288b0d0806d HTTP/1.0" 202 Accepted 2026-03-22 17:37:19,926 - lava_callback - INFO - Starting async job submit for node 69c02868c3d73288b0d0806d 2026-03-22 17:37:19,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:20,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02868c3d73288b0d0806d HTTP/1.1" 200 None 2026-03-22 17:37:20,046 - lava_callback - DEBUG - Processing node 69c02868c3d73288b0d0806d: baseline-nfs-arm64 2026-03-22 17:37:20,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:20,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:20,423 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0287cc3d73288b0d080ba/lava-logs.yaml 2026-03-22 17:37:20,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:20,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:20,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:20,728 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0287cc3d73288b0d080ba/lava_callback.json.gz 2026-03-22 17:37:20,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:20,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0287cc3d73288b0d080ba HTTP/1.1" 200 None 2026-03-22 17:37:20,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:20,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:20,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:20,789 - lava_callback - INFO - Received callback for node 69c02893c3d73288b0d08113 2026-03-22 17:37:20,799 - lava_callback - INFO - Starting async job submit for node 69c02893c3d73288b0d08113 INFO: 172.20.0.1:51628 - "POST /node/69c02893c3d73288b0d08113 HTTP/1.0" 202 Accepted 2026-03-22 17:37:20,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:20,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:20,832 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02868c3d73288b0d0806d/log.txt.gz 2026-03-22 17:37:20,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0287cc3d73288b0d080ba HTTP/1.1" 200 None 2026-03-22 17:37:20,918 - lava_callback - INFO - Completed processing callback for node 69c0287cc3d73288b0d080ba 2026-03-22 17:37:20,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02893c3d73288b0d08113 HTTP/1.1" 200 None 2026-03-22 17:37:20,942 - lava_callback - DEBUG - Processing node 69c02893c3d73288b0d08113: baseline-arm64-pengutronix 2026-03-22 17:37:21,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:21,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:21,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:21,640 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c02893c3d73288b0d08113/log.txt.gz 2026-03-22 17:37:21,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:21,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:21,771 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02868c3d73288b0d0806d/lava-logs.yaml 2026-03-22 17:37:21,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:21,840 - lava_callback - INFO - Received callback for node 69c02862c3d73288b0d0805d INFO: 172.20.0.1:51636 - "POST /node/69c02862c3d73288b0d0805d HTTP/1.0" 202 Accepted 2026-03-22 17:37:21,847 - lava_callback - INFO - Starting async job submit for node 69c02862c3d73288b0d0805d 2026-03-22 17:37:21,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:21,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02862c3d73288b0d0805d HTTP/1.1" 200 None 2026-03-22 17:37:21,899 - lava_callback - DEBUG - Processing node 69c02862c3d73288b0d0805d: baseline-nfs-arm64 2026-03-22 17:37:21,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:21,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:22,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02868c3d73288b0d0806d/lava_callback.json.gz 2026-03-22 17:37:22,074 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c02893c3d73288b0d08113/lava-logs.yaml 2026-03-22 17:37:22,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:22,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:22,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02868c3d73288b0d0806d HTTP/1.1" 200 None 2026-03-22 17:37:22,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:22,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:22,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:22,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:22,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02868c3d73288b0d0806d HTTP/1.1" 200 None 2026-03-22 17:37:22,316 - lava_callback - INFO - Completed processing callback for node 69c02868c3d73288b0d0806d 2026-03-22 17:37:22,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:22,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02862c3d73288b0d0805d/log.txt.gz 2026-03-22 17:37:22,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:22,575 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c02893c3d73288b0d08113/lava_callback.json.gz 2026-03-22 17:37:22,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:22,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02893c3d73288b0d08113 HTTP/1.1" 200 None 2026-03-22 17:37:22,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:22,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:22,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:22,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:22,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02893c3d73288b0d08113 HTTP/1.1" 200 None 2026-03-22 17:37:22,838 - lava_callback - INFO - Completed processing callback for node 69c02893c3d73288b0d08113 2026-03-22 17:37:23,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:23,067 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02862c3d73288b0d0805d/lava-logs.yaml 2026-03-22 17:37:23,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:23,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:23,242 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02862c3d73288b0d0805d/lava_callback.json.gz 2026-03-22 17:37:23,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:23,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02862c3d73288b0d0805d HTTP/1.1" 200 None 2026-03-22 17:37:23,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:23,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:23,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:23,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02862c3d73288b0d0805d HTTP/1.1" 200 None 2026-03-22 17:37:23,338 - lava_callback - INFO - Completed processing callback for node 69c02862c3d73288b0d0805d INFO: 172.20.0.1:51638 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:37:27,893 - lava_callback - INFO - Received callback for node 69c02895c3d73288b0d08116 INFO: 172.20.0.1:51646 - "POST /node/69c02895c3d73288b0d08116 HTTP/1.0" 202 Accepted 2026-03-22 17:37:27,899 - lava_callback - INFO - Starting async job submit for node 69c02895c3d73288b0d08116 2026-03-22 17:37:27,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:27,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02895c3d73288b0d08116 HTTP/1.1" 200 None 2026-03-22 17:37:27,950 - lava_callback - DEBUG - Processing node 69c02895c3d73288b0d08116: baseline-arm64-pengutronix 2026-03-22 17:37:28,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:28,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:28,331 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c02895c3d73288b0d08116/log.txt.gz 2026-03-22 17:37:28,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:28,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:28,636 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c02895c3d73288b0d08116/lava-logs.yaml 2026-03-22 17:37:28,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:28,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:28,813 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c02895c3d73288b0d08116/lava_callback.json.gz 2026-03-22 17:37:28,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:28,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02895c3d73288b0d08116 HTTP/1.1" 200 None 2026-03-22 17:37:28,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:28,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:28,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:28,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02895c3d73288b0d08116 HTTP/1.1" 200 None 2026-03-22 17:37:28,895 - lava_callback - INFO - Completed processing callback for node 69c02895c3d73288b0d08116 2026-03-22 17:37:28,942 - lava_callback - INFO - Received callback for node 69c02864c3d73288b0d08063 INFO: 172.20.0.1:51662 - "POST /node/69c02864c3d73288b0d08063 HTTP/1.0" 202 Accepted 2026-03-22 17:37:28,954 - lava_callback - INFO - Starting async job submit for node 69c02864c3d73288b0d08063 2026-03-22 17:37:28,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:29,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02864c3d73288b0d08063 HTTP/1.1" 200 None 2026-03-22 17:37:29,006 - lava_callback - DEBUG - Processing node 69c02864c3d73288b0d08063: baseline-nfs-arm64 2026-03-22 17:37:29,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:29,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:29,544 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02864c3d73288b0d08063/log.txt.gz 2026-03-22 17:37:29,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:29,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:29,964 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02864c3d73288b0d08063/lava-logs.yaml 2026-03-22 17:37:29,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:30,109 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:30,111 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02864c3d73288b0d08063/lava_callback.json.gz 2026-03-22 17:37:30,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:30,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02864c3d73288b0d08063 HTTP/1.1" 200 None 2026-03-22 17:37:30,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:30,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:30,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:30,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02864c3d73288b0d08063 HTTP/1.1" 200 None 2026-03-22 17:37:30,220 - lava_callback - INFO - Completed processing callback for node 69c02864c3d73288b0d08063 2026-03-22 17:37:32,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:32,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:52570 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:37:38,300 - lava_callback - INFO - Received callback for node 69c02896c3d73288b0d08119 INFO: 172.20.0.1:52584 - "POST /node/69c02896c3d73288b0d08119 HTTP/1.0" 202 Accepted 2026-03-22 17:37:38,306 - lava_callback - INFO - Starting async job submit for node 69c02896c3d73288b0d08119 2026-03-22 17:37:38,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:38,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02896c3d73288b0d08119 HTTP/1.1" 200 None 2026-03-22 17:37:38,341 - lava_callback - DEBUG - Processing node 69c02896c3d73288b0d08119: baseline-arm64-pengutronix 2026-03-22 17:37:38,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:38,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:38,697 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c02896c3d73288b0d08119/log.txt.gz 2026-03-22 17:37:38,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:39,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:39,004 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c02896c3d73288b0d08119/lava-logs.yaml 2026-03-22 17:37:39,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:39,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:39,193 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c02896c3d73288b0d08119/lava_callback.json.gz 2026-03-22 17:37:39,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:39,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02896c3d73288b0d08119 HTTP/1.1" 200 None 2026-03-22 17:37:39,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:39,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:39,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:39,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02896c3d73288b0d08119 HTTP/1.1" 200 None 2026-03-22 17:37:39,276 - lava_callback - INFO - Completed processing callback for node 69c02896c3d73288b0d08119 2026-03-22 17:37:39,467 - lava_callback - INFO - Received callback for node 69c02884c3d73288b0d080d4 INFO: 172.20.0.1:52590 - "POST /node/69c02884c3d73288b0d080d4 HTTP/1.0" 202 Accepted 2026-03-22 17:37:39,473 - lava_callback - INFO - Starting async job submit for node 69c02884c3d73288b0d080d4 2026-03-22 17:37:39,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:39,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02884c3d73288b0d080d4 HTTP/1.1" 200 None 2026-03-22 17:37:39,521 - lava_callback - DEBUG - Processing node 69c02884c3d73288b0d080d4: baseline-arm64-broonie 2026-03-22 17:37:39,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:39,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:39,924 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02884c3d73288b0d080d4/log.txt.gz 2026-03-22 17:37:40,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:40,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:40,243 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02884c3d73288b0d080d4/lava-logs.yaml 2026-03-22 17:37:40,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:40,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:40,382 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02884c3d73288b0d080d4/lava_callback.json.gz 2026-03-22 17:37:40,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:40,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02884c3d73288b0d080d4 HTTP/1.1" 200 None 2026-03-22 17:37:40,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:40,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:40,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:40,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02884c3d73288b0d080d4 HTTP/1.1" 200 None 2026-03-22 17:37:40,465 - lava_callback - INFO - Completed processing callback for node 69c02884c3d73288b0d080d4 INFO: 172.20.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53256 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:37:50,937 - lava_callback - INFO - Received callback for node 69c02880c3d73288b0d080c3 INFO: 172.20.0.1:53458 - "POST /node/69c02880c3d73288b0d080c3 HTTP/1.0" 202 Accepted 2026-03-22 17:37:50,943 - lava_callback - INFO - Starting async job submit for node 69c02880c3d73288b0d080c3 2026-03-22 17:37:50,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:50,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02880c3d73288b0d080c3 HTTP/1.1" 200 None 2026-03-22 17:37:50,987 - lava_callback - DEBUG - Processing node 69c02880c3d73288b0d080c3: baseline-arm64-broonie 2026-03-22 17:37:51,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:51,382 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:51,384 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02880c3d73288b0d080c3/log.txt.gz 2026-03-22 17:37:51,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:51,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:51,769 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02880c3d73288b0d080c3/lava-logs.yaml 2026-03-22 17:37:51,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:51,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:51,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02880c3d73288b0d080c3/lava_callback.json.gz 2026-03-22 17:37:51,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:51,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02880c3d73288b0d080c3 HTTP/1.1" 200 None 2026-03-22 17:37:51,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:51,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:51,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:51,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02880c3d73288b0d080c3 HTTP/1.1" 200 None 2026-03-22 17:37:51,994 - lava_callback - INFO - Completed processing callback for node 69c02880c3d73288b0d080c3 2026-03-22 17:37:52,360 - lava_callback - INFO - Received callback for node 69c02882c3d73288b0d080ca INFO: 172.20.0.1:53470 - "POST /node/69c02882c3d73288b0d080ca HTTP/1.0" 202 Accepted 2026-03-22 17:37:52,367 - lava_callback - INFO - Starting async job submit for node 69c02882c3d73288b0d080ca 2026-03-22 17:37:52,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:52,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02882c3d73288b0d080ca HTTP/1.1" 200 None 2026-03-22 17:37:52,410 - lava_callback - DEBUG - Processing node 69c02882c3d73288b0d080ca: baseline-arm64-broonie 2026-03-22 17:37:52,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:52,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:52,784 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02882c3d73288b0d080ca/log.txt.gz 2026-03-22 17:37:52,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:53,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:53,155 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02882c3d73288b0d080ca/lava-logs.yaml 2026-03-22 17:37:53,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:53,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:53,302 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02882c3d73288b0d080ca/lava_callback.json.gz 2026-03-22 17:37:53,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:53,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02882c3d73288b0d080ca HTTP/1.1" 200 None 2026-03-22 17:37:53,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:53,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:53,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:53,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02882c3d73288b0d080ca HTTP/1.1" 200 None 2026-03-22 17:37:53,381 - lava_callback - INFO - Completed processing callback for node 69c02882c3d73288b0d080ca 2026-03-22 17:37:53,731 - lava_callback - INFO - Received callback for node 69c02888c3d73288b0d080f0 INFO: 172.20.0.1:53482 - "POST /node/69c02888c3d73288b0d080f0 HTTP/1.0" 202 Accepted 2026-03-22 17:37:53,738 - lava_callback - INFO - Starting async job submit for node 69c02888c3d73288b0d080f0 2026-03-22 17:37:53,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:53,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02888c3d73288b0d080f0 HTTP/1.1" 200 None 2026-03-22 17:37:53,784 - lava_callback - DEBUG - Processing node 69c02888c3d73288b0d080f0: baseline-arm64-broonie 2026-03-22 17:37:54,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:54,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:54,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02888c3d73288b0d080f0/log.txt.gz 2026-03-22 17:37:54,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:54,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:54,510 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02888c3d73288b0d080f0/lava-logs.yaml 2026-03-22 17:37:54,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:37:54,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:37:54,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02888c3d73288b0d080f0/lava_callback.json.gz 2026-03-22 17:37:54,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:54,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02888c3d73288b0d080f0 HTTP/1.1" 200 None 2026-03-22 17:37:54,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:54,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:37:54,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:54,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02888c3d73288b0d080f0 HTTP/1.1" 200 None 2026-03-22 17:37:54,758 - lava_callback - INFO - Completed processing callback for node 69c02888c3d73288b0d080f0 2026-03-22 17:37:55,272 - lava_callback - INFO - Received callback for node 69bd85e70e93e2314723ed36 INFO: 172.20.0.1:53486 - "POST /node/69bd85e70e93e2314723ed36 HTTP/1.0" 202 Accepted 2026-03-22 17:37:55,280 - lava_callback - INFO - Starting async job submit for node 69bd85e70e93e2314723ed36 2026-03-22 17:37:55,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:55,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd85e70e93e2314723ed36 HTTP/1.1" 200 None 2026-03-22 17:37:55,299 - lava_callback - DEBUG - Processing node 69bd85e70e93e2314723ed36: baseline-arm64-clabbe 2026-03-22 17:37:55,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd85e70e93e2314723ed36 2026-03-22 17:37:55,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:55,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd85e70e93e2314723ed36 HTTP/1.1" 200 None 2026-03-22 17:37:55,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:55,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:37:55,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:37:55,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd85e70e93e2314723ed36 HTTP/1.1" 200 None 2026-03-22 17:37:55,369 - lava_callback - ERROR - Error processing callback for node 69bd85e70e93e2314723ed36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53500 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:38:02,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:02,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:52462 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:38:10,944 - lava_callback - INFO - Received callback for node 69c02886c3d73288b0d080d7 INFO: 172.20.0.1:38784 - "POST /node/69c02886c3d73288b0d080d7 HTTP/1.0" 202 Accepted 2026-03-22 17:38:10,950 - lava_callback - INFO - Starting async job submit for node 69c02886c3d73288b0d080d7 2026-03-22 17:38:10,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:10,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02886c3d73288b0d080d7 HTTP/1.1" 200 None 2026-03-22 17:38:10,990 - lava_callback - DEBUG - Processing node 69c02886c3d73288b0d080d7: baseline-arm64-broonie 2026-03-22 17:38:11,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:11,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:11,414 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02886c3d73288b0d080d7/log.txt.gz 2026-03-22 17:38:11,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:11,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:11,751 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02886c3d73288b0d080d7/lava-logs.yaml 2026-03-22 17:38:11,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:11,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:11,905 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02886c3d73288b0d080d7/lava_callback.json.gz 2026-03-22 17:38:11,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:11,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02886c3d73288b0d080d7 HTTP/1.1" 200 None 2026-03-22 17:38:11,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:11,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:38:11,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:11,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02886c3d73288b0d080d7 HTTP/1.1" 200 None 2026-03-22 17:38:11,990 - lava_callback - INFO - Completed processing callback for node 69c02886c3d73288b0d080d7 INFO: 172.20.0.1:38792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38804 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:38:19,551 - lava_callback - INFO - Received callback for node 69c02889c3d73288b0d080f3 INFO: 172.20.0.1:38808 - "POST /node/69c02889c3d73288b0d080f3 HTTP/1.0" 202 Accepted 2026-03-22 17:38:19,557 - lava_callback - INFO - Starting async job submit for node 69c02889c3d73288b0d080f3 2026-03-22 17:38:19,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:19,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02889c3d73288b0d080f3 HTTP/1.1" 200 None 2026-03-22 17:38:19,594 - lava_callback - DEBUG - Processing node 69c02889c3d73288b0d080f3: baseline-arm64-broonie 2026-03-22 17:38:19,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:20,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:20,022 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02889c3d73288b0d080f3/log.txt.gz 2026-03-22 17:38:20,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:20,354 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:20,356 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02889c3d73288b0d080f3/lava-logs.yaml 2026-03-22 17:38:20,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:20,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:20,523 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02889c3d73288b0d080f3/lava_callback.json.gz 2026-03-22 17:38:20,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:20,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02889c3d73288b0d080f3 HTTP/1.1" 200 None 2026-03-22 17:38:20,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:20,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:38:20,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:20,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02889c3d73288b0d080f3 HTTP/1.1" 200 None 2026-03-22 17:38:20,603 - lava_callback - INFO - Completed processing callback for node 69c02889c3d73288b0d080f3 INFO: 172.20.0.1:36802 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:38:32,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:32,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35610 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:38:39,523 - lava_callback - INFO - Received callback for node 69c0287dc3d73288b0d080bd INFO: 172.20.0.1:35618 - "POST /node/69c0287dc3d73288b0d080bd HTTP/1.0" 202 Accepted 2026-03-22 17:38:39,529 - lava_callback - INFO - Starting async job submit for node 69c0287dc3d73288b0d080bd 2026-03-22 17:38:39,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:39,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0287dc3d73288b0d080bd HTTP/1.1" 200 None 2026-03-22 17:38:39,566 - lava_callback - DEBUG - Processing node 69c0287dc3d73288b0d080bd: baseline-arm64-broonie 2026-03-22 17:38:39,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:39,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:39,942 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0287dc3d73288b0d080bd/log.txt.gz 2026-03-22 17:38:40,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:40,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:40,273 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0287dc3d73288b0d080bd/lava-logs.yaml 2026-03-22 17:38:40,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:40,448 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:40,451 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0287dc3d73288b0d080bd/lava_callback.json.gz 2026-03-22 17:38:40,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:40,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0287dc3d73288b0d080bd HTTP/1.1" 200 None 2026-03-22 17:38:40,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:40,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:38:40,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:40,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0287dc3d73288b0d080bd HTTP/1.1" 200 None 2026-03-22 17:38:40,540 - lava_callback - INFO - Completed processing callback for node 69c0287dc3d73288b0d080bd INFO: 172.20.0.1:43434 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:38:41,738 - lava_callback - INFO - Received callback for node 69c02853c3d73288b0d08038 INFO: 172.20.0.1:43438 - "POST /node/69c02853c3d73288b0d08038 HTTP/1.0" 202 Accepted 2026-03-22 17:38:41,743 - lava_callback - INFO - Starting async job submit for node 69c02853c3d73288b0d08038 2026-03-22 17:38:41,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:41,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02853c3d73288b0d08038 HTTP/1.1" 200 None 2026-03-22 17:38:41,776 - lava_callback - DEBUG - Processing node 69c02853c3d73288b0d08038: baseline-arm64 2026-03-22 17:38:42,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:42,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:42,246 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02853c3d73288b0d08038/log.txt.gz 2026-03-22 17:38:42,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:42,688 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:42,690 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02853c3d73288b0d08038/lava-logs.yaml 2026-03-22 17:38:42,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:38:42,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:38:42,835 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02853c3d73288b0d08038/lava_callback.json.gz 2026-03-22 17:38:42,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:42,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02853c3d73288b0d08038 HTTP/1.1" 200 None 2026-03-22 17:38:42,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:42,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:38:42,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:38:42,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02853c3d73288b0d08038 HTTP/1.1" 200 None 2026-03-22 17:38:42,908 - lava_callback - INFO - Completed processing callback for node 69c02853c3d73288b0d08038 INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43846 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:39:02,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:39:02,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46062 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:39:31,190 - lava_callback - INFO - Received callback for node 69c02898c3d73288b0d0811f INFO: 172.20.0.1:42552 - "POST /node/69c02898c3d73288b0d0811f HTTP/1.0" 202 Accepted 2026-03-22 17:39:31,199 - lava_callback - INFO - Starting async job submit for node 69c02898c3d73288b0d0811f 2026-03-22 17:39:31,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:39:31,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02898c3d73288b0d0811f HTTP/1.1" 200 None 2026-03-22 17:39:31,253 - lava_callback - DEBUG - Processing node 69c02898c3d73288b0d0811f: baseline-arm64-foundriesio 2026-03-22 17:39:32,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:39:32,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:39:32,267 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c02898c3d73288b0d0811f/log.txt.gz 2026-03-22 17:39:32,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:39:32,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:39:32,999 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c02898c3d73288b0d0811f/lava-logs.yaml 2026-03-22 17:39:33,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:39:33,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:39:33,222 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c02898c3d73288b0d0811f/lava_callback.json.gz DEBUG: 69c02898c3d73288b0d0811f Transitting job node result: pass -> incomplete 2026-03-22 17:39:33,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:39:33,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02898c3d73288b0d0811f HTTP/1.1" 200 None 2026-03-22 17:39:33,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:39:33,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:39:33,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:39:33,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02898c3d73288b0d0811f HTTP/1.1" 200 None 2026-03-22 17:39:33,323 - lava_callback - INFO - Completed processing callback for node 69c02898c3d73288b0d0811f INFO: 172.20.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48456 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:39:49,926 - lava_callback - INFO - Received callback for node 69c02567c3d73288b0d07f6f INFO: 172.20.0.1:36390 - "POST /node/69c02567c3d73288b0d07f6f HTTP/1.0" 202 Accepted 2026-03-22 17:39:49,933 - lava_callback - INFO - Starting async job submit for node 69c02567c3d73288b0d07f6f 2026-03-22 17:39:49,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:39:50,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02567c3d73288b0d07f6f HTTP/1.1" 200 None 2026-03-22 17:39:50,007 - lava_callback - DEBUG - Processing node 69c02567c3d73288b0d07f6f: baseline-arm 2026-03-22 17:39:50,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:39:50,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:39:50,714 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c02567c3d73288b0d07f6f/log.txt.gz 2026-03-22 17:39:51,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:39:51,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:39:51,329 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c02567c3d73288b0d07f6f/lava-logs.yaml 2026-03-22 17:39:51,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:39:51,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:39:51,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c02567c3d73288b0d07f6f/lava_callback.json.gz DEBUG: Setup failed for 69c02567c3d73288b0d07f6f. Transitting job node result: incomplete -> fail 2026-03-22 17:39:51,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:39:51,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02567c3d73288b0d07f6f HTTP/1.1" 200 None 2026-03-22 17:39:51,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:39:51,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 17:39:51,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:39:51,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02567c3d73288b0d07f6f HTTP/1.1" 200 None 2026-03-22 17:39:51,807 - lava_callback - INFO - Completed processing callback for node 69c02567c3d73288b0d07f6f INFO: 172.20.0.1:36402 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:40:00,154 - lava_callback - INFO - Received callback for node 69c02883c3d73288b0d080d1 INFO: 172.20.0.1:43618 - "POST /node/69c02883c3d73288b0d080d1 HTTP/1.0" 202 Accepted 2026-03-22 17:40:00,160 - lava_callback - INFO - Starting async job submit for node 69c02883c3d73288b0d080d1 2026-03-22 17:40:00,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:40:00,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02883c3d73288b0d080d1 HTTP/1.1" 200 None 2026-03-22 17:40:00,198 - lava_callback - DEBUG - Processing node 69c02883c3d73288b0d080d1: baseline-arm64-broonie 2026-03-22 17:40:00,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:40:00,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:40:00,669 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02883c3d73288b0d080d1/log.txt.gz 2026-03-22 17:40:00,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:40:01,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:40:01,009 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02883c3d73288b0d080d1/lava-logs.yaml 2026-03-22 17:40:01,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:40:01,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:40:01,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c02883c3d73288b0d080d1/lava_callback.json.gz 2026-03-22 17:40:01,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:40:01,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02883c3d73288b0d080d1 HTTP/1.1" 200 None 2026-03-22 17:40:01,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:40:01,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:40:01,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:40:01,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02883c3d73288b0d080d1 HTTP/1.1" 200 None 2026-03-22 17:40:01,250 - lava_callback - INFO - Completed processing callback for node 69c02883c3d73288b0d080d1 2026-03-22 17:40:02,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:40:02,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57036 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:40:35,319 - lava_callback - INFO - Received callback for node 69bd868a0e93e2314723ed4e INFO: 172.20.0.1:55212 - "POST /node/69bd868a0e93e2314723ed4e HTTP/1.0" 202 Accepted 2026-03-22 17:40:35,325 - lava_callback - INFO - Starting async job submit for node 69bd868a0e93e2314723ed4e 2026-03-22 17:40:35,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:40:35,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd868a0e93e2314723ed4e HTTP/1.1" 200 None 2026-03-22 17:40:35,340 - lava_callback - DEBUG - Processing node 69bd868a0e93e2314723ed4e: baseline-arm64-clabbe 2026-03-22 17:40:35,340 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd868a0e93e2314723ed4e 2026-03-22 17:40:35,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:40:35,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd868a0e93e2314723ed4e HTTP/1.1" 200 None 2026-03-22 17:40:35,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:40:35,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:40:35,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:40:35,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd868a0e93e2314723ed4e HTTP/1.1" 200 None 2026-03-22 17:40:35,397 - lava_callback - ERROR - Error processing callback for node 69bd868a0e93e2314723ed4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58204 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:41:55,311 - lava_callback - INFO - Received callback for node 69bd86da0e93e2314723ed58 INFO: 172.20.0.1:53048 - "POST /node/69bd86da0e93e2314723ed58 HTTP/1.0" 202 Accepted 2026-03-22 17:41:55,317 - lava_callback - INFO - Starting async job submit for node 69bd86da0e93e2314723ed58 2026-03-22 17:41:55,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:41:55,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd86da0e93e2314723ed58 HTTP/1.1" 200 None 2026-03-22 17:41:55,333 - lava_callback - DEBUG - Processing node 69bd86da0e93e2314723ed58: baseline-arm64-clabbe 2026-03-22 17:41:55,334 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd86da0e93e2314723ed58 2026-03-22 17:41:55,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:41:55,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd86da0e93e2314723ed58 HTTP/1.1" 200 None 2026-03-22 17:41:55,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:41:55,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 17:41:55,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:41:55,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd86da0e93e2314723ed58 HTTP/1.1" 200 None 2026-03-22 17:41:55,395 - lava_callback - ERROR - Error processing callback for node 69bd86da0e93e2314723ed58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41446 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:42:31,253 - lava_callback - INFO - Received callback for node 69c02863c3d73288b0d08060 INFO: 172.20.0.1:39892 - "POST /node/69c02863c3d73288b0d08060 HTTP/1.0" 202 Accepted 2026-03-22 17:42:31,259 - lava_callback - INFO - Starting async job submit for node 69c02863c3d73288b0d08060 2026-03-22 17:42:31,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:42:31,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02863c3d73288b0d08060 HTTP/1.1" 200 None 2026-03-22 17:42:31,294 - lava_callback - DEBUG - Processing node 69c02863c3d73288b0d08060: baseline-nfs-arm64 2026-03-22 17:42:31,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:42:31,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:42:31,849 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02863c3d73288b0d08060/log.txt.gz 2026-03-22 17:42:32,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:42:32,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:42:32,298 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02863c3d73288b0d08060/lava-logs.yaml 2026-03-22 17:42:32,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:42:32,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:42:32,478 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02863c3d73288b0d08060/lava_callback.json.gz 2026-03-22 17:42:32,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:42:32,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02863c3d73288b0d08060 HTTP/1.1" 200 None 2026-03-22 17:42:32,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:42:32,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:42:32,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:42:32,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02863c3d73288b0d08060 HTTP/1.1" 200 None 2026-03-22 17:42:32,565 - lava_callback - INFO - Completed processing callback for node 69c02863c3d73288b0d08060 2026-03-22 17:42:32,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:42:32,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52844 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:42:54,864 - lava_callback - INFO - Received callback for node 69c02a0cc3d73288b0d08261 INFO: 172.20.0.1:48406 - "POST /node/69c02a0cc3d73288b0d08261 HTTP/1.0" 202 Accepted 2026-03-22 17:42:54,870 - lava_callback - INFO - Starting async job submit for node 69c02a0cc3d73288b0d08261 2026-03-22 17:42:54,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:42:54,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02a0cc3d73288b0d08261 HTTP/1.1" 200 None 2026-03-22 17:42:54,887 - lava_callback - DEBUG - Processing node 69c02a0cc3d73288b0d08261: baseline-nfs-arm64 2026-03-22 17:42:54,889 - lava_callback - WARNING - Failed to upload log 2026-03-22 17:42:54,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:42:55,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:42:55,004 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02a0cc3d73288b0d08261/lava-logs.yaml 2026-03-22 17:42:55,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:42:55,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:42:55,117 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02a0cc3d73288b0d08261/lava_callback.json.gz 2026-03-22 17:42:55,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:42:55,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02a0cc3d73288b0d08261 HTTP/1.1" 200 None 2026-03-22 17:42:55,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:42:55,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:42:55,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:42:55,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02a0cc3d73288b0d08261 HTTP/1.1" 200 None 2026-03-22 17:42:55,182 - lava_callback - INFO - Completed processing callback for node 69c02a0cc3d73288b0d08261 INFO: 172.20.0.1:48416 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:43:02,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:02,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58158 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:43:09,668 - lava_callback - INFO - Received callback for node 69c02860c3d73288b0d0805a INFO: 172.20.0.1:58174 - "POST /node/69c02860c3d73288b0d0805a HTTP/1.0" 202 Accepted 2026-03-22 17:43:09,675 - lava_callback - INFO - Starting async job submit for node 69c02860c3d73288b0d0805a 2026-03-22 17:43:09,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:09,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02860c3d73288b0d0805a HTTP/1.1" 200 None 2026-03-22 17:43:09,727 - lava_callback - DEBUG - Processing node 69c02860c3d73288b0d0805a: baseline-arm64 2026-03-22 17:43:10,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:43:10,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:43:10,223 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02860c3d73288b0d0805a/log.txt.gz 2026-03-22 17:43:10,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:43:10,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:43:10,654 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02860c3d73288b0d0805a/lava-logs.yaml 2026-03-22 17:43:10,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:43:10,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:43:10,816 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02860c3d73288b0d0805a/lava_callback.json.gz DEBUG: Setup failed for 69c02860c3d73288b0d0805a. Transitting job node result: incomplete -> fail 2026-03-22 17:43:10,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:10,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02860c3d73288b0d0805a HTTP/1.1" 200 None 2026-03-22 17:43:10,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:10,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:43:10,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:10,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02860c3d73288b0d0805a HTTP/1.1" 200 None 2026-03-22 17:43:11,030 - lava_callback - INFO - Completed processing callback for node 69c02860c3d73288b0d0805a 2026-03-22 17:43:11,236 - lava_callback - INFO - Received callback for node 69bd87240e93e2314723ed60 INFO: 172.20.0.1:51186 - "POST /node/69bd87240e93e2314723ed60 HTTP/1.0" 202 Accepted 2026-03-22 17:43:11,242 - lava_callback - INFO - Starting async job submit for node 69bd87240e93e2314723ed60 2026-03-22 17:43:11,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:11,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd87240e93e2314723ed60 HTTP/1.1" 200 None 2026-03-22 17:43:11,258 - lava_callback - DEBUG - Processing node 69bd87240e93e2314723ed60: baseline-arm64-baylibre 2026-03-22 17:43:11,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd87240e93e2314723ed60 2026-03-22 17:43:11,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:11,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd87240e93e2314723ed60 HTTP/1.1" 200 None 2026-03-22 17:43:11,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:11,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 17:43:11,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:11,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd87240e93e2314723ed60 HTTP/1.1" 200 None 2026-03-22 17:43:11,313 - lava_callback - ERROR - Error processing callback for node 69bd87240e93e2314723ed60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44446 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:43:32,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:32,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52650 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:43:58,863 - lava_callback - INFO - Received callback for node 69c025dac3d73288b0d07f8f INFO: 172.20.0.1:52658 - "POST /node/69c025dac3d73288b0d07f8f HTTP/1.0" 202 Accepted 2026-03-22 17:43:58,913 - lava_callback - INFO - Starting async job submit for node 69c025dac3d73288b0d07f8f 2026-03-22 17:43:58,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:58,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c025dac3d73288b0d07f8f HTTP/1.1" 200 None 2026-03-22 17:43:58,953 - lava_callback - DEBUG - Processing node 69c025dac3d73288b0d07f8f: baseline-arm-pengutronix 2026-03-22 17:43:59,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:43:59,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:43:59,268 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c025dac3d73288b0d07f8f/log.txt.gz 2026-03-22 17:43:59,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:43:59,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:43:59,560 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c025dac3d73288b0d07f8f/lava-logs.yaml 2026-03-22 17:43:59,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:43:59,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:43:59,702 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c025dac3d73288b0d07f8f/lava_callback.json.gz DEBUG: Setup failed for 69c025dac3d73288b0d07f8f. Transitting job node result: incomplete -> fail 2026-03-22 17:43:59,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:59,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c025dac3d73288b0d07f8f HTTP/1.1" 200 None 2026-03-22 17:43:59,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:59,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 17:43:59,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:43:59,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c025dac3d73288b0d07f8f HTTP/1.1" 200 None 2026-03-22 17:43:59,875 - lava_callback - INFO - Completed processing callback for node 69c025dac3d73288b0d07f8f 2026-03-22 17:44:02,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:44:03,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37108 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:44:43,027 - lava_callback - INFO - Received callback for node 69c02a4fc3d73288b0d08276 INFO: 172.20.0.1:37120 - "POST /node/69c02a4fc3d73288b0d08276 HTTP/1.0" 202 Accepted 2026-03-22 17:44:43,034 - lava_callback - INFO - Starting async job submit for node 69c02a4fc3d73288b0d08276 2026-03-22 17:44:43,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:44:43,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02a4fc3d73288b0d08276 HTTP/1.1" 200 None 2026-03-22 17:44:43,080 - lava_callback - DEBUG - Processing node 69c02a4fc3d73288b0d08276: baseline-arm64 2026-03-22 17:44:43,216 - lava_callback - WARNING - Failed to upload log 2026-03-22 17:44:43,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:44:43,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:44:43,479 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02a4fc3d73288b0d08276/lava-logs.yaml 2026-03-22 17:44:43,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:44:43,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:44:43,640 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02a4fc3d73288b0d08276/lava_callback.json.gz 2026-03-22 17:44:43,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:44:43,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02a4fc3d73288b0d08276 HTTP/1.1" 200 None 2026-03-22 17:44:43,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:44:43,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:44:43,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:44:43,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02a4fc3d73288b0d08276 HTTP/1.1" 200 None 2026-03-22 17:44:43,793 - lava_callback - INFO - Completed processing callback for node 69c02a4fc3d73288b0d08276 INFO: 172.20.0.1:37126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40208 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:45:03,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:45:03,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51378 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:46:04,003 - lava_callback - INFO - Received callback for node 69c0285bc3d73288b0d0804e INFO: 172.20.0.1:39096 - "POST /node/69c0285bc3d73288b0d0804e HTTP/1.0" 202 Accepted 2026-03-22 17:46:04,010 - lava_callback - INFO - Starting async job submit for node 69c0285bc3d73288b0d0804e 2026-03-22 17:46:04,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:46:04,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0285bc3d73288b0d0804e HTTP/1.1" 200 None 2026-03-22 17:46:04,049 - lava_callback - DEBUG - Processing node 69c0285bc3d73288b0d0804e: baseline-arm64 2026-03-22 17:46:04,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:46:04,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:46:04,726 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285bc3d73288b0d0804e/log.txt.gz 2026-03-22 17:46:04,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:46:05,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:46:05,246 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285bc3d73288b0d0804e/lava-logs.yaml 2026-03-22 17:46:05,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:46:05,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:46:05,440 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285bc3d73288b0d0804e/lava_callback.json.gz 2026-03-22 17:46:05,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:46:05,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0285bc3d73288b0d0804e HTTP/1.1" 200 None 2026-03-22 17:46:05,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:46:05,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:46:05,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:46:05,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0285bc3d73288b0d0804e HTTP/1.1" 200 None 2026-03-22 17:46:05,578 - lava_callback - INFO - Completed processing callback for node 69c0285bc3d73288b0d0804e INFO: 172.20.0.1:39100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43386 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:46:33,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:46:33,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35830 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:47:50,010 - lava_callback - INFO - Received callback for node 69c02ae4c3d73288b0d08297 INFO: 172.20.0.1:34900 - "POST /node/69c02ae4c3d73288b0d08297 HTTP/1.0" 202 Accepted 2026-03-22 17:47:50,017 - lava_callback - INFO - Starting async job submit for node 69c02ae4c3d73288b0d08297 2026-03-22 17:47:50,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:47:50,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02ae4c3d73288b0d08297 HTTP/1.1" 200 None 2026-03-22 17:47:50,065 - lava_callback - DEBUG - Processing node 69c02ae4c3d73288b0d08297: baseline-arm64 2026-03-22 17:47:50,282 - lava_callback - WARNING - Failed to upload log 2026-03-22 17:47:50,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:47:50,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:47:50,593 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02ae4c3d73288b0d08297/lava-logs.yaml 2026-03-22 17:47:50,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:47:50,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:47:50,728 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02ae4c3d73288b0d08297/lava_callback.json.gz 2026-03-22 17:47:50,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:47:50,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02ae4c3d73288b0d08297 HTTP/1.1" 200 None 2026-03-22 17:47:50,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:47:50,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:47:50,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:47:50,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02ae4c3d73288b0d08297 HTTP/1.1" 200 None 2026-03-22 17:47:50,870 - lava_callback - INFO - Completed processing callback for node 69c02ae4c3d73288b0d08297 2026-03-22 17:47:51,499 - lava_callback - INFO - Received callback for node 69bd88380e93e2314723ed9a INFO: 172.20.0.1:34910 - "POST /node/69bd88380e93e2314723ed9a HTTP/1.0" 202 Accepted 2026-03-22 17:47:51,505 - lava_callback - INFO - Starting async job submit for node 69bd88380e93e2314723ed9a 2026-03-22 17:47:51,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:47:51,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd88380e93e2314723ed9a HTTP/1.1" 200 None 2026-03-22 17:47:51,524 - lava_callback - DEBUG - Processing node 69bd88380e93e2314723ed9a: baseline-arm64-baylibre 2026-03-22 17:47:51,524 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd88380e93e2314723ed9a 2026-03-22 17:47:51,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:47:51,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd88380e93e2314723ed9a HTTP/1.1" 200 None 2026-03-22 17:47:51,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:47:51,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 17:47:51,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:47:51,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd88380e93e2314723ed9a HTTP/1.1" 200 None 2026-03-22 17:47:51,575 - lava_callback - ERROR - Error processing callback for node 69bd88380e93e2314723ed9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34926 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:48:03,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:48:03,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35438 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:49:11,608 - lava_callback - INFO - Received callback for node 69bd88830e93e2314723eda7 INFO: 172.20.0.1:35296 - "POST /node/69bd88830e93e2314723eda7 HTTP/1.0" 202 Accepted 2026-03-22 17:49:11,614 - lava_callback - INFO - Starting async job submit for node 69bd88830e93e2314723eda7 2026-03-22 17:49:11,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:49:11,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd88830e93e2314723eda7 HTTP/1.1" 200 None 2026-03-22 17:49:11,629 - lava_callback - DEBUG - Processing node 69bd88830e93e2314723eda7: baseline-arm64-baylibre 2026-03-22 17:49:11,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd88830e93e2314723eda7 2026-03-22 17:49:11,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:49:11,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd88830e93e2314723eda7 HTTP/1.1" 200 None 2026-03-22 17:49:11,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:49:11,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 17:49:11,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:49:11,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd88830e93e2314723eda7 HTTP/1.1" 200 None 2026-03-22 17:49:11,684 - lava_callback - ERROR - Error processing callback for node 69bd88830e93e2314723eda7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42360 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:49:58,952 - lava_callback - INFO - Received callback for node 69c0285fc3d73288b0d08057 INFO: 172.20.0.1:42376 - "POST /node/69c0285fc3d73288b0d08057 HTTP/1.0" 202 Accepted 2026-03-22 17:49:58,959 - lava_callback - INFO - Starting async job submit for node 69c0285fc3d73288b0d08057 2026-03-22 17:49:59,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:49:59,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0285fc3d73288b0d08057 HTTP/1.1" 200 None 2026-03-22 17:49:59,021 - lava_callback - DEBUG - Processing node 69c0285fc3d73288b0d08057: baseline-arm64 2026-03-22 17:49:59,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:49:59,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:49:59,988 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285fc3d73288b0d08057/log.txt.gz 2026-03-22 17:50:00,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:50:00,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:50:00,701 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285fc3d73288b0d08057/lava-logs.yaml 2026-03-22 17:50:00,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:50:00,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:50:00,894 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0285fc3d73288b0d08057/lava_callback.json.gz DEBUG: Setup failed for 69c0285fc3d73288b0d08057. Transitting job node result: incomplete -> fail 2026-03-22 17:50:00,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:01,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0285fc3d73288b0d08057 HTTP/1.1" 200 None 2026-03-22 17:50:01,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:01,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:50:01,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:01,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0285fc3d73288b0d08057 HTTP/1.1" 200 None 2026-03-22 17:50:01,171 - lava_callback - INFO - Completed processing callback for node 69c0285fc3d73288b0d08057 2026-03-22 17:50:03,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:03,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52976 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:50:21,811 - lava_callback - INFO - Received callback for node 69c02b8fc3d73288b0d082b4 INFO: 172.20.0.1:35588 - "POST /node/69c02b8fc3d73288b0d082b4 HTTP/1.0" 202 Accepted 2026-03-22 17:50:21,818 - lava_callback - INFO - Starting async job submit for node 69c02b8fc3d73288b0d082b4 2026-03-22 17:50:21,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:21,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02b8fc3d73288b0d082b4 HTTP/1.1" 200 None 2026-03-22 17:50:21,872 - lava_callback - DEBUG - Processing node 69c02b8fc3d73288b0d082b4: baseline-arm64-foundriesio 2026-03-22 17:50:22,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:50:22,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:50:22,469 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c02b8fc3d73288b0d082b4/log.txt.gz 2026-03-22 17:50:22,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:50:22,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:50:22,958 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c02b8fc3d73288b0d082b4/lava-logs.yaml 2026-03-22 17:50:22,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:50:23,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:50:23,139 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c02b8fc3d73288b0d082b4/lava_callback.json.gz 2026-03-22 17:50:23,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:23,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02b8fc3d73288b0d082b4 HTTP/1.1" 200 None 2026-03-22 17:50:23,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:23,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:50:23,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:23,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02b8fc3d73288b0d082b4 HTTP/1.1" 200 None 2026-03-22 17:50:23,249 - lava_callback - INFO - Completed processing callback for node 69c02b8fc3d73288b0d082b4 2026-03-22 17:50:24,104 - lava_callback - INFO - Received callback for node 69c0286fc3d73288b0d0809f INFO: 172.20.0.1:35604 - "POST /node/69c0286fc3d73288b0d0809f HTTP/1.0" 202 Accepted 2026-03-22 17:50:24,113 - lava_callback - INFO - Starting async job submit for node 69c0286fc3d73288b0d0809f 2026-03-22 17:50:24,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:24,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0286fc3d73288b0d0809f HTTP/1.1" 200 None 2026-03-22 17:50:24,179 - lava_callback - DEBUG - Processing node 69c0286fc3d73288b0d0809f: baseline-nfs-arm64 2026-03-22 17:50:24,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:50:25,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:50:25,094 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286fc3d73288b0d0809f/log.txt.gz 2026-03-22 17:50:25,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:50:25,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:50:25,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286fc3d73288b0d0809f/lava-logs.yaml 2026-03-22 17:50:25,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:50:26,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:50:26,009 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286fc3d73288b0d0809f/lava_callback.json.gz DEBUG: Setup failed for 69c0286fc3d73288b0d0809f. Transitting job node result: incomplete -> fail 2026-03-22 17:50:26,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:26,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0286fc3d73288b0d0809f HTTP/1.1" 200 None 2026-03-22 17:50:26,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:26,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:50:26,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:26,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0286fc3d73288b0d0809f HTTP/1.1" 200 None 2026-03-22 17:50:26,269 - lava_callback - INFO - Completed processing callback for node 69c0286fc3d73288b0d0809f INFO: 172.20.0.1:35616 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:50:33,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:33,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47658 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:50:51,908 - lava_callback - INFO - Received callback for node 69bd88f60e93e2314724147c INFO: 172.20.0.1:33382 - "POST /node/69bd88f60e93e2314724147c HTTP/1.0" 202 Accepted 2026-03-22 17:50:51,914 - lava_callback - INFO - Starting async job submit for node 69bd88f60e93e2314724147c 2026-03-22 17:50:51,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:51,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd88f60e93e2314724147c HTTP/1.1" 200 None 2026-03-22 17:50:51,931 - lava_callback - DEBUG - Processing node 69bd88f60e93e2314724147c: baseline-arm64-baylibre 2026-03-22 17:50:51,931 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd88f60e93e2314724147c 2026-03-22 17:50:51,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:51,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd88f60e93e2314724147c HTTP/1.1" 200 None 2026-03-22 17:50:51,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:51,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 17:50:51,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:50:51,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd88f60e93e2314724147c HTTP/1.1" 200 None 2026-03-22 17:50:51,982 - lava_callback - ERROR - Error processing callback for node 69bd88f60e93e2314724147c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43374 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:51:34,086 - lava_callback - INFO - Received callback for node 69c0288fc3d73288b0d0810a INFO: 172.20.0.1:39256 - "POST /node/69c0288fc3d73288b0d0810a HTTP/1.0" 202 Accepted 2026-03-22 17:51:34,093 - lava_callback - INFO - Starting async job submit for node 69c0288fc3d73288b0d0810a 2026-03-22 17:51:34,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:51:34,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0288fc3d73288b0d0810a HTTP/1.1" 200 None 2026-03-22 17:51:34,149 - lava_callback - DEBUG - Processing node 69c0288fc3d73288b0d0810a: baseline-arm64-clabbe 2026-03-22 17:51:34,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:51:34,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:51:34,920 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c0288fc3d73288b0d0810a/log.txt.gz 2026-03-22 17:51:35,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:51:35,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:51:35,545 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c0288fc3d73288b0d0810a/lava-logs.yaml 2026-03-22 17:51:35,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:51:35,733 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:51:35,735 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c0288fc3d73288b0d0810a/lava_callback.json.gz 2026-03-22 17:51:35,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:51:35,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0288fc3d73288b0d0810a HTTP/1.1" 200 None 2026-03-22 17:51:35,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:51:35,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:51:35,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:51:35,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0288fc3d73288b0d0810a HTTP/1.1" 200 None 2026-03-22 17:51:35,854 - lava_callback - INFO - Completed processing callback for node 69c0288fc3d73288b0d0810a INFO: 172.20.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47550 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:52:03,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:52:03,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35264 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:52:11,731 - lava_callback - INFO - Received callback for node 69bd89450e93e23147241483 INFO: 172.20.0.1:60066 - "POST /node/69bd89450e93e23147241483 HTTP/1.0" 202 Accepted 2026-03-22 17:52:11,736 - lava_callback - INFO - Starting async job submit for node 69bd89450e93e23147241483 2026-03-22 17:52:11,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:52:11,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd89450e93e23147241483 HTTP/1.1" 200 None 2026-03-22 17:52:11,760 - lava_callback - DEBUG - Processing node 69bd89450e93e23147241483: baseline-arm64-baylibre 2026-03-22 17:52:11,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd89450e93e23147241483 2026-03-22 17:52:11,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:52:11,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd89450e93e23147241483 HTTP/1.1" 200 None 2026-03-22 17:52:11,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:52:11,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 17:52:11,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:52:11,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd89450e93e23147241483 HTTP/1.1" 200 None 2026-03-22 17:52:11,828 - lava_callback - ERROR - Error processing callback for node 69bd89450e93e23147241483: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49560 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:54:46,449 - lava_callback - INFO - Received callback for node 69c02cc9c3d73288b0d0830c INFO: 172.20.0.1:49570 - "POST /node/69c02cc9c3d73288b0d0830c HTTP/1.0" 202 Accepted 2026-03-22 17:54:46,455 - lava_callback - INFO - Starting async job submit for node 69c02cc9c3d73288b0d0830c 2026-03-22 17:54:46,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:54:46,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02cc9c3d73288b0d0830c HTTP/1.1" 200 None 2026-03-22 17:54:46,480 - lava_callback - DEBUG - Processing node 69c02cc9c3d73288b0d0830c: baseline-nfs-arm64 2026-03-22 17:54:46,482 - lava_callback - WARNING - Failed to upload log 2026-03-22 17:54:46,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:49580 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:54:47,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:54:47,317 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02cc9c3d73288b0d0830c/lava-logs.yaml 2026-03-22 17:54:47,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:54:47,479 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:54:47,481 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02cc9c3d73288b0d0830c/lava_callback.json.gz 2026-03-22 17:54:47,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:54:47,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02cc9c3d73288b0d0830c HTTP/1.1" 200 None 2026-03-22 17:54:47,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:54:47,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:54:47,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:54:47,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02cc9c3d73288b0d0830c HTTP/1.1" 200 None 2026-03-22 17:54:47,536 - lava_callback - INFO - Completed processing callback for node 69c02cc9c3d73288b0d0830c 2026-03-22 17:54:55,853 - lava_callback - INFO - Received callback for node 69bd89e90e93e231472414a3 INFO: 172.20.0.1:56968 - "POST /node/69bd89e90e93e231472414a3 HTTP/1.0" 202 Accepted 2026-03-22 17:54:55,859 - lava_callback - INFO - Starting async job submit for node 69bd89e90e93e231472414a3 2026-03-22 17:54:55,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:54:55,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd89e90e93e231472414a3 HTTP/1.1" 200 None 2026-03-22 17:54:55,883 - lava_callback - DEBUG - Processing node 69bd89e90e93e231472414a3: baseline-arm64-clabbe 2026-03-22 17:54:55,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd89e90e93e231472414a3 2026-03-22 17:54:55,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:54:55,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd89e90e93e231472414a3 HTTP/1.1" 200 None 2026-03-22 17:54:55,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:54:55,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 17:54:55,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:54:55,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd89e90e93e231472414a3 HTTP/1.1" 200 None 2026-03-22 17:54:55,950 - lava_callback - ERROR - Error processing callback for node 69bd89e90e93e231472414a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56976 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:55:03,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:55:03,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55016 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:55:47,727 - lava_callback - INFO - Received callback for node 69c02865c3d73288b0d08066 INFO: 172.20.0.1:55030 - "POST /node/69c02865c3d73288b0d08066 HTTP/1.0" 202 Accepted 2026-03-22 17:55:47,735 - lava_callback - INFO - Starting async job submit for node 69c02865c3d73288b0d08066 2026-03-22 17:55:47,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:55:47,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02865c3d73288b0d08066 HTTP/1.1" 200 None 2026-03-22 17:55:47,799 - lava_callback - DEBUG - Processing node 69c02865c3d73288b0d08066: baseline-nfs-arm64 2026-03-22 17:55:48,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:55:49,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:55:49,114 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02865c3d73288b0d08066/log.txt.gz 2026-03-22 17:55:49,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:55:49,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:55:49,992 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02865c3d73288b0d08066/lava-logs.yaml 2026-03-22 17:55:50,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:55:50,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:55:50,200 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02865c3d73288b0d08066/lava_callback.json.gz DEBUG: Setup failed for 69c02865c3d73288b0d08066. Transitting job node result: incomplete -> fail 2026-03-22 17:55:50,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:55:50,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02865c3d73288b0d08066 HTTP/1.1" 200 None 2026-03-22 17:55:50,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:55:50,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:55:50,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:55:50,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02865c3d73288b0d08066 HTTP/1.1" 200 None 2026-03-22 17:55:50,444 - lava_callback - INFO - Completed processing callback for node 69c02865c3d73288b0d08066 INFO: 172.20.0.1:41002 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:56:03,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:56:03,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55716 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:57:53,725 - lava_callback - INFO - Received callback for node 69c02d77c3d73288b0d0832f INFO: 172.20.0.1:45564 - "POST /node/69c02d77c3d73288b0d0832f HTTP/1.0" 202 Accepted 2026-03-22 17:57:53,732 - lava_callback - INFO - Starting async job submit for node 69c02d77c3d73288b0d0832f 2026-03-22 17:57:53,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:57:53,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02d77c3d73288b0d0832f HTTP/1.1" 200 None 2026-03-22 17:57:53,787 - lava_callback - DEBUG - Processing node 69c02d77c3d73288b0d0832f: baseline-arm64 2026-03-22 17:57:53,921 - lava_callback - WARNING - Failed to upload log 2026-03-22 17:57:54,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:57:54,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:57:54,197 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02d77c3d73288b0d0832f/lava-logs.yaml 2026-03-22 17:57:54,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:57:54,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:57:54,312 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02d77c3d73288b0d0832f/lava_callback.json.gz 2026-03-22 17:57:54,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:57:54,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02d77c3d73288b0d0832f HTTP/1.1" 200 None 2026-03-22 17:57:54,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:57:54,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:57:54,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:57:54,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02d77c3d73288b0d0832f HTTP/1.1" 200 None 2026-03-22 17:57:54,464 - lava_callback - INFO - Completed processing callback for node 69c02d77c3d73288b0d0832f INFO: 172.20.0.1:45574 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:58:03,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:58:03,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52126 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:58:48,243 - lava_callback - INFO - Received callback for node 69c0286bc3d73288b0d08096 INFO: 172.20.0.1:52132 - "POST /node/69c0286bc3d73288b0d08096 HTTP/1.0" 202 Accepted 2026-03-22 17:58:48,250 - lava_callback - INFO - Starting async job submit for node 69c0286bc3d73288b0d08096 2026-03-22 17:58:48,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:58:48,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0286bc3d73288b0d08096 HTTP/1.1" 200 None 2026-03-22 17:58:48,290 - lava_callback - DEBUG - Processing node 69c0286bc3d73288b0d08096: baseline-nfs-arm64 2026-03-22 17:58:48,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:58:48,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:58:48,948 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286bc3d73288b0d08096/log.txt.gz 2026-03-22 17:58:49,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:58:49,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:58:49,468 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286bc3d73288b0d08096/lava-logs.yaml 2026-03-22 17:58:49,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 17:58:49,657 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 17:58:49,659 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0286bc3d73288b0d08096/lava_callback.json.gz 2026-03-22 17:58:49,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:58:49,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0286bc3d73288b0d08096 HTTP/1.1" 200 None 2026-03-22 17:58:49,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:58:49,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 17:58:49,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:58:49,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0286bc3d73288b0d08096 HTTP/1.1" 200 None 2026-03-22 17:58:49,793 - lava_callback - INFO - Completed processing callback for node 69c0286bc3d73288b0d08096 INFO: 172.20.0.1:41704 - "GET / HTTP/1.0" 200 OK 2026-03-22 17:59:03,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 17:59:03,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39560 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:00:52,192 - lava_callback - INFO - Received callback for node 69bd8b420e93e231472414f2 INFO: 172.20.0.1:55664 - "POST /node/69bd8b420e93e231472414f2 HTTP/1.0" 202 Accepted 2026-03-22 18:00:52,197 - lava_callback - INFO - Starting async job submit for node 69bd8b420e93e231472414f2 2026-03-22 18:00:52,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:52,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8b420e93e231472414f2 HTTP/1.1" 200 None 2026-03-22 18:00:52,214 - lava_callback - DEBUG - Processing node 69bd8b420e93e231472414f2: baseline-arm64-baylibre 2026-03-22 18:00:52,214 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8b420e93e231472414f2 2026-03-22 18:00:52,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:52,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8b420e93e231472414f2 HTTP/1.1" 200 None 2026-03-22 18:00:52,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:52,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 18:00:52,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:52,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8b420e93e231472414f2 HTTP/1.1" 200 None 2026-03-22 18:00:52,271 - lava_callback - ERROR - Error processing callback for node 69bd8b420e93e231472414f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 18:00:55,186 - lava_callback - INFO - Received callback for node 69c02e08c3d73288b0d08344 INFO: 172.20.0.1:55666 - "POST /node/69c02e08c3d73288b0d08344 HTTP/1.0" 202 Accepted 2026-03-22 18:00:55,194 - lava_callback - INFO - Starting async job submit for node 69c02e08c3d73288b0d08344 2026-03-22 18:00:55,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:55,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02e08c3d73288b0d08344 HTTP/1.1" 200 None 2026-03-22 18:00:55,240 - lava_callback - DEBUG - Processing node 69c02e08c3d73288b0d08344: baseline-arm64 2026-03-22 18:00:55,438 - lava_callback - WARNING - Failed to upload log 2026-03-22 18:00:55,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:00:55,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:00:55,764 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02e08c3d73288b0d08344/lava-logs.yaml 2026-03-22 18:00:55,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:00:55,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:00:55,917 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02e08c3d73288b0d08344/lava_callback.json.gz 2026-03-22 18:00:55,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:55,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02e08c3d73288b0d08344 HTTP/1.1" 200 None 2026-03-22 18:00:55,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:55,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:00:55,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:56,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02e08c3d73288b0d08344 HTTP/1.1" 200 None 2026-03-22 18:00:56,137 - lava_callback - INFO - Completed processing callback for node 69c02e08c3d73288b0d08344 2026-03-22 18:00:56,413 - lava_callback - INFO - Received callback for node 69c02d10c3d73288b0d08317 INFO: 172.20.0.1:55678 - "POST /node/69c02d10c3d73288b0d08317 HTTP/1.0" 202 Accepted 2026-03-22 18:00:56,421 - lava_callback - INFO - Starting async job submit for node 69c02d10c3d73288b0d08317 2026-03-22 18:00:56,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:56,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02d10c3d73288b0d08317 HTTP/1.1" 200 None 2026-03-22 18:00:56,483 - lava_callback - DEBUG - Processing node 69c02d10c3d73288b0d08317: baseline-arm64 INFO: 172.20.0.1:55686 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:00:56,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:00:56,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:00:56,980 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02d10c3d73288b0d08317/log.txt.gz 2026-03-22 18:00:57,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:00:57,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:00:57,415 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02d10c3d73288b0d08317/lava-logs.yaml 2026-03-22 18:00:57,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:00:57,582 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:00:57,584 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02d10c3d73288b0d08317/lava_callback.json.gz DEBUG: Setup failed for 69c02d10c3d73288b0d08317. Transitting job node result: incomplete -> fail 2026-03-22 18:00:57,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:57,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02d10c3d73288b0d08317 HTTP/1.1" 200 None 2026-03-22 18:00:57,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:57,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:00:57,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:00:57,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02d10c3d73288b0d08317 HTTP/1.1" 200 None 2026-03-22 18:00:57,802 - lava_callback - INFO - Completed processing callback for node 69c02d10c3d73288b0d08317 2026-03-22 18:01:03,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:01:03,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53172 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:04:56,288 - lava_callback - INFO - Received callback for node 69bd8c380e93e2314724153a INFO: 172.20.0.1:44640 - "POST /node/69bd8c380e93e2314724153a HTTP/1.0" 202 Accepted 2026-03-22 18:04:56,294 - lava_callback - INFO - Starting async job submit for node 69bd8c380e93e2314724153a 2026-03-22 18:04:56,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:04:56,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8c380e93e2314724153a HTTP/1.1" 200 None 2026-03-22 18:04:56,312 - lava_callback - DEBUG - Processing node 69bd8c380e93e2314724153a: baseline-arm64-clabbe 2026-03-22 18:04:56,313 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8c380e93e2314724153a 2026-03-22 18:04:56,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:04:56,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8c380e93e2314724153a HTTP/1.1" 200 None 2026-03-22 18:04:56,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:04:56,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 18:04:56,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:04:56,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8c380e93e2314724153a HTTP/1.1" 200 None 2026-03-22 18:04:56,377 - lava_callback - ERROR - Error processing callback for node 69bd8c380e93e2314724153a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44612 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-22 18:06:16,297 - lava_callback - INFO - Received callback for node 69bd8c8a0e93e2314724154e INFO: 172.20.0.1:44622 - "POST /node/69bd8c8a0e93e2314724154e HTTP/1.0" 202 Accepted 2026-03-22 18:06:16,302 - lava_callback - INFO - Starting async job submit for node 69bd8c8a0e93e2314724154e 2026-03-22 18:06:16,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:06:16,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8c8a0e93e2314724154e HTTP/1.1" 200 None 2026-03-22 18:06:16,318 - lava_callback - DEBUG - Processing node 69bd8c8a0e93e2314724154e: baseline-arm64-clabbe 2026-03-22 18:06:16,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8c8a0e93e2314724154e 2026-03-22 18:06:16,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:06:16,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8c8a0e93e2314724154e HTTP/1.1" 200 None 2026-03-22 18:06:16,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:06:16,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:06:16,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:06:16,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8c8a0e93e2314724154e HTTP/1.1" 200 None 2026-03-22 18:06:16,430 - lava_callback - ERROR - Error processing callback for node 69bd8c8a0e93e2314724154e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38366 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:06:29,545 - lava_callback - INFO - Received callback for node 69c02bc1c3d73288b0d082c0 INFO: 172.20.0.1:38372 - "POST /node/69c02bc1c3d73288b0d082c0 HTTP/1.0" 202 Accepted 2026-03-22 18:06:29,552 - lava_callback - INFO - Starting async job submit for node 69c02bc1c3d73288b0d082c0 2026-03-22 18:06:29,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:06:29,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02bc1c3d73288b0d082c0 HTTP/1.1" 200 None 2026-03-22 18:06:29,644 - lava_callback - DEBUG - Processing node 69c02bc1c3d73288b0d082c0: baseline-arm 2026-03-22 18:06:30,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:06:30,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:06:30,390 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c02bc1c3d73288b0d082c0/log.txt.gz 2026-03-22 18:06:30,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:06:31,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:06:31,013 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c02bc1c3d73288b0d082c0/lava-logs.yaml 2026-03-22 18:06:31,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:06:31,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:06:31,222 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c02bc1c3d73288b0d082c0/lava_callback.json.gz DEBUG: Setup failed for 69c02bc1c3d73288b0d082c0. Transitting job node result: incomplete -> fail 2026-03-22 18:06:31,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:06:31,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02bc1c3d73288b0d082c0 HTTP/1.1" 200 None 2026-03-22 18:06:31,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:06:31,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 18:06:31,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:06:31,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02bc1c3d73288b0d082c0 HTTP/1.1" 200 None 2026-03-22 18:06:31,525 - lava_callback - INFO - Completed processing callback for node 69c02bc1c3d73288b0d082c0 2026-03-22 18:06:33,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:06:33,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57054 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:07:26,957 - lava_callback - INFO - Received callback for node 69c02fc9c3d73288b0d0838a INFO: 172.20.0.1:57070 - "POST /node/69c02fc9c3d73288b0d0838a HTTP/1.0" 202 Accepted 2026-03-22 18:07:26,963 - lava_callback - INFO - Starting async job submit for node 69c02fc9c3d73288b0d0838a 2026-03-22 18:07:26,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:07:26,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02fc9c3d73288b0d0838a HTTP/1.1" 200 None 2026-03-22 18:07:26,980 - lava_callback - DEBUG - Processing node 69c02fc9c3d73288b0d0838a: baseline-nfs-arm64 2026-03-22 18:07:26,981 - lava_callback - WARNING - Failed to upload log 2026-03-22 18:07:26,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:07:27,128 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:07:27,130 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02fc9c3d73288b0d0838a/lava-logs.yaml 2026-03-22 18:07:27,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:07:27,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:07:27,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02fc9c3d73288b0d0838a/lava_callback.json.gz 2026-03-22 18:07:27,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:07:27,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02fc9c3d73288b0d0838a HTTP/1.1" 200 None 2026-03-22 18:07:27,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:07:27,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:07:27,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:07:27,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02fc9c3d73288b0d0838a HTTP/1.1" 200 None 2026-03-22 18:07:27,337 - lava_callback - INFO - Completed processing callback for node 69c02fc9c3d73288b0d0838a 2026-03-22 18:07:33,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:07:33,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42394 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:08:56,434 - lava_callback - INFO - Received callback for node 69bd8d2d0e93e23147241577 INFO: 172.20.0.1:48556 - "POST /node/69bd8d2d0e93e23147241577 HTTP/1.0" 202 Accepted 2026-03-22 18:08:56,440 - lava_callback - INFO - Starting async job submit for node 69bd8d2d0e93e23147241577 2026-03-22 18:08:56,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:08:56,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8d2d0e93e23147241577 HTTP/1.1" 200 None 2026-03-22 18:08:56,456 - lava_callback - DEBUG - Processing node 69bd8d2d0e93e23147241577: baseline-arm64-clabbe 2026-03-22 18:08:56,456 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8d2d0e93e23147241577 2026-03-22 18:08:56,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:08:56,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8d2d0e93e23147241577 HTTP/1.1" 200 None 2026-03-22 18:08:56,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:08:56,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:08:56,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:48558 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:08:56,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8d2d0e93e23147241577 HTTP/1.1" 200 None 2026-03-22 18:08:56,515 - lava_callback - ERROR - Error processing callback for node 69bd8d2d0e93e23147241577: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53094 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-22 18:10:16,432 - lava_callback - INFO - Received callback for node 69bd8d810e93e23147241580 INFO: 172.20.0.1:53106 - "POST /node/69bd8d810e93e23147241580 HTTP/1.0" 202 Accepted 2026-03-22 18:10:16,437 - lava_callback - INFO - Starting async job submit for node 69bd8d810e93e23147241580 2026-03-22 18:10:16,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:16,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8d810e93e23147241580 HTTP/1.1" 200 None 2026-03-22 18:10:16,453 - lava_callback - DEBUG - Processing node 69bd8d810e93e23147241580: baseline-arm64-clabbe 2026-03-22 18:10:16,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8d810e93e23147241580 2026-03-22 18:10:16,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:16,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8d810e93e23147241580 HTTP/1.1" 200 None 2026-03-22 18:10:16,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:16,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 18:10:16,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:16,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8d810e93e23147241580 HTTP/1.1" 200 None 2026-03-22 18:10:16,513 - lava_callback - ERROR - Error processing callback for node 69bd8d810e93e23147241580: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53116 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:10:20,616 - lava_callback - INFO - Received callback for node 69c0287fc3d73288b0d080c0 INFO: 172.20.0.1:39142 - "POST /node/69c0287fc3d73288b0d080c0 HTTP/1.0" 202 Accepted 2026-03-22 18:10:20,622 - lava_callback - INFO - Starting async job submit for node 69c0287fc3d73288b0d080c0 2026-03-22 18:10:20,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:20,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0287fc3d73288b0d080c0 HTTP/1.1" 200 None 2026-03-22 18:10:20,657 - lava_callback - DEBUG - Processing node 69c0287fc3d73288b0d080c0: baseline-arm64-broonie 2026-03-22 18:10:20,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:10:20,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:10:20,894 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0287fc3d73288b0d080c0/log.txt.gz 2026-03-22 18:10:20,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:10:21,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:10:21,114 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0287fc3d73288b0d080c0/lava-logs.yaml 2026-03-22 18:10:21,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:10:21,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:10:21,249 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0287fc3d73288b0d080c0/lava_callback.json.gz 2026-03-22 18:10:21,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:21,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0287fc3d73288b0d080c0 HTTP/1.1" 200 None 2026-03-22 18:10:21,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:21,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:10:21,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:21,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0287fc3d73288b0d080c0 HTTP/1.1" 200 None 2026-03-22 18:10:21,386 - lava_callback - INFO - Completed processing callback for node 69c0287fc3d73288b0d080c0 2026-03-22 18:10:25,103 - lava_callback - INFO - Received callback for node 69c03054c3d73288b0d0839d INFO: 172.20.0.1:39146 - "POST /node/69c03054c3d73288b0d0839d HTTP/1.0" 202 Accepted 2026-03-22 18:10:25,109 - lava_callback - INFO - Starting async job submit for node 69c03054c3d73288b0d0839d 2026-03-22 18:10:25,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:25,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03054c3d73288b0d0839d HTTP/1.1" 200 None 2026-03-22 18:10:25,173 - lava_callback - DEBUG - Processing node 69c03054c3d73288b0d0839d: baseline-arm64 2026-03-22 18:10:25,307 - lava_callback - WARNING - Failed to upload log 2026-03-22 18:10:25,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:10:25,564 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:10:25,565 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c03054c3d73288b0d0839d/lava-logs.yaml 2026-03-22 18:10:25,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:10:25,725 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:10:25,726 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c03054c3d73288b0d0839d/lava_callback.json.gz 2026-03-22 18:10:25,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:25,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03054c3d73288b0d0839d HTTP/1.1" 200 None 2026-03-22 18:10:25,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:25,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:10:25,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:25,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03054c3d73288b0d0839d HTTP/1.1" 200 None 2026-03-22 18:10:25,884 - lava_callback - INFO - Completed processing callback for node 69c03054c3d73288b0d0839d INFO: 172.20.0.1:39152 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:10:33,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:10:33,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50998 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:11:16,838 - lava_callback - INFO - Received callback for node 69c02dc3c3d73288b0d08339 INFO: 172.20.0.1:51000 - "POST /node/69c02dc3c3d73288b0d08339 HTTP/1.0" 202 Accepted 2026-03-22 18:11:16,844 - lava_callback - INFO - Starting async job submit for node 69c02dc3c3d73288b0d08339 2026-03-22 18:11:16,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:11:16,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02dc3c3d73288b0d08339 HTTP/1.1" 200 None 2026-03-22 18:11:16,884 - lava_callback - DEBUG - Processing node 69c02dc3c3d73288b0d08339: baseline-arm64 2026-03-22 18:11:17,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:11:17,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:11:17,674 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02dc3c3d73288b0d08339/log.txt.gz 2026-03-22 18:11:17,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:11:18,125 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:11:18,127 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02dc3c3d73288b0d08339/lava-logs.yaml 2026-03-22 18:11:18,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:11:18,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:11:18,316 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02dc3c3d73288b0d08339/lava_callback.json.gz 2026-03-22 18:11:18,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:11:18,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02dc3c3d73288b0d08339 HTTP/1.1" 200 None 2026-03-22 18:11:18,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:11:18,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:11:18,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:11:18,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02dc3c3d73288b0d08339 HTTP/1.1" 200 None 2026-03-22 18:11:18,483 - lava_callback - INFO - Completed processing callback for node 69c02dc3c3d73288b0d08339 INFO: 172.20.0.1:58828 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:11:33,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:11:33,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47450 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:11:56,576 - lava_callback - INFO - Received callback for node 69bd8dd60e93e2314724158a INFO: 172.20.0.1:47456 - "POST /node/69bd8dd60e93e2314724158a HTTP/1.0" 202 Accepted 2026-03-22 18:11:56,581 - lava_callback - INFO - Starting async job submit for node 69bd8dd60e93e2314724158a 2026-03-22 18:11:56,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:11:56,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8dd60e93e2314724158a HTTP/1.1" 200 None 2026-03-22 18:11:56,597 - lava_callback - DEBUG - Processing node 69bd8dd60e93e2314724158a: baseline-arm64-clabbe 2026-03-22 18:11:56,598 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8dd60e93e2314724158a 2026-03-22 18:11:56,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:11:56,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8dd60e93e2314724158a HTTP/1.1" 200 None 2026-03-22 18:11:56,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:11:56,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 18:11:56,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:11:56,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8dd60e93e2314724158a HTTP/1.1" 200 None 2026-03-22 18:11:56,654 - lava_callback - ERROR - Error processing callback for node 69bd8dd60e93e2314724158a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47804 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:13:16,637 - lava_callback - INFO - Received callback for node 69bd8e270e93e23147241593 INFO: 172.20.0.1:47814 - "POST /node/69bd8e270e93e23147241593 HTTP/1.0" 202 Accepted 2026-03-22 18:13:16,643 - lava_callback - INFO - Starting async job submit for node 69bd8e270e93e23147241593 2026-03-22 18:13:16,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:13:16,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8e270e93e23147241593 HTTP/1.1" 200 None 2026-03-22 18:13:16,659 - lava_callback - DEBUG - Processing node 69bd8e270e93e23147241593: baseline-arm64-clabbe 2026-03-22 18:13:16,659 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8e270e93e23147241593 2026-03-22 18:13:16,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:13:16,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8e270e93e23147241593 HTTP/1.1" 200 None 2026-03-22 18:13:16,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:13:16,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:13:16,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:13:16,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8e270e93e23147241593 HTTP/1.1" 200 None 2026-03-22 18:13:16,723 - lava_callback - ERROR - Error processing callback for node 69bd8e270e93e23147241593: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42760 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:13:55,773 - lava_callback - INFO - Received callback for node 69c03130c3d73288b0d083c1 INFO: 172.20.0.1:52826 - "POST /node/69c03130c3d73288b0d083c1 HTTP/1.0" 202 Accepted 2026-03-22 18:13:55,780 - lava_callback - INFO - Starting async job submit for node 69c03130c3d73288b0d083c1 2026-03-22 18:13:55,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:13:55,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03130c3d73288b0d083c1 HTTP/1.1" 200 None 2026-03-22 18:13:55,827 - lava_callback - DEBUG - Processing node 69c03130c3d73288b0d083c1: baseline-arm64 2026-03-22 18:13:55,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:13:56,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:13:56,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c03130c3d73288b0d083c1/log.txt.gz 2026-03-22 18:13:56,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:13:56,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:13:56,400 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c03130c3d73288b0d083c1/lava-logs.yaml 2026-03-22 18:13:56,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:13:56,522 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:13:56,524 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c03130c3d73288b0d083c1/lava_callback.json.gz INFO: 172.20.0.1:52832 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:13:56,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:13:56,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03130c3d73288b0d083c1 HTTP/1.1" 200 None 2026-03-22 18:13:56,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:13:56,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:13:56,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:13:56,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03130c3d73288b0d083c1 HTTP/1.1" 200 None 2026-03-22 18:13:56,677 - lava_callback - INFO - Completed processing callback for node 69c03130c3d73288b0d083c1 2026-03-22 18:14:03,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:14:03,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41482 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:15:16,757 - lava_callback - INFO - Received callback for node 69bd8ea20e93e231472415ce INFO: 172.20.0.1:41486 - "POST /node/69bd8ea20e93e231472415ce HTTP/1.0" 202 Accepted 2026-03-22 18:15:16,762 - lava_callback - INFO - Starting async job submit for node 69bd8ea20e93e231472415ce 2026-03-22 18:15:16,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:15:16,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8ea20e93e231472415ce HTTP/1.1" 200 None 2026-03-22 18:15:16,782 - lava_callback - DEBUG - Processing node 69bd8ea20e93e231472415ce: baseline-arm64-clabbe 2026-03-22 18:15:16,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8ea20e93e231472415ce 2026-03-22 18:15:16,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:15:16,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8ea20e93e231472415ce HTTP/1.1" 200 None 2026-03-22 18:15:16,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:15:16,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 18:15:16,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:15:16,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8ea20e93e231472415ce HTTP/1.1" 200 None 2026-03-22 18:15:16,840 - lava_callback - ERROR - Error processing callback for node 69bd8ea20e93e231472415ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45268 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:16:00,525 - lava_callback - INFO - Received callback for node 69c02d5ec3d73288b0d08328 INFO: 172.20.0.1:45246 - "POST /node/69c02d5ec3d73288b0d08328 HTTP/1.0" 202 Accepted 2026-03-22 18:16:00,573 - lava_callback - INFO - Starting async job submit for node 69c02d5ec3d73288b0d08328 2026-03-22 18:16:00,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:00,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02d5ec3d73288b0d08328 HTTP/1.1" 200 None 2026-03-22 18:16:00,613 - lava_callback - DEBUG - Processing node 69c02d5ec3d73288b0d08328: baseline-arm-pengutronix 2026-03-22 18:16:00,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:16:00,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:16:00,880 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c02d5ec3d73288b0d08328/log.txt.gz 2026-03-22 18:16:00,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:16:01,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:16:01,123 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c02d5ec3d73288b0d08328/lava-logs.yaml 2026-03-22 18:16:01,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:16:01,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:16:01,262 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c02d5ec3d73288b0d08328/lava_callback.json.gz DEBUG: Setup failed for 69c02d5ec3d73288b0d08328. Transitting job node result: incomplete -> fail 2026-03-22 18:16:01,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:01,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02d5ec3d73288b0d08328 HTTP/1.1" 200 None 2026-03-22 18:16:01,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:01,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 18:16:01,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:01,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02d5ec3d73288b0d08328 HTTP/1.1" 200 None 2026-03-22 18:16:01,427 - lava_callback - INFO - Completed processing callback for node 69c02d5ec3d73288b0d08328 2026-03-22 18:16:03,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:03,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34552 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:16:18,941 - lava_callback - INFO - Received callback for node 69c02ea0c3d73288b0d08362 INFO: 172.20.0.1:34568 - "POST /node/69c02ea0c3d73288b0d08362 HTTP/1.0" 202 Accepted 2026-03-22 18:16:18,948 - lava_callback - INFO - Starting async job submit for node 69c02ea0c3d73288b0d08362 2026-03-22 18:16:18,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:19,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02ea0c3d73288b0d08362 HTTP/1.1" 200 None 2026-03-22 18:16:19,016 - lava_callback - DEBUG - Processing node 69c02ea0c3d73288b0d08362: baseline-arm64 2026-03-22 18:16:19,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:16:19,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:16:19,914 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02ea0c3d73288b0d08362/log.txt.gz 2026-03-22 18:16:20,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:16:20,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:16:20,602 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02ea0c3d73288b0d08362/lava-logs.yaml 2026-03-22 18:16:20,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:16:20,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:16:20,778 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c02ea0c3d73288b0d08362/lava_callback.json.gz DEBUG: Setup failed for 69c02ea0c3d73288b0d08362. Transitting job node result: incomplete -> fail 2026-03-22 18:16:20,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:20,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02ea0c3d73288b0d08362 HTTP/1.1" 200 None 2026-03-22 18:16:20,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:20,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:16:20,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:20,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02ea0c3d73288b0d08362 HTTP/1.1" 200 None 2026-03-22 18:16:21,067 - lava_callback - INFO - Completed processing callback for node 69c02ea0c3d73288b0d08362 INFO: 172.20.0.1:32922 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:16:33,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:33,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41656 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:16:36,751 - lava_callback - INFO - Received callback for node 69bd8ef60e93e231472415d9 INFO: 172.20.0.1:41662 - "POST /node/69bd8ef60e93e231472415d9 HTTP/1.0" 202 Accepted 2026-03-22 18:16:36,757 - lava_callback - INFO - Starting async job submit for node 69bd8ef60e93e231472415d9 2026-03-22 18:16:36,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:36,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8ef60e93e231472415d9 HTTP/1.1" 200 None 2026-03-22 18:16:36,799 - lava_callback - DEBUG - Processing node 69bd8ef60e93e231472415d9: baseline-arm64-clabbe 2026-03-22 18:16:36,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8ef60e93e231472415d9 2026-03-22 18:16:36,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:36,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8ef60e93e231472415d9 HTTP/1.1" 200 None 2026-03-22 18:16:36,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:36,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:16:36,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:16:36,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8ef60e93e231472415d9 HTTP/1.1" 200 None 2026-03-22 18:16:36,860 - lava_callback - ERROR - Error processing callback for node 69bd8ef60e93e231472415d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41152 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:17:54,869 - lava_callback - INFO - Received callback for node 69c02eeec3d73288b0d08368 INFO: 172.20.0.1:57478 - "POST /node/69c02eeec3d73288b0d08368 HTTP/1.0" 202 Accepted 2026-03-22 18:17:54,877 - lava_callback - INFO - Starting async job submit for node 69c02eeec3d73288b0d08368 2026-03-22 18:17:54,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:17:54,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02eeec3d73288b0d08368 HTTP/1.1" 200 None 2026-03-22 18:17:54,943 - lava_callback - DEBUG - Processing node 69c02eeec3d73288b0d08368: baseline-nfs-arm64 2026-03-22 18:17:55,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:17:55,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:17:55,915 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02eeec3d73288b0d08368/log.txt.gz 2026-03-22 18:17:56,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:57480 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:17:56,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:17:56,646 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02eeec3d73288b0d08368/lava-logs.yaml 2026-03-22 18:17:56,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:17:56,818 - lava_callback - INFO - Received callback for node 69bd8f4b0e93e231472415e1 INFO: 172.20.0.1:57488 - "POST /node/69bd8f4b0e93e231472415e1 HTTP/1.0" 202 Accepted 2026-03-22 18:17:56,824 - lava_callback - INFO - Starting async job submit for node 69bd8f4b0e93e231472415e1 2026-03-22 18:17:56,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:17:56,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:17:56,834 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c02eeec3d73288b0d08368/lava_callback.json.gz 2026-03-22 18:17:56,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8f4b0e93e231472415e1 HTTP/1.1" 200 None 2026-03-22 18:17:56,894 - lava_callback - DEBUG - Processing node 69bd8f4b0e93e231472415e1: baseline-arm64-clabbe 2026-03-22 18:17:56,894 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8f4b0e93e231472415e1 2026-03-22 18:17:56,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 DEBUG: Setup failed for 69c02eeec3d73288b0d08368. Transitting job node result: incomplete -> fail 2026-03-22 18:17:56,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:17:56,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8f4b0e93e231472415e1 HTTP/1.1" 200 None 2026-03-22 18:17:56,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:17:56,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c02eeec3d73288b0d08368 HTTP/1.1" 200 None 2026-03-22 18:17:56,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:17:56,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:17:56,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:17:56,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:17:56,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:17:57,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8f4b0e93e231472415e1 HTTP/1.1" 200 None 2026-03-22 18:17:57,034 - lava_callback - ERROR - Error processing callback for node 69bd8f4b0e93e231472415e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 18:17:57,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c02eeec3d73288b0d08368 HTTP/1.1" 200 None 2026-03-22 18:17:57,181 - lava_callback - INFO - Completed processing callback for node 69c02eeec3d73288b0d08368 2026-03-22 18:18:03,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:18:03,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60372 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:19:16,930 - lava_callback - INFO - Received callback for node 69bd8fa10e93e231472415e7 INFO: 172.20.0.1:60388 - "POST /node/69bd8fa10e93e231472415e7 HTTP/1.0" 202 Accepted 2026-03-22 18:19:16,936 - lava_callback - INFO - Starting async job submit for node 69bd8fa10e93e231472415e7 2026-03-22 18:19:16,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:19:16,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8fa10e93e231472415e7 HTTP/1.1" 200 None 2026-03-22 18:19:16,952 - lava_callback - DEBUG - Processing node 69bd8fa10e93e231472415e7: baseline-arm64-clabbe 2026-03-22 18:19:16,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd8fa10e93e231472415e7 2026-03-22 18:19:16,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:19:16,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd8fa10e93e231472415e7 HTTP/1.1" 200 None 2026-03-22 18:19:16,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:19:16,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 18:19:16,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:19:17,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd8fa10e93e231472415e7 HTTP/1.1" 200 None 2026-03-22 18:19:17,007 - lava_callback - ERROR - Error processing callback for node 69bd8fa10e93e231472415e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53084 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:21:37,025 - lava_callback - INFO - Received callback for node 69bd90210e93e23147241625 INFO: 172.20.0.1:53100 - "POST /node/69bd90210e93e23147241625 HTTP/1.0" 202 Accepted 2026-03-22 18:21:37,030 - lava_callback - INFO - Starting async job submit for node 69bd90210e93e23147241625 2026-03-22 18:21:37,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:21:37,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd90210e93e23147241625 HTTP/1.1" 200 None 2026-03-22 18:21:37,049 - lava_callback - DEBUG - Processing node 69bd90210e93e23147241625: baseline-arm64-clabbe 2026-03-22 18:21:37,049 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd90210e93e23147241625 2026-03-22 18:21:37,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:21:37,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd90210e93e23147241625 HTTP/1.1" 200 None 2026-03-22 18:21:37,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:21:37,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 18:21:37,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:21:37,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd90210e93e23147241625 HTTP/1.1" 200 None 2026-03-22 18:21:37,141 - lava_callback - ERROR - Error processing callback for node 69bd90210e93e23147241625: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33550 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:22:01,170 - lava_callback - INFO - Received callback for node 69c0328dc3d73288b0d0840a INFO: 172.20.0.1:47742 - "POST /node/69c0328dc3d73288b0d0840a HTTP/1.0" 202 Accepted 2026-03-22 18:22:01,176 - lava_callback - INFO - Starting async job submit for node 69c0328dc3d73288b0d0840a 2026-03-22 18:22:01,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:22:01,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0328dc3d73288b0d0840a HTTP/1.1" 200 None 2026-03-22 18:22:01,209 - lava_callback - DEBUG - Processing node 69c0328dc3d73288b0d0840a: baseline-arm64-broonie 2026-03-22 18:22:01,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:22:01,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:22:01,666 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0328dc3d73288b0d0840a/log.txt.gz 2026-03-22 18:22:01,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:22:02,066 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:22:02,068 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0328dc3d73288b0d0840a/lava-logs.yaml 2026-03-22 18:22:02,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:22:02,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:22:02,247 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c0328dc3d73288b0d0840a/lava_callback.json.gz 2026-03-22 18:22:02,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:22:02,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0328dc3d73288b0d0840a HTTP/1.1" 200 None 2026-03-22 18:22:02,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:22:02,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:22:02,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:22:02,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0328dc3d73288b0d0840a HTTP/1.1" 200 None 2026-03-22 18:22:02,410 - lava_callback - INFO - Completed processing callback for node 69c0328dc3d73288b0d0840a 2026-03-22 18:22:03,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:22:03,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47568 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:22:29,500 - lava_callback - INFO - Received callback for node 69c0317ec3d73288b0d083cd INFO: 172.20.0.1:47576 - "POST /node/69c0317ec3d73288b0d083cd HTTP/1.0" 202 Accepted 2026-03-22 18:22:29,507 - lava_callback - INFO - Starting async job submit for node 69c0317ec3d73288b0d083cd 2026-03-22 18:22:29,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:22:29,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0317ec3d73288b0d083cd HTTP/1.1" 200 None 2026-03-22 18:22:29,559 - lava_callback - DEBUG - Processing node 69c0317ec3d73288b0d083cd: baseline-arm64 2026-03-22 18:22:29,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:22:30,050 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:22:30,052 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0317ec3d73288b0d083cd/log.txt.gz 2026-03-22 18:22:30,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:22:30,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:22:30,593 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0317ec3d73288b0d083cd/lava-logs.yaml 2026-03-22 18:22:30,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:22:30,778 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:22:30,779 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0317ec3d73288b0d083cd/lava_callback.json.gz DEBUG: Setup failed for 69c0317ec3d73288b0d083cd. Transitting job node result: incomplete -> fail 2026-03-22 18:22:30,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:22:30,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0317ec3d73288b0d083cd HTTP/1.1" 200 None 2026-03-22 18:22:30,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:22:30,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:22:30,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:22:30,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0317ec3d73288b0d083cd HTTP/1.1" 200 None 2026-03-22 18:22:30,993 - lava_callback - INFO - Completed processing callback for node 69c0317ec3d73288b0d083cd 2026-03-22 18:22:33,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:22:33,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40672 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:24:09,205 - lava_callback - INFO - Received callback for node 69c030a7c3d73288b0d083ae INFO: 172.20.0.1:40684 - "POST /node/69c030a7c3d73288b0d083ae HTTP/1.0" 202 Accepted 2026-03-22 18:24:09,212 - lava_callback - INFO - Starting async job submit for node 69c030a7c3d73288b0d083ae 2026-03-22 18:24:09,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:24:09,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c030a7c3d73288b0d083ae HTTP/1.1" 200 None 2026-03-22 18:24:09,253 - lava_callback - DEBUG - Processing node 69c030a7c3d73288b0d083ae: baseline-nfs-arm64 2026-03-22 18:24:09,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:24:09,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:24:09,948 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c030a7c3d73288b0d083ae/log.txt.gz 2026-03-22 18:24:10,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:24:10,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:24:10,468 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c030a7c3d73288b0d083ae/lava-logs.yaml 2026-03-22 18:24:10,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:24:10,657 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:24:10,659 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c030a7c3d73288b0d083ae/lava_callback.json.gz 2026-03-22 18:24:10,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:24:10,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c030a7c3d73288b0d083ae HTTP/1.1" 200 None 2026-03-22 18:24:10,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:24:10,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:24:10,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:24:10,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c030a7c3d73288b0d083ae HTTP/1.1" 200 None 2026-03-22 18:24:10,799 - lava_callback - INFO - Completed processing callback for node 69c030a7c3d73288b0d083ae 2026-03-22 18:24:13,931 - lava_callback - INFO - Received callback for node 69bd90c10e93e2314724163b INFO: 172.20.0.1:53378 - "POST /node/69bd90c10e93e2314724163b HTTP/1.0" 202 Accepted 2026-03-22 18:24:13,937 - lava_callback - INFO - Starting async job submit for node 69bd90c10e93e2314724163b 2026-03-22 18:24:13,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:24:13,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd90c10e93e2314724163b HTTP/1.1" 200 None 2026-03-22 18:24:13,953 - lava_callback - DEBUG - Processing node 69bd90c10e93e2314724163b: baseline-arm64-baylibre 2026-03-22 18:24:13,953 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd90c10e93e2314724163b 2026-03-22 18:24:13,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:24:13,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd90c10e93e2314724163b HTTP/1.1" 200 None 2026-03-22 18:24:13,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:24:14,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:24:14,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:24:14,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd90c10e93e2314724163b HTTP/1.1" 200 None 2026-03-22 18:24:14,040 - lava_callback - ERROR - Error processing callback for node 69bd90c10e93e2314724163b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51418 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:24:33,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:24:33,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34404 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:25:33,494 - lava_callback - INFO - Received callback for node 69bd91180e93e23147241644 INFO: 172.20.0.1:52726 - "POST /node/69bd91180e93e23147241644 HTTP/1.0" 202 Accepted 2026-03-22 18:25:33,499 - lava_callback - INFO - Starting async job submit for node 69bd91180e93e23147241644 2026-03-22 18:25:33,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:25:33,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd91180e93e23147241644 HTTP/1.1" 200 None 2026-03-22 18:25:33,516 - lava_callback - DEBUG - Processing node 69bd91180e93e23147241644: baseline-arm64-baylibre 2026-03-22 18:25:33,516 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd91180e93e23147241644 2026-03-22 18:25:33,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:25:33,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd91180e93e23147241644 HTTP/1.1" 200 None 2026-03-22 18:25:33,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:25:33,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:25:33,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:25:33,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd91180e93e23147241644 HTTP/1.1" 200 None 2026-03-22 18:25:33,579 - lava_callback - ERROR - Error processing callback for node 69bd91180e93e23147241644: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37800 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:27:13,604 - lava_callback - INFO - Received callback for node 69bd91700e93e2314724164e INFO: 172.20.0.1:50272 - "POST /node/69bd91700e93e2314724164e HTTP/1.0" 202 Accepted 2026-03-22 18:27:13,609 - lava_callback - INFO - Starting async job submit for node 69bd91700e93e2314724164e 2026-03-22 18:27:13,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:27:13,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd91700e93e2314724164e HTTP/1.1" 200 None 2026-03-22 18:27:13,625 - lava_callback - DEBUG - Processing node 69bd91700e93e2314724164e: baseline-arm64-baylibre 2026-03-22 18:27:13,625 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd91700e93e2314724164e 2026-03-22 18:27:13,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:27:13,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd91700e93e2314724164e HTTP/1.1" 200 None 2026-03-22 18:27:13,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:27:13,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:27:13,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:27:13,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd91700e93e2314724164e HTTP/1.1" 200 None 2026-03-22 18:27:13,686 - lava_callback - ERROR - Error processing callback for node 69bd91700e93e2314724164e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53276 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:28:14,584 - lava_callback - INFO - Received callback for node 69c03011c3d73288b0d08395 INFO: 172.20.0.1:58068 - "POST /node/69c03011c3d73288b0d08395 HTTP/1.0" 202 Accepted 2026-03-22 18:28:14,593 - lava_callback - INFO - Starting async job submit for node 69c03011c3d73288b0d08395 2026-03-22 18:28:14,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:28:14,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03011c3d73288b0d08395 HTTP/1.1" 200 None 2026-03-22 18:28:14,654 - lava_callback - DEBUG - Processing node 69c03011c3d73288b0d08395: baseline-nfs-arm64 INFO: 172.20.0.1:58076 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-22 18:28:15,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:28:15,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:28:15,915 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c03011c3d73288b0d08395/log.txt.gz INFO: 172.20.0.1:58082 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:28:16,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:28:16,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:28:16,869 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c03011c3d73288b0d08395/lava-logs.yaml 2026-03-22 18:28:16,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:28:17,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:28:17,095 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c03011c3d73288b0d08395/lava_callback.json.gz DEBUG: Setup failed for 69c03011c3d73288b0d08395. Transitting job node result: incomplete -> fail 2026-03-22 18:28:17,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:28:17,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03011c3d73288b0d08395 HTTP/1.1" 200 None 2026-03-22 18:28:17,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:28:17,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:28:17,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:28:17,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03011c3d73288b0d08395 HTTP/1.1" 200 None 2026-03-22 18:28:17,381 - lava_callback - INFO - Completed processing callback for node 69c03011c3d73288b0d08395 INFO: 172.20.0.1:33884 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:28:33,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:28:33,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-22 18:28:33,684 - lava_callback - INFO - Received callback for node 69bd91c70e93e23147241660 INFO: 172.20.0.1:54836 - "POST /node/69bd91c70e93e23147241660 HTTP/1.0" 202 Accepted 2026-03-22 18:28:33,690 - lava_callback - INFO - Starting async job submit for node 69bd91c70e93e23147241660 2026-03-22 18:28:33,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:28:33,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd91c70e93e23147241660 HTTP/1.1" 200 None 2026-03-22 18:28:33,707 - lava_callback - DEBUG - Processing node 69bd91c70e93e23147241660: baseline-arm64-baylibre 2026-03-22 18:28:33,707 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd91c70e93e23147241660 2026-03-22 18:28:33,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:28:33,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd91c70e93e23147241660 HTTP/1.1" 200 None 2026-03-22 18:28:33,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:28:33,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:28:33,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:28:33,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd91c70e93e23147241660 HTTP/1.1" 200 None 2026-03-22 18:28:33,761 - lava_callback - ERROR - Error processing callback for node 69bd91c70e93e23147241660: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42758 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:31:33,927 - lava_callback - INFO - Received callback for node 69bd92750e93e23147241672 INFO: 172.20.0.1:51264 - "POST /node/69bd92750e93e23147241672 HTTP/1.0" 202 Accepted 2026-03-22 18:31:33,935 - lava_callback - INFO - Starting async job submit for node 69bd92750e93e23147241672 2026-03-22 18:31:33,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:31:33,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd92750e93e23147241672 HTTP/1.1" 200 None 2026-03-22 18:31:33,959 - lava_callback - DEBUG - Processing node 69bd92750e93e23147241672: baseline-arm64-baylibre 2026-03-22 18:31:33,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd92750e93e23147241672 2026-03-22 18:31:33,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:31:33,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd92750e93e23147241672 HTTP/1.1" 200 None 2026-03-22 18:31:33,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:31:33,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:31:33,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:31:34,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd92750e93e23147241672 HTTP/1.1" 200 None 2026-03-22 18:31:34,014 - lava_callback - ERROR - Error processing callback for node 69bd92750e93e23147241672: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56268 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:32:53,953 - lava_callback - INFO - Received callback for node 69bd92c40e93e2314724167d INFO: 172.20.0.1:33780 - "POST /node/69bd92c40e93e2314724167d HTTP/1.0" 202 Accepted 2026-03-22 18:32:53,958 - lava_callback - INFO - Starting async job submit for node 69bd92c40e93e2314724167d 2026-03-22 18:32:53,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:32:53,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd92c40e93e2314724167d HTTP/1.1" 200 None 2026-03-22 18:32:53,997 - lava_callback - DEBUG - Processing node 69bd92c40e93e2314724167d: baseline-arm64-baylibre 2026-03-22 18:32:53,997 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd92c40e93e2314724167d 2026-03-22 18:32:53,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:32:54,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd92c40e93e2314724167d HTTP/1.1" 200 None 2026-03-22 18:32:54,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:32:54,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:32:54,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:32:54,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd92c40e93e2314724167d HTTP/1.1" 200 None 2026-03-22 18:32:54,063 - lava_callback - ERROR - Error processing callback for node 69bd92c40e93e2314724167d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57530 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:34:13,990 - lava_callback - INFO - Received callback for node 69bd93190e93e2314724168a INFO: 172.20.0.1:52578 - "POST /node/69bd93190e93e2314724168a HTTP/1.0" 202 Accepted 2026-03-22 18:34:13,996 - lava_callback - INFO - Starting async job submit for node 69bd93190e93e2314724168a 2026-03-22 18:34:13,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:34:14,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd93190e93e2314724168a HTTP/1.1" 200 None 2026-03-22 18:34:14,064 - lava_callback - DEBUG - Processing node 69bd93190e93e2314724168a: baseline-arm64-baylibre 2026-03-22 18:34:14,064 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd93190e93e2314724168a 2026-03-22 18:34:14,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:34:14,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd93190e93e2314724168a HTTP/1.1" 200 None 2026-03-22 18:34:14,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:34:14,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:34:14,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:34:14,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd93190e93e2314724168a HTTP/1.1" 200 None 2026-03-22 18:34:14,137 - lava_callback - ERROR - Error processing callback for node 69bd93190e93e2314724168a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58130 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:34:27,944 - lava_callback - INFO - Received callback for node 69c0320ec3d73288b0d083ee INFO: 172.20.0.1:58142 - "POST /node/69c0320ec3d73288b0d083ee HTTP/1.0" 202 Accepted 2026-03-22 18:34:27,951 - lava_callback - INFO - Starting async job submit for node 69c0320ec3d73288b0d083ee 2026-03-22 18:34:28,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:34:28,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0320ec3d73288b0d083ee HTTP/1.1" 200 None 2026-03-22 18:34:28,021 - lava_callback - DEBUG - Processing node 69c0320ec3d73288b0d083ee: baseline-arm 2026-03-22 18:34:28,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:34:28,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:34:28,749 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c0320ec3d73288b0d083ee/log.txt.gz 2026-03-22 18:34:29,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:34:29,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:34:29,357 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c0320ec3d73288b0d083ee/lava-logs.yaml 2026-03-22 18:34:29,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:34:29,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:34:29,552 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c0320ec3d73288b0d083ee/lava_callback.json.gz DEBUG: Setup failed for 69c0320ec3d73288b0d083ee. Transitting job node result: incomplete -> fail 2026-03-22 18:34:29,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:34:29,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0320ec3d73288b0d083ee HTTP/1.1" 200 None 2026-03-22 18:34:29,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:34:29,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 18:34:29,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:34:29,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0320ec3d73288b0d083ee HTTP/1.1" 200 None 2026-03-22 18:34:29,847 - lava_callback - INFO - Completed processing callback for node 69c0320ec3d73288b0d083ee 2026-03-22 18:34:33,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:34:33,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32886 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:35:34,064 - lava_callback - INFO - Received callback for node 69bd936a0e93e23147241698 INFO: 172.20.0.1:41210 - "POST /node/69bd936a0e93e23147241698 HTTP/1.0" 202 Accepted 2026-03-22 18:35:34,070 - lava_callback - INFO - Starting async job submit for node 69bd936a0e93e23147241698 2026-03-22 18:35:34,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:35:34,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd936a0e93e23147241698 HTTP/1.1" 200 None 2026-03-22 18:35:34,086 - lava_callback - DEBUG - Processing node 69bd936a0e93e23147241698: baseline-arm64-baylibre 2026-03-22 18:35:34,086 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd936a0e93e23147241698 2026-03-22 18:35:34,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:35:34,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd936a0e93e23147241698 HTTP/1.1" 200 None 2026-03-22 18:35:34,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:35:34,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:35:34,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:35:34,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd936a0e93e23147241698 HTTP/1.1" 200 None 2026-03-22 18:35:34,149 - lava_callback - ERROR - Error processing callback for node 69bd936a0e93e23147241698: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41318 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:36:37,317 - lava_callback - INFO - Received callback for node 69c032c9c3d73288b0d08412 INFO: 172.20.0.1:41322 - "POST /node/69c032c9c3d73288b0d08412 HTTP/1.0" 202 Accepted 2026-03-22 18:36:37,324 - lava_callback - INFO - Starting async job submit for node 69c032c9c3d73288b0d08412 2026-03-22 18:36:37,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:36:37,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c032c9c3d73288b0d08412 HTTP/1.1" 200 None 2026-03-22 18:36:37,363 - lava_callback - DEBUG - Processing node 69c032c9c3d73288b0d08412: baseline-arm64 2026-03-22 18:36:37,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:36:38,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:36:38,119 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c032c9c3d73288b0d08412/log.txt.gz 2026-03-22 18:36:38,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:36:38,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:36:38,597 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c032c9c3d73288b0d08412/lava-logs.yaml 2026-03-22 18:36:38,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:36:38,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:36:38,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c032c9c3d73288b0d08412/lava_callback.json.gz 2026-03-22 18:36:38,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:36:38,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c032c9c3d73288b0d08412 HTTP/1.1" 200 None 2026-03-22 18:36:38,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:36:38,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:36:38,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:36:38,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c032c9c3d73288b0d08412 HTTP/1.1" 200 None 2026-03-22 18:36:38,968 - lava_callback - INFO - Completed processing callback for node 69c032c9c3d73288b0d08412 INFO: 172.20.0.1:33156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36314 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:36:57,755 - lava_callback - INFO - Received callback for node 69bd93bf0e93e231472416a8 INFO: 172.20.0.1:36326 - "POST /node/69bd93bf0e93e231472416a8 HTTP/1.0" 202 Accepted 2026-03-22 18:36:57,760 - lava_callback - INFO - Starting async job submit for node 69bd93bf0e93e231472416a8 2026-03-22 18:36:57,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:36:57,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd93bf0e93e231472416a8 HTTP/1.1" 200 None 2026-03-22 18:36:57,779 - lava_callback - DEBUG - Processing node 69bd93bf0e93e231472416a8: baseline-arm64-clabbe 2026-03-22 18:36:57,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd93bf0e93e231472416a8 2026-03-22 18:36:57,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:36:57,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd93bf0e93e231472416a8 HTTP/1.1" 200 None 2026-03-22 18:36:57,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:36:57,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:36:57,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:36:57,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd93bf0e93e231472416a8 HTTP/1.1" 200 None 2026-03-22 18:36:57,833 - lava_callback - ERROR - Error processing callback for node 69bd93bf0e93e231472416a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 18:37:03,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:37:03,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39428 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:38:04,550 - lava_callback - INFO - Received callback for node 69c03543c3d73288b0d08478 INFO: 172.20.0.1:57992 - "POST /node/69c03543c3d73288b0d08478 HTTP/1.0" 202 Accepted 2026-03-22 18:38:04,557 - lava_callback - INFO - Starting async job submit for node 69c03543c3d73288b0d08478 2026-03-22 18:38:04,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:38:04,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03543c3d73288b0d08478 HTTP/1.1" 200 None 2026-03-22 18:38:04,609 - lava_callback - DEBUG - Processing node 69c03543c3d73288b0d08478: baseline-arm64 2026-03-22 18:38:04,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:38:05,146 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:38:05,148 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c03543c3d73288b0d08478/log.txt.gz 2026-03-22 18:38:05,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:38:05,578 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:38:05,580 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c03543c3d73288b0d08478/lava-logs.yaml 2026-03-22 18:38:05,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:38:05,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:38:05,785 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c03543c3d73288b0d08478/lava_callback.json.gz DEBUG: Setup failed for 69c03543c3d73288b0d08478. Transitting job node result: incomplete -> fail 2026-03-22 18:38:05,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:38:05,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03543c3d73288b0d08478 HTTP/1.1" 200 None 2026-03-22 18:38:05,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:38:05,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:38:05,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:38:05,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03543c3d73288b0d08478 HTTP/1.1" 200 None 2026-03-22 18:38:06,004 - lava_callback - INFO - Completed processing callback for node 69c03543c3d73288b0d08478 INFO: 172.20.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58594 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:38:33,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:38:33,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48386 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:38:44,817 - lava_callback - INFO - Received callback for node 69c033cdc3d73288b0d08447 INFO: 172.20.0.1:48390 - "POST /node/69c033cdc3d73288b0d08447 HTTP/1.0" 202 Accepted 2026-03-22 18:38:44,824 - lava_callback - INFO - Starting async job submit for node 69c033cdc3d73288b0d08447 2026-03-22 18:38:44,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:38:44,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c033cdc3d73288b0d08447 HTTP/1.1" 200 None 2026-03-22 18:38:44,887 - lava_callback - DEBUG - Processing node 69c033cdc3d73288b0d08447: baseline-arm64 2026-03-22 18:38:45,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:38:45,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:38:45,746 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c033cdc3d73288b0d08447/log.txt.gz 2026-03-22 18:38:46,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:38:46,394 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:38:46,396 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c033cdc3d73288b0d08447/lava-logs.yaml 2026-03-22 18:38:46,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:48398 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:38:46,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:38:46,589 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c033cdc3d73288b0d08447/lava_callback.json.gz DEBUG: Setup failed for 69c033cdc3d73288b0d08447. Transitting job node result: incomplete -> fail 2026-03-22 18:38:46,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:38:46,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c033cdc3d73288b0d08447 HTTP/1.1" 200 None 2026-03-22 18:38:46,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:38:46,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:38:46,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:38:46,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c033cdc3d73288b0d08447 HTTP/1.1" 200 None 2026-03-22 18:38:46,881 - lava_callback - INFO - Completed processing callback for node 69c033cdc3d73288b0d08447 INFO: 172.20.0.1:44978 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:39:03,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:39:03,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48916 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:39:37,841 - lava_callback - INFO - Received callback for node 69bd94640e93e231472416c1 INFO: 172.20.0.1:48924 - "POST /node/69bd94640e93e231472416c1 HTTP/1.0" 202 Accepted 2026-03-22 18:39:37,847 - lava_callback - INFO - Starting async job submit for node 69bd94640e93e231472416c1 2026-03-22 18:39:37,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:39:37,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd94640e93e231472416c1 HTTP/1.1" 200 None 2026-03-22 18:39:37,863 - lava_callback - DEBUG - Processing node 69bd94640e93e231472416c1: baseline-arm64-clabbe 2026-03-22 18:39:37,863 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd94640e93e231472416c1 2026-03-22 18:39:37,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:39:37,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd94640e93e231472416c1 HTTP/1.1" 200 None 2026-03-22 18:39:37,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:39:37,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:39:37,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:39:37,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd94640e93e231472416c1 HTTP/1.1" 200 None 2026-03-22 18:39:37,919 - lava_callback - ERROR - Error processing callback for node 69bd94640e93e231472416c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55336 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:39:46,586 - lava_callback - INFO - Received callback for node 69c0341ac3d73288b0d0844f INFO: 172.20.0.1:55352 - "POST /node/69c0341ac3d73288b0d0844f HTTP/1.0" 202 Accepted 2026-03-22 18:39:46,594 - lava_callback - INFO - Starting async job submit for node 69c0341ac3d73288b0d0844f 2026-03-22 18:39:46,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:39:46,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0341ac3d73288b0d0844f HTTP/1.1" 200 None 2026-03-22 18:39:46,679 - lava_callback - DEBUG - Processing node 69c0341ac3d73288b0d0844f: baseline-nfs-arm64 2026-03-22 18:39:47,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:39:47,599 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:39:47,601 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0341ac3d73288b0d0844f/log.txt.gz 2026-03-22 18:39:48,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:39:48,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:39:48,331 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0341ac3d73288b0d0844f/lava-logs.yaml 2026-03-22 18:39:48,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:39:48,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:39:48,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0341ac3d73288b0d0844f/lava_callback.json.gz DEBUG: Setup failed for 69c0341ac3d73288b0d0844f. Transitting job node result: incomplete -> fail 2026-03-22 18:39:48,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:39:48,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0341ac3d73288b0d0844f HTTP/1.1" 200 None 2026-03-22 18:39:48,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:39:48,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:39:48,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:39:48,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0341ac3d73288b0d0844f HTTP/1.1" 200 None 2026-03-22 18:39:48,857 - lava_callback - INFO - Completed processing callback for node 69c0341ac3d73288b0d0844f INFO: 172.20.0.1:44618 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:40:03,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:40:03,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44176 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:43:01,559 - lava_callback - INFO - Received callback for node 69c033b0c3d73288b0d0843d INFO: 172.20.0.1:44592 - "POST /node/69c033b0c3d73288b0d0843d HTTP/1.0" 202 Accepted 2026-03-22 18:43:01,605 - lava_callback - INFO - Starting async job submit for node 69c033b0c3d73288b0d0843d 2026-03-22 18:43:01,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:43:01,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c033b0c3d73288b0d0843d HTTP/1.1" 200 None 2026-03-22 18:43:01,668 - lava_callback - DEBUG - Processing node 69c033b0c3d73288b0d0843d: baseline-arm-pengutronix 2026-03-22 18:43:01,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:43:01,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:43:01,934 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c033b0c3d73288b0d0843d/log.txt.gz 2026-03-22 18:43:01,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:43:02,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:43:02,227 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c033b0c3d73288b0d0843d/lava-logs.yaml 2026-03-22 18:43:02,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:43:02,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:43:02,386 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c033b0c3d73288b0d0843d/lava_callback.json.gz DEBUG: Setup failed for 69c033b0c3d73288b0d0843d. Transitting job node result: incomplete -> fail 2026-03-22 18:43:02,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:43:02,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c033b0c3d73288b0d0843d HTTP/1.1" 200 None 2026-03-22 18:43:02,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:43:02,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c01633c3d73288b0d052f0 HTTP/1.1" 200 None 2026-03-22 18:43:02,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:43:02,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c033b0c3d73288b0d0843d HTTP/1.1" 200 None 2026-03-22 18:43:02,578 - lava_callback - INFO - Completed processing callback for node 69c033b0c3d73288b0d0843d 2026-03-22 18:43:03,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:43:03,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40188 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:43:34,580 - lava_callback - INFO - Received callback for node 69bd954f0e93e23147241741 INFO: 172.20.0.1:33994 - "POST /node/69bd954f0e93e23147241741 HTTP/1.0" 202 Accepted 2026-03-22 18:43:34,586 - lava_callback - INFO - Starting async job submit for node 69bd954f0e93e23147241741 2026-03-22 18:43:34,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:43:34,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd954f0e93e23147241741 HTTP/1.1" 200 None 2026-03-22 18:43:34,602 - lava_callback - DEBUG - Processing node 69bd954f0e93e23147241741: baseline-arm64-baylibre 2026-03-22 18:43:34,603 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd954f0e93e23147241741 2026-03-22 18:43:34,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:43:34,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd954f0e93e23147241741 HTTP/1.1" 200 None 2026-03-22 18:43:34,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:43:34,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:43:34,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:43:34,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd954f0e93e23147241741 HTTP/1.1" 200 None 2026-03-22 18:43:34,660 - lava_callback - ERROR - Error processing callback for node 69bd954f0e93e23147241741: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48714 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:45:18,052 - lava_callback - INFO - Received callback for node 69bd95a80e93e23147241753 INFO: 172.20.0.1:48722 - "POST /node/69bd95a80e93e23147241753 HTTP/1.0" 202 Accepted 2026-03-22 18:45:18,058 - lava_callback - INFO - Starting async job submit for node 69bd95a80e93e23147241753 2026-03-22 18:45:18,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:45:18,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd95a80e93e23147241753 HTTP/1.1" 200 None 2026-03-22 18:45:18,074 - lava_callback - DEBUG - Processing node 69bd95a80e93e23147241753: baseline-arm64-clabbe 2026-03-22 18:45:18,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd95a80e93e23147241753 2026-03-22 18:45:18,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:45:18,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd95a80e93e23147241753 HTTP/1.1" 200 None 2026-03-22 18:45:18,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:45:18,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:45:18,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:45:18,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd95a80e93e23147241753 HTTP/1.1" 200 None 2026-03-22 18:45:18,137 - lava_callback - ERROR - Error processing callback for node 69bd95a80e93e23147241753: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42052 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:57376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44362 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:47:58,234 - lava_callback - INFO - Received callback for node 69bd96490e93e2314724176c INFO: 172.20.0.1:44376 - "POST /node/69bd96490e93e2314724176c HTTP/1.0" 202 Accepted 2026-03-22 18:47:58,240 - lava_callback - INFO - Starting async job submit for node 69bd96490e93e2314724176c 2026-03-22 18:47:58,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:47:58,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd96490e93e2314724176c HTTP/1.1" 200 None 2026-03-22 18:47:58,256 - lava_callback - DEBUG - Processing node 69bd96490e93e2314724176c: baseline-arm64-clabbe 2026-03-22 18:47:58,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd96490e93e2314724176c 2026-03-22 18:47:58,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:47:58,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd96490e93e2314724176c HTTP/1.1" 200 None 2026-03-22 18:47:58,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:47:58,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:47:58,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:47:58,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd96490e93e2314724176c HTTP/1.1" 200 None 2026-03-22 18:47:58,321 - lava_callback - ERROR - Error processing callback for node 69bd96490e93e2314724176c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39952 - "GET /login HTTP/1.0" 404 Not Found INFO: 172.20.0.1:39964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39342 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:48:54,819 - lava_callback - INFO - Received callback for node 69bd96940e93e23147241782 INFO: 172.20.0.1:55890 - "POST /node/69bd96940e93e23147241782 HTTP/1.0" 202 Accepted 2026-03-22 18:48:54,824 - lava_callback - INFO - Starting async job submit for node 69bd96940e93e23147241782 2026-03-22 18:48:54,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:48:54,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd96940e93e23147241782 HTTP/1.1" 200 None 2026-03-22 18:48:54,841 - lava_callback - DEBUG - Processing node 69bd96940e93e23147241782: baseline-arm64-baylibre 2026-03-22 18:48:54,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd96940e93e23147241782 2026-03-22 18:48:54,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:48:54,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd96940e93e23147241782 HTTP/1.1" 200 None 2026-03-22 18:48:54,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:48:54,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:48:54,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:48:54,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd96940e93e23147241782 HTTP/1.1" 200 None 2026-03-22 18:48:54,905 - lava_callback - ERROR - Error processing callback for node 69bd96940e93e23147241782: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41986 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:49:30,004 - lava_callback - INFO - Received callback for node 69c0358fc3d73288b0d08484 INFO: 172.20.0.1:43306 - "POST /node/69c0358fc3d73288b0d08484 HTTP/1.0" 202 Accepted 2026-03-22 18:49:30,011 - lava_callback - INFO - Starting async job submit for node 69c0358fc3d73288b0d08484 2026-03-22 18:49:30,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:49:30,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0358fc3d73288b0d08484 HTTP/1.1" 200 None 2026-03-22 18:49:30,055 - lava_callback - DEBUG - Processing node 69c0358fc3d73288b0d08484: baseline-nfs-arm64 2026-03-22 18:49:30,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:49:30,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:49:30,737 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0358fc3d73288b0d08484/log.txt.gz 2026-03-22 18:49:31,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:49:31,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:49:31,281 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0358fc3d73288b0d08484/lava-logs.yaml 2026-03-22 18:49:31,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:49:31,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:49:31,451 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c0358fc3d73288b0d08484/lava_callback.json.gz 2026-03-22 18:49:31,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:49:31,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0358fc3d73288b0d08484 HTTP/1.1" 200 None 2026-03-22 18:49:31,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:49:31,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:49:31,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:49:31,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0358fc3d73288b0d08484 HTTP/1.1" 200 None 2026-03-22 18:49:31,590 - lava_callback - INFO - Completed processing callback for node 69c0358fc3d73288b0d08484 2026-03-22 18:49:33,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:49:33,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55814 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43338 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:50:38,255 - lava_callback - INFO - Received callback for node 69bd96ec0e93e231472417a0 INFO: 172.20.0.1:43352 - "POST /node/69bd96ec0e93e231472417a0 HTTP/1.0" 202 Accepted 2026-03-22 18:50:38,261 - lava_callback - INFO - Starting async job submit for node 69bd96ec0e93e231472417a0 2026-03-22 18:50:38,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:50:38,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd96ec0e93e231472417a0 HTTP/1.1" 200 None 2026-03-22 18:50:38,277 - lava_callback - DEBUG - Processing node 69bd96ec0e93e231472417a0: baseline-arm64-clabbe 2026-03-22 18:50:38,277 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd96ec0e93e231472417a0 2026-03-22 18:50:38,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:50:38,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd96ec0e93e231472417a0 HTTP/1.1" 200 None 2026-03-22 18:50:38,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:50:38,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:50:38,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:50:38,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd96ec0e93e231472417a0 HTTP/1.1" 200 None 2026-03-22 18:50:38,364 - lava_callback - ERROR - Error processing callback for node 69bd96ec0e93e231472417a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45052 - "GET /sitemap.xml HTTP/1.0" 404 Not Found INFO: 172.20.0.1:37108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51136 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:51:58,299 - lava_callback - INFO - Received callback for node 69bd973d0e93e231472417af INFO: 172.20.0.1:51148 - "POST /node/69bd973d0e93e231472417af HTTP/1.0" 202 Accepted 2026-03-22 18:51:58,304 - lava_callback - INFO - Starting async job submit for node 69bd973d0e93e231472417af 2026-03-22 18:51:58,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:51:58,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd973d0e93e231472417af HTTP/1.1" 200 None 2026-03-22 18:51:58,323 - lava_callback - DEBUG - Processing node 69bd973d0e93e231472417af: baseline-arm64-clabbe 2026-03-22 18:51:58,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd973d0e93e231472417af 2026-03-22 18:51:58,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:51:58,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd973d0e93e231472417af HTTP/1.1" 200 None 2026-03-22 18:51:58,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:51:58,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:51:58,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:51:58,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd973d0e93e231472417af HTTP/1.1" 200 None 2026-03-22 18:51:58,382 - lava_callback - ERROR - Error processing callback for node 69bd973d0e93e231472417af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41968 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:52:55,087 - lava_callback - INFO - Received callback for node 69c035d9c3d73288b0d0848a INFO: 172.20.0.1:40470 - "POST /node/69c035d9c3d73288b0d0848a HTTP/1.0" 202 Accepted 2026-03-22 18:52:55,095 - lava_callback - INFO - Starting async job submit for node 69c035d9c3d73288b0d0848a 2026-03-22 18:52:55,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:52:55,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c035d9c3d73288b0d0848a HTTP/1.1" 200 None 2026-03-22 18:52:55,175 - lava_callback - DEBUG - Processing node 69c035d9c3d73288b0d0848a: baseline-nfs-arm64 2026-03-22 18:52:56,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:52:56,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:52:56,482 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c035d9c3d73288b0d0848a/log.txt.gz INFO: 172.20.0.1:40476 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:52:57,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:52:57,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:52:57,344 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c035d9c3d73288b0d0848a/lava-logs.yaml 2026-03-22 18:52:57,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:52:57,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:52:57,555 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c035d9c3d73288b0d0848a/lava_callback.json.gz DEBUG: Setup failed for 69c035d9c3d73288b0d0848a. Transitting job node result: incomplete -> fail 2026-03-22 18:52:57,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:52:57,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c035d9c3d73288b0d0848a HTTP/1.1" 200 None 2026-03-22 18:52:57,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:52:57,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:52:57,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:52:57,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c035d9c3d73288b0d0848a HTTP/1.1" 200 None 2026-03-22 18:52:57,816 - lava_callback - INFO - Completed processing callback for node 69c035d9c3d73288b0d0848a 2026-03-22 18:53:00,102 - lava_callback - INFO - Received callback for node 69c0373dc3d73288b0d084ae INFO: 172.20.0.1:58310 - "POST /node/69c0373dc3d73288b0d084ae HTTP/1.0" 202 Accepted 2026-03-22 18:53:00,109 - lava_callback - INFO - Starting async job submit for node 69c0373dc3d73288b0d084ae 2026-03-22 18:53:00,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:53:00,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0373dc3d73288b0d084ae HTTP/1.1" 200 None 2026-03-22 18:53:00,172 - lava_callback - DEBUG - Processing node 69c0373dc3d73288b0d084ae: baseline-arm64 2026-03-22 18:53:00,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:53:01,027 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:53:01,029 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0373dc3d73288b0d084ae/log.txt.gz 2026-03-22 18:53:01,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:53:01,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:53:01,677 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0373dc3d73288b0d084ae/lava-logs.yaml 2026-03-22 18:53:01,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:53:01,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:53:01,834 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c0373dc3d73288b0d084ae/lava_callback.json.gz DEBUG: Setup failed for 69c0373dc3d73288b0d084ae. Transitting job node result: incomplete -> fail 2026-03-22 18:53:01,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:53:01,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0373dc3d73288b0d084ae HTTP/1.1" 200 None 2026-03-22 18:53:01,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:53:01,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:53:01,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:53:02,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0373dc3d73288b0d084ae HTTP/1.1" 200 None 2026-03-22 18:53:02,133 - lava_callback - INFO - Completed processing callback for node 69c0373dc3d73288b0d084ae 2026-03-22 18:53:03,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:53:03,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51464 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:53:58,441 - lava_callback - INFO - Received callback for node 69bd97b90e93e231472417d7 INFO: 172.20.0.1:51470 - "POST /node/69bd97b90e93e231472417d7 HTTP/1.0" 202 Accepted 2026-03-22 18:53:58,448 - lava_callback - INFO - Starting async job submit for node 69bd97b90e93e231472417d7 2026-03-22 18:53:58,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:53:58,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd97b90e93e231472417d7 HTTP/1.1" 200 None 2026-03-22 18:53:58,485 - lava_callback - DEBUG - Processing node 69bd97b90e93e231472417d7: baseline-arm64-clabbe 2026-03-22 18:53:58,485 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd97b90e93e231472417d7 2026-03-22 18:53:58,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:53:58,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd97b90e93e231472417d7 HTTP/1.1" 200 None 2026-03-22 18:53:58,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:53:58,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:53:58,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:53:58,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd97b90e93e231472417d7 HTTP/1.1" 200 None 2026-03-22 18:53:58,562 - lava_callback - ERROR - Error processing callback for node 69bd97b90e93e231472417d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45870 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:55:57,166 - lava_callback - INFO - Received callback for node 69c037d0c3d73288b0d084c9 INFO: 172.20.0.1:45878 - "POST /node/69c037d0c3d73288b0d084c9 HTTP/1.0" 202 Accepted 2026-03-22 18:55:57,174 - lava_callback - INFO - Starting async job submit for node 69c037d0c3d73288b0d084c9 2026-03-22 18:55:57,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:55:57,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c037d0c3d73288b0d084c9 HTTP/1.1" 200 None 2026-03-22 18:55:57,251 - lava_callback - DEBUG - Processing node 69c037d0c3d73288b0d084c9: baseline-nfs-arm64 2026-03-22 18:55:57,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:55:58,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:55:58,120 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c037d0c3d73288b0d084c9/log.txt.gz 2026-03-22 18:55:58,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:55:58,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:55:58,812 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c037d0c3d73288b0d084c9/lava-logs.yaml 2026-03-22 18:55:58,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 18:55:59,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 18:55:59,006 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c037d0c3d73288b0d084c9/lava_callback.json.gz DEBUG: Setup failed for 69c037d0c3d73288b0d084c9. Transitting job node result: incomplete -> fail 2026-03-22 18:55:59,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:55:59,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c037d0c3d73288b0d084c9 HTTP/1.1" 200 None 2026-03-22 18:55:59,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:55:59,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 18:55:59,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:55:59,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c037d0c3d73288b0d084c9 HTTP/1.1" 200 None 2026-03-22 18:55:59,271 - lava_callback - INFO - Completed processing callback for node 69c037d0c3d73288b0d084c9 2026-03-22 18:56:03,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:56:03,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40144 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:56:38,537 - lava_callback - INFO - Received callback for node 69bd985d0e93e23147241805 INFO: 172.20.0.1:40156 - "POST /node/69bd985d0e93e23147241805 HTTP/1.0" 202 Accepted 2026-03-22 18:56:38,542 - lava_callback - INFO - Starting async job submit for node 69bd985d0e93e23147241805 2026-03-22 18:56:38,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:56:38,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd985d0e93e23147241805 HTTP/1.1" 200 None 2026-03-22 18:56:38,561 - lava_callback - DEBUG - Processing node 69bd985d0e93e23147241805: baseline-arm64-clabbe 2026-03-22 18:56:38,561 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd985d0e93e23147241805 2026-03-22 18:56:38,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:56:38,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd985d0e93e23147241805 HTTP/1.1" 200 None 2026-03-22 18:56:38,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:56:38,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:56:38,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:56:38,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd985d0e93e23147241805 HTTP/1.1" 200 None 2026-03-22 18:56:38,618 - lava_callback - ERROR - Error processing callback for node 69bd985d0e93e23147241805: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39936 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:57:58,646 - lava_callback - INFO - Received callback for node 69bd98ae0e93e23147241822 INFO: 172.20.0.1:39944 - "POST /node/69bd98ae0e93e23147241822 HTTP/1.0" 202 Accepted 2026-03-22 18:57:58,652 - lava_callback - INFO - Starting async job submit for node 69bd98ae0e93e23147241822 2026-03-22 18:57:58,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:57:58,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd98ae0e93e23147241822 HTTP/1.1" 200 None 2026-03-22 18:57:58,683 - lava_callback - DEBUG - Processing node 69bd98ae0e93e23147241822: baseline-arm64-clabbe 2026-03-22 18:57:58,684 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd98ae0e93e23147241822 2026-03-22 18:57:58,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:57:58,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd98ae0e93e23147241822 HTTP/1.1" 200 None 2026-03-22 18:57:58,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:57:58,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:57:58,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:57:58,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd98ae0e93e23147241822 HTTP/1.1" 200 None 2026-03-22 18:57:58,763 - lava_callback - ERROR - Error processing callback for node 69bd98ae0e93e23147241822: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33746 - "GET / HTTP/1.0" 200 OK 2026-03-22 18:59:58,731 - lava_callback - INFO - Received callback for node 69bd99290e93e23147241846 INFO: 172.20.0.1:33748 - "POST /node/69bd99290e93e23147241846 HTTP/1.0" 202 Accepted 2026-03-22 18:59:58,736 - lava_callback - INFO - Starting async job submit for node 69bd99290e93e23147241846 2026-03-22 18:59:58,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:59:58,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd99290e93e23147241846 HTTP/1.1" 200 None 2026-03-22 18:59:58,760 - lava_callback - DEBUG - Processing node 69bd99290e93e23147241846: baseline-arm64-clabbe 2026-03-22 18:59:58,761 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd99290e93e23147241846 2026-03-22 18:59:58,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:59:58,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd99290e93e23147241846 HTTP/1.1" 200 None 2026-03-22 18:59:58,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:59:58,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 18:59:58,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 18:59:58,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd99290e93e23147241846 HTTP/1.1" 200 None 2026-03-22 18:59:58,819 - lava_callback - ERROR - Error processing callback for node 69bd99290e93e23147241846: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43782 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:03:38,925 - lava_callback - INFO - Received callback for node 69bd99f50e93e2314724187c INFO: 172.20.0.1:43786 - "POST /node/69bd99f50e93e2314724187c HTTP/1.0" 202 Accepted 2026-03-22 19:03:38,931 - lava_callback - INFO - Starting async job submit for node 69bd99f50e93e2314724187c 2026-03-22 19:03:38,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:03:38,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd99f50e93e2314724187c HTTP/1.1" 200 None 2026-03-22 19:03:38,948 - lava_callback - DEBUG - Processing node 69bd99f50e93e2314724187c: baseline-arm64-clabbe 2026-03-22 19:03:38,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd99f50e93e2314724187c 2026-03-22 19:03:38,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:03:38,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd99f50e93e2314724187c HTTP/1.1" 200 None 2026-03-22 19:03:38,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:03:38,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 19:03:38,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:03:38,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd99f50e93e2314724187c HTTP/1.1" 200 None 2026-03-22 19:03:39,003 - lava_callback - ERROR - Error processing callback for node 69bd99f50e93e2314724187c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60598 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:04:27,816 - lava_callback - INFO - Received callback for node 69c03505c3d73288b0d08472 INFO: 172.20.0.1:60606 - "POST /node/69c03505c3d73288b0d08472 HTTP/1.0" 202 Accepted 2026-03-22 19:04:27,822 - lava_callback - INFO - Starting async job submit for node 69c03505c3d73288b0d08472 2026-03-22 19:04:27,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:04:27,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03505c3d73288b0d08472 HTTP/1.1" 200 None 2026-03-22 19:04:27,857 - lava_callback - DEBUG - Processing node 69c03505c3d73288b0d08472: baseline-arm64-broonie 2026-03-22 19:04:27,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 19:04:28,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 19:04:28,078 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c03505c3d73288b0d08472/log.txt.gz 2026-03-22 19:04:28,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 19:04:28,400 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 19:04:28,402 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c03505c3d73288b0d08472/lava-logs.yaml 2026-03-22 19:04:28,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 19:04:28,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 19:04:28,550 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c03505c3d73288b0d08472/lava_callback.json.gz 2026-03-22 19:04:28,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:04:28,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03505c3d73288b0d08472 HTTP/1.1" 200 None 2026-03-22 19:04:28,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:04:28,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 19:04:28,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:04:28,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03505c3d73288b0d08472 HTTP/1.1" 200 None 2026-03-22 19:04:28,683 - lava_callback - INFO - Completed processing callback for node 69c03505c3d73288b0d08472 2026-03-22 19:04:33,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:04:33,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59268 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:04:58,935 - lava_callback - INFO - Received callback for node 69bd9a4a0e93e2314724188e INFO: 172.20.0.1:59278 - "POST /node/69bd9a4a0e93e2314724188e HTTP/1.0" 202 Accepted 2026-03-22 19:04:58,941 - lava_callback - INFO - Starting async job submit for node 69bd9a4a0e93e2314724188e 2026-03-22 19:04:58,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:04:58,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9a4a0e93e2314724188e HTTP/1.1" 200 None 2026-03-22 19:04:58,958 - lava_callback - DEBUG - Processing node 69bd9a4a0e93e2314724188e: baseline-arm64-clabbe 2026-03-22 19:04:58,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9a4a0e93e2314724188e 2026-03-22 19:04:58,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:04:58,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9a4a0e93e2314724188e HTTP/1.1" 200 None 2026-03-22 19:04:58,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:04:58,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-22 19:04:58,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:04:59,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9a4a0e93e2314724188e HTTP/1.1" 200 None 2026-03-22 19:04:59,026 - lava_callback - ERROR - Error processing callback for node 69bd9a4a0e93e2314724188e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36126 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:12:55,660 - lava_callback - INFO - Received callback for node 69c03a93c3d73288b0d08566 INFO: 172.20.0.1:54954 - "POST /node/69c03a93c3d73288b0d08566 HTTP/1.0" 202 Accepted 2026-03-22 19:12:55,668 - lava_callback - INFO - Starting async job submit for node 69c03a93c3d73288b0d08566 2026-03-22 19:12:55,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:12:55,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03a93c3d73288b0d08566 HTTP/1.1" 200 None 2026-03-22 19:12:55,728 - lava_callback - DEBUG - Processing node 69c03a93c3d73288b0d08566: baseline-nfs-arm64 INFO: 172.20.0.1:54968 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:12:56,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 19:12:57,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 19:12:57,047 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c03a93c3d73288b0d08566/log.txt.gz 2026-03-22 19:12:57,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 19:12:57,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 19:12:57,928 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c03a93c3d73288b0d08566/lava-logs.yaml 2026-03-22 19:12:57,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 19:12:58,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 19:12:58,127 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c03a93c3d73288b0d08566/lava_callback.json.gz DEBUG: Setup failed for 69c03a93c3d73288b0d08566. Transitting job node result: incomplete -> fail 2026-03-22 19:12:58,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:12:58,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03a93c3d73288b0d08566 HTTP/1.1" 200 None 2026-03-22 19:12:58,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:12:58,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 19:12:58,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:12:58,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03a93c3d73288b0d08566 HTTP/1.1" 200 None 2026-03-22 19:12:58,400 - lava_callback - INFO - Completed processing callback for node 69c03a93c3d73288b0d08566 2026-03-22 19:13:03,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:13:03,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38042 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:16:39,403 - lava_callback - INFO - Received callback for node 69bd9d0e0e93e23147241957 INFO: 172.20.0.1:38052 - "POST /node/69bd9d0e0e93e23147241957 HTTP/1.0" 202 Accepted 2026-03-22 19:16:39,409 - lava_callback - INFO - Starting async job submit for node 69bd9d0e0e93e23147241957 2026-03-22 19:16:39,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:16:39,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9d0e0e93e23147241957 HTTP/1.1" 200 None 2026-03-22 19:16:39,461 - lava_callback - DEBUG - Processing node 69bd9d0e0e93e23147241957: baseline-arm-clabbe 2026-03-22 19:16:39,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9d0e0e93e23147241957 2026-03-22 19:16:39,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:16:39,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9d0e0e93e23147241957 HTTP/1.1" 200 None 2026-03-22 19:16:39,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:16:39,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-22 19:16:39,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:16:39,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9d0e0e93e23147241957 HTTP/1.1" 200 None 2026-03-22 19:16:39,517 - lava_callback - ERROR - Error processing callback for node 69bd9d0e0e93e23147241957: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40126 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-22 19:17:16,345 - lava_callback - INFO - Received callback for node 69bd9d320e93e23147241964 INFO: 172.20.0.1:40134 - "POST /node/69bd9d320e93e23147241964 HTTP/1.0" 202 Accepted 2026-03-22 19:17:16,351 - lava_callback - INFO - Starting async job submit for node 69bd9d320e93e23147241964 2026-03-22 19:17:16,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:17:16,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9d320e93e23147241964 HTTP/1.1" 200 None 2026-03-22 19:17:16,367 - lava_callback - DEBUG - Processing node 69bd9d320e93e23147241964: baseline-arm-baylibre 2026-03-22 19:17:16,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9d320e93e23147241964 2026-03-22 19:17:16,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:17:16,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9d320e93e23147241964 HTTP/1.1" 200 None 2026-03-22 19:17:16,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:17:16,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:17:16,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:17:16,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9d320e93e23147241964 HTTP/1.1" 200 None 2026-03-22 19:17:16,427 - lava_callback - ERROR - Error processing callback for node 69bd9d320e93e23147241964: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52910 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:17:59,391 - lava_callback - INFO - Received callback for node 69bd9d600e93e23147241972 INFO: 172.20.0.1:52924 - "POST /node/69bd9d600e93e23147241972 HTTP/1.0" 202 Accepted 2026-03-22 19:17:59,396 - lava_callback - INFO - Starting async job submit for node 69bd9d600e93e23147241972 2026-03-22 19:17:59,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:17:59,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9d600e93e23147241972 HTTP/1.1" 200 None 2026-03-22 19:17:59,412 - lava_callback - DEBUG - Processing node 69bd9d600e93e23147241972: baseline-arm-clabbe 2026-03-22 19:17:59,412 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9d600e93e23147241972 2026-03-22 19:17:59,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:17:59,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9d600e93e23147241972 HTTP/1.1" 200 None 2026-03-22 19:17:59,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:17:59,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-22 19:17:59,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:17:59,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9d600e93e23147241972 HTTP/1.1" 200 None 2026-03-22 19:17:59,464 - lava_callback - ERROR - Error processing callback for node 69bd9d600e93e23147241972: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56520 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:18:39,415 - lava_callback - INFO - Received callback for node 69bd9d8b0e93e23147241979 INFO: 172.20.0.1:56524 - "POST /node/69bd9d8b0e93e23147241979 HTTP/1.0" 202 Accepted 2026-03-22 19:18:39,421 - lava_callback - INFO - Starting async job submit for node 69bd9d8b0e93e23147241979 2026-03-22 19:18:39,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:18:39,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9d8b0e93e23147241979 HTTP/1.1" 200 None 2026-03-22 19:18:39,440 - lava_callback - DEBUG - Processing node 69bd9d8b0e93e23147241979: baseline-arm-clabbe 2026-03-22 19:18:39,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9d8b0e93e23147241979 2026-03-22 19:18:39,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:18:39,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9d8b0e93e23147241979 HTTP/1.1" 200 None 2026-03-22 19:18:39,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:18:39,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-22 19:18:39,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:18:39,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9d8b0e93e23147241979 HTTP/1.1" 200 None 2026-03-22 19:18:39,495 - lava_callback - ERROR - Error processing callback for node 69bd9d8b0e93e23147241979: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55032 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-22 19:19:16,465 - lava_callback - INFO - Received callback for node 69bd9daf0e93e2314724198a INFO: 172.20.0.1:55042 - "POST /node/69bd9daf0e93e2314724198a HTTP/1.0" 202 Accepted 2026-03-22 19:19:16,470 - lava_callback - INFO - Starting async job submit for node 69bd9daf0e93e2314724198a 2026-03-22 19:19:16,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:19:16,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9daf0e93e2314724198a HTTP/1.1" 200 None 2026-03-22 19:19:16,486 - lava_callback - DEBUG - Processing node 69bd9daf0e93e2314724198a: baseline-arm-baylibre 2026-03-22 19:19:16,487 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9daf0e93e2314724198a 2026-03-22 19:19:16,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:19:16,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9daf0e93e2314724198a HTTP/1.1" 200 None 2026-03-22 19:19:16,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:55054 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:19:16,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:19:16,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:19:16,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9daf0e93e2314724198a HTTP/1.1" 200 None 2026-03-22 19:19:16,544 - lava_callback - ERROR - Error processing callback for node 69bd9daf0e93e2314724198a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53522 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:19:56,379 - lava_callback - INFO - Received callback for node 69bd9dd70e93e23147241996 INFO: 172.20.0.1:44660 - "POST /node/69bd9dd70e93e23147241996 HTTP/1.0" 202 Accepted 2026-03-22 19:19:56,384 - lava_callback - INFO - Starting async job submit for node 69bd9dd70e93e23147241996 2026-03-22 19:19:56,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:19:56,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9dd70e93e23147241996 HTTP/1.1" 200 None 2026-03-22 19:19:56,400 - lava_callback - DEBUG - Processing node 69bd9dd70e93e23147241996: baseline-arm-baylibre 2026-03-22 19:19:56,400 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9dd70e93e23147241996 2026-03-22 19:19:56,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:19:56,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9dd70e93e23147241996 HTTP/1.1" 200 None 2026-03-22 19:19:56,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:19:56,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:19:56,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:19:56,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9dd70e93e23147241996 HTTP/1.1" 200 None 2026-03-22 19:19:56,465 - lava_callback - ERROR - Error processing callback for node 69bd9dd70e93e23147241996: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33264 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:20:36,501 - lava_callback - INFO - Received callback for node 69bd9dff0e93e231472419a2 INFO: 172.20.0.1:51602 - "POST /node/69bd9dff0e93e231472419a2 HTTP/1.0" 202 Accepted 2026-03-22 19:20:36,507 - lava_callback - INFO - Starting async job submit for node 69bd9dff0e93e231472419a2 2026-03-22 19:20:36,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:51618 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:20:36,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9dff0e93e231472419a2 HTTP/1.1" 200 None 2026-03-22 19:20:36,524 - lava_callback - DEBUG - Processing node 69bd9dff0e93e231472419a2: baseline-arm-baylibre 2026-03-22 19:20:36,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9dff0e93e231472419a2 2026-03-22 19:20:36,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:20:36,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9dff0e93e231472419a2 HTTP/1.1" 200 None 2026-03-22 19:20:36,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:20:36,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:20:36,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:20:36,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9dff0e93e231472419a2 HTTP/1.1" 200 None 2026-03-22 19:20:36,579 - lava_callback - ERROR - Error processing callback for node 69bd9dff0e93e231472419a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48088 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-22 19:21:16,459 - lava_callback - INFO - Received callback for node 69bd9e280e93e231472419ad INFO: 172.20.0.1:48098 - "POST /node/69bd9e280e93e231472419ad HTTP/1.0" 202 Accepted 2026-03-22 19:21:16,465 - lava_callback - INFO - Starting async job submit for node 69bd9e280e93e231472419ad 2026-03-22 19:21:16,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:21:16,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9e280e93e231472419ad HTTP/1.1" 200 None 2026-03-22 19:21:16,481 - lava_callback - DEBUG - Processing node 69bd9e280e93e231472419ad: baseline-arm-baylibre 2026-03-22 19:21:16,481 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9e280e93e231472419ad 2026-03-22 19:21:16,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:21:16,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9e280e93e231472419ad HTTP/1.1" 200 None 2026-03-22 19:21:16,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:21:16,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:21:16,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:48102 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:21:16,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9e280e93e231472419ad HTTP/1.1" 200 None 2026-03-22 19:21:16,537 - lava_callback - ERROR - Error processing callback for node 69bd9e280e93e231472419ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54882 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:21:56,541 - lava_callback - INFO - Received callback for node 69bd9e520e93e231472419b7 INFO: 172.20.0.1:54896 - "POST /node/69bd9e520e93e231472419b7 HTTP/1.0" 202 Accepted 2026-03-22 19:21:56,547 - lava_callback - INFO - Starting async job submit for node 69bd9e520e93e231472419b7 2026-03-22 19:21:56,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:21:56,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9e520e93e231472419b7 HTTP/1.1" 200 None 2026-03-22 19:21:56,591 - lava_callback - DEBUG - Processing node 69bd9e520e93e231472419b7: baseline-arm-baylibre 2026-03-22 19:21:56,592 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9e520e93e231472419b7 2026-03-22 19:21:56,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:21:56,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9e520e93e231472419b7 HTTP/1.1" 200 None 2026-03-22 19:21:56,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:21:56,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:21:56,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:21:56,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9e520e93e231472419b7 HTTP/1.1" 200 None 2026-03-22 19:21:56,651 - lava_callback - ERROR - Error processing callback for node 69bd9e520e93e231472419b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34840 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:22:59,622 - lava_callback - INFO - Received callback for node 69bd9e800e93e231472419c5 INFO: 172.20.0.1:34852 - "POST /node/69bd9e800e93e231472419c5 HTTP/1.0" 202 Accepted 2026-03-22 19:22:59,627 - lava_callback - INFO - Starting async job submit for node 69bd9e800e93e231472419c5 2026-03-22 19:22:59,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:22:59,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9e800e93e231472419c5 HTTP/1.1" 200 None 2026-03-22 19:22:59,644 - lava_callback - DEBUG - Processing node 69bd9e800e93e231472419c5: baseline-arm-clabbe 2026-03-22 19:22:59,644 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9e800e93e231472419c5 2026-03-22 19:22:59,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:22:59,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9e800e93e231472419c5 HTTP/1.1" 200 None 2026-03-22 19:22:59,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:22:59,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-22 19:22:59,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:22:59,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9e800e93e231472419c5 HTTP/1.1" 200 None 2026-03-22 19:22:59,736 - lava_callback - ERROR - Error processing callback for node 69bd9e800e93e231472419c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45136 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:23:36,748 - lava_callback - INFO - Received callback for node 69bd9ea40e93e231472419d1 INFO: 172.20.0.1:45144 - "POST /node/69bd9ea40e93e231472419d1 HTTP/1.0" 202 Accepted 2026-03-22 19:23:36,753 - lava_callback - INFO - Starting async job submit for node 69bd9ea40e93e231472419d1 2026-03-22 19:23:36,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:23:36,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9ea40e93e231472419d1 HTTP/1.1" 200 None 2026-03-22 19:23:36,769 - lava_callback - DEBUG - Processing node 69bd9ea40e93e231472419d1: baseline-arm-baylibre 2026-03-22 19:23:36,769 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9ea40e93e231472419d1 2026-03-22 19:23:36,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:23:36,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9ea40e93e231472419d1 HTTP/1.1" 200 None 2026-03-22 19:23:36,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:23:36,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:23:36,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:23:36,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9ea40e93e231472419d1 HTTP/1.1" 200 None 2026-03-22 19:23:36,864 - lava_callback - ERROR - Error processing callback for node 69bd9ea40e93e231472419d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51244 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:24:19,700 - lava_callback - INFO - Received callback for node 69bd9ed30e93e231472419db INFO: 172.20.0.1:47900 - "POST /node/69bd9ed30e93e231472419db HTTP/1.0" 202 Accepted 2026-03-22 19:24:19,706 - lava_callback - INFO - Starting async job submit for node 69bd9ed30e93e231472419db 2026-03-22 19:24:19,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:24:19,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9ed30e93e231472419db HTTP/1.1" 200 None 2026-03-22 19:24:19,722 - lava_callback - DEBUG - Processing node 69bd9ed30e93e231472419db: baseline-arm-clabbe 2026-03-22 19:24:19,722 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9ed30e93e231472419db 2026-03-22 19:24:19,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:24:19,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9ed30e93e231472419db HTTP/1.1" 200 None 2026-03-22 19:24:19,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:24:19,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-22 19:24:19,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:24:19,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9ed30e93e231472419db HTTP/1.1" 200 None 2026-03-22 19:24:19,804 - lava_callback - ERROR - Error processing callback for node 69bd9ed30e93e231472419db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38620 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:24:56,726 - lava_callback - INFO - Received callback for node 69bd9ef80e93e231472419e6 INFO: 172.20.0.1:38636 - "POST /node/69bd9ef80e93e231472419e6 HTTP/1.0" 202 Accepted 2026-03-22 19:24:56,731 - lava_callback - INFO - Starting async job submit for node 69bd9ef80e93e231472419e6 2026-03-22 19:24:56,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:24:56,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9ef80e93e231472419e6 HTTP/1.1" 200 None 2026-03-22 19:24:56,748 - lava_callback - DEBUG - Processing node 69bd9ef80e93e231472419e6: baseline-arm-baylibre 2026-03-22 19:24:56,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9ef80e93e231472419e6 2026-03-22 19:24:56,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:24:56,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9ef80e93e231472419e6 HTTP/1.1" 200 None 2026-03-22 19:24:56,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:24:56,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:24:56,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:24:56,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9ef80e93e231472419e6 HTTP/1.1" 200 None 2026-03-22 19:24:56,808 - lava_callback - ERROR - Error processing callback for node 69bd9ef80e93e231472419e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44422 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:25:36,760 - lava_callback - INFO - Received callback for node 69bd9f200e93e231472419f4 INFO: 172.20.0.1:44436 - "POST /node/69bd9f200e93e231472419f4 HTTP/1.0" 202 Accepted 2026-03-22 19:25:36,765 - lava_callback - INFO - Starting async job submit for node 69bd9f200e93e231472419f4 2026-03-22 19:25:36,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:25:36,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9f200e93e231472419f4 HTTP/1.1" 200 None 2026-03-22 19:25:36,782 - lava_callback - DEBUG - Processing node 69bd9f200e93e231472419f4: baseline-arm-baylibre 2026-03-22 19:25:36,783 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9f200e93e231472419f4 2026-03-22 19:25:36,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:25:36,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9f200e93e231472419f4 HTTP/1.1" 200 None 2026-03-22 19:25:36,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:25:36,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:25:36,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:25:36,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9f200e93e231472419f4 HTTP/1.1" 200 None 2026-03-22 19:25:36,842 - lava_callback - ERROR - Error processing callback for node 69bd9f200e93e231472419f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56358 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:26:16,770 - lava_callback - INFO - Received callback for node 69bd9f4c0e93e23147241a00 INFO: 172.20.0.1:56368 - "POST /node/69bd9f4c0e93e23147241a00 HTTP/1.0" 202 Accepted 2026-03-22 19:26:16,775 - lava_callback - INFO - Starting async job submit for node 69bd9f4c0e93e23147241a00 2026-03-22 19:26:16,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:26:16,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9f4c0e93e23147241a00 HTTP/1.1" 200 None 2026-03-22 19:26:16,792 - lava_callback - DEBUG - Processing node 69bd9f4c0e93e23147241a00: baseline-arm-baylibre 2026-03-22 19:26:16,792 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9f4c0e93e23147241a00 2026-03-22 19:26:16,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:26:16,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9f4c0e93e23147241a00 HTTP/1.1" 200 None 2026-03-22 19:26:16,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:26:16,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:26:16,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:26:16,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9f4c0e93e23147241a00 HTTP/1.1" 200 None 2026-03-22 19:26:16,856 - lava_callback - ERROR - Error processing callback for node 69bd9f4c0e93e23147241a00: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40544 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:26:56,835 - lava_callback - INFO - Received callback for node 69bd9f740e93e23147241a0d INFO: 172.20.0.1:40554 - "POST /node/69bd9f740e93e23147241a0d HTTP/1.0" 202 Accepted 2026-03-22 19:26:56,840 - lava_callback - INFO - Starting async job submit for node 69bd9f740e93e23147241a0d 2026-03-22 19:26:56,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:26:56,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9f740e93e23147241a0d HTTP/1.1" 200 None 2026-03-22 19:26:56,856 - lava_callback - DEBUG - Processing node 69bd9f740e93e23147241a0d: baseline-arm-baylibre 2026-03-22 19:26:56,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9f740e93e23147241a0d 2026-03-22 19:26:56,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:26:56,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9f740e93e23147241a0d HTTP/1.1" 200 None 2026-03-22 19:26:56,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:26:56,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:26:56,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:26:56,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9f740e93e23147241a0d HTTP/1.1" 200 None 2026-03-22 19:26:56,914 - lava_callback - ERROR - Error processing callback for node 69bd9f740e93e23147241a0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-22 19:27:03,650 - lava_callback - INFO - Received callback for node 69c03d50c3d73288b0d085a8 INFO: 172.20.0.1:39546 - "POST /node/69c03d50c3d73288b0d085a8 HTTP/1.0" 202 Accepted 2026-03-22 19:27:03,656 - lava_callback - INFO - Starting async job submit for node 69c03d50c3d73288b0d085a8 2026-03-22 19:27:03,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:27:03,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03d50c3d73288b0d085a8 HTTP/1.1" 200 None 2026-03-22 19:27:03,736 - lava_callback - DEBUG - Processing node 69c03d50c3d73288b0d085a8: baseline-arm64-broonie 2026-03-22 19:27:04,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 19:27:04,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 19:27:04,242 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c03d50c3d73288b0d085a8/log.txt.gz 2026-03-22 19:27:04,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 19:27:04,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 19:27:04,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c03d50c3d73288b0d085a8/lava-logs.yaml 2026-03-22 19:27:04,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-22 19:27:04,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-22 19:27:04,841 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c03d50c3d73288b0d085a8/lava_callback.json.gz 2026-03-22 19:27:04,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:27:04,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c03d50c3d73288b0d085a8 HTTP/1.1" 200 None 2026-03-22 19:27:04,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:27:04,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0163fc3d73288b0d05306 HTTP/1.1" 200 None 2026-03-22 19:27:04,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:27:04,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c03d50c3d73288b0d085a8 HTTP/1.1" 200 None 2026-03-22 19:27:04,927 - lava_callback - INFO - Completed processing callback for node 69c03d50c3d73288b0d085a8 INFO: 172.20.0.1:39550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48626 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:27:33,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:27:33,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35726 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:27:36,890 - lava_callback - INFO - Received callback for node 69bd9f9f0e93e23147241a1d INFO: 172.20.0.1:35728 - "POST /node/69bd9f9f0e93e23147241a1d HTTP/1.0" 202 Accepted 2026-03-22 19:27:36,896 - lava_callback - INFO - Starting async job submit for node 69bd9f9f0e93e23147241a1d 2026-03-22 19:27:36,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:27:36,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9f9f0e93e23147241a1d HTTP/1.1" 200 None 2026-03-22 19:27:36,912 - lava_callback - DEBUG - Processing node 69bd9f9f0e93e23147241a1d: baseline-arm-baylibre 2026-03-22 19:27:36,912 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9f9f0e93e23147241a1d 2026-03-22 19:27:36,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:27:36,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9f9f0e93e23147241a1d HTTP/1.1" 200 None 2026-03-22 19:27:36,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:27:36,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:27:36,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:27:36,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9f9f0e93e23147241a1d HTTP/1.1" 200 None 2026-03-22 19:27:36,969 - lava_callback - ERROR - Error processing callback for node 69bd9f9f0e93e23147241a1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56664 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:28:16,867 - lava_callback - INFO - Received callback for node 69bd9fc80e93e23147241a2c INFO: 172.20.0.1:56668 - "POST /node/69bd9fc80e93e23147241a2c HTTP/1.0" 202 Accepted 2026-03-22 19:28:16,876 - lava_callback - INFO - Starting async job submit for node 69bd9fc80e93e23147241a2c 2026-03-22 19:28:16,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:28:16,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9fc80e93e23147241a2c HTTP/1.1" 200 None 2026-03-22 19:28:16,891 - lava_callback - DEBUG - Processing node 69bd9fc80e93e23147241a2c: baseline-arm-baylibre 2026-03-22 19:28:16,892 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9fc80e93e23147241a2c 2026-03-22 19:28:16,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:28:16,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9fc80e93e23147241a2c HTTP/1.1" 200 None 2026-03-22 19:28:16,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:28:16,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:28:16,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:28:16,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9fc80e93e23147241a2c HTTP/1.1" 200 None 2026-03-22 19:28:16,951 - lava_callback - ERROR - Error processing callback for node 69bd9fc80e93e23147241a2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59442 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:28:57,006 - lava_callback - INFO - Received callback for node 69bd9ff00e93e23147241a3e INFO: 172.20.0.1:59448 - "POST /node/69bd9ff00e93e23147241a3e HTTP/1.0" 202 Accepted 2026-03-22 19:28:57,011 - lava_callback - INFO - Starting async job submit for node 69bd9ff00e93e23147241a3e 2026-03-22 19:28:57,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:28:57,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9ff00e93e23147241a3e HTTP/1.1" 200 None 2026-03-22 19:28:57,028 - lava_callback - DEBUG - Processing node 69bd9ff00e93e23147241a3e: baseline-arm-baylibre 2026-03-22 19:28:57,029 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bd9ff00e93e23147241a3e 2026-03-22 19:28:57,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:28:57,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd9ff00e93e23147241a3e HTTP/1.1" 200 None 2026-03-22 19:28:57,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:28:57,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:28:57,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:28:57,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bd9ff00e93e23147241a3e HTTP/1.1" 200 None 2026-03-22 19:28:57,084 - lava_callback - ERROR - Error processing callback for node 69bd9ff00e93e23147241a3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58250 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:29:36,983 - lava_callback - INFO - Received callback for node 69bda0170e93e23147241a68 INFO: 172.20.0.1:58262 - "POST /node/69bda0170e93e23147241a68 HTTP/1.0" 202 Accepted 2026-03-22 19:29:36,989 - lava_callback - INFO - Starting async job submit for node 69bda0170e93e23147241a68 2026-03-22 19:29:36,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:29:37,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda0170e93e23147241a68 HTTP/1.1" 200 None 2026-03-22 19:29:37,008 - lava_callback - DEBUG - Processing node 69bda0170e93e23147241a68: baseline-arm-baylibre 2026-03-22 19:29:37,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda0170e93e23147241a68 2026-03-22 19:29:37,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:29:37,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda0170e93e23147241a68 HTTP/1.1" 200 None 2026-03-22 19:29:37,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:29:37,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:29:37,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:29:37,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda0170e93e23147241a68 HTTP/1.1" 200 None 2026-03-22 19:29:37,066 - lava_callback - ERROR - Error processing callback for node 69bda0170e93e23147241a68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40418 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:30:19,897 - lava_callback - INFO - Received callback for node 69bda0440e93e23147241a83 INFO: 172.20.0.1:50598 - "POST /node/69bda0440e93e23147241a83 HTTP/1.0" 202 Accepted 2026-03-22 19:30:19,902 - lava_callback - INFO - Starting async job submit for node 69bda0440e93e23147241a83 2026-03-22 19:30:19,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:30:19,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda0440e93e23147241a83 HTTP/1.1" 200 None 2026-03-22 19:30:19,920 - lava_callback - DEBUG - Processing node 69bda0440e93e23147241a83: baseline-arm-clabbe 2026-03-22 19:30:19,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda0440e93e23147241a83 2026-03-22 19:30:19,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:30:19,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda0440e93e23147241a83 HTTP/1.1" 200 None 2026-03-22 19:30:19,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:30:19,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:30:19,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:30:19,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda0440e93e23147241a83 HTTP/1.1" 200 None 2026-03-22 19:30:19,981 - lava_callback - ERROR - Error processing callback for node 69bda0440e93e23147241a83: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32978 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:30:59,892 - lava_callback - INFO - Received callback for node 69bda06e0e93e23147241a96 INFO: 172.20.0.1:43752 - "POST /node/69bda06e0e93e23147241a96 HTTP/1.0" 202 Accepted 2026-03-22 19:30:59,897 - lava_callback - INFO - Starting async job submit for node 69bda06e0e93e23147241a96 2026-03-22 19:30:59,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:30:59,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda06e0e93e23147241a96 HTTP/1.1" 200 None 2026-03-22 19:30:59,913 - lava_callback - DEBUG - Processing node 69bda06e0e93e23147241a96: baseline-arm-clabbe 2026-03-22 19:30:59,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda06e0e93e23147241a96 2026-03-22 19:30:59,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:30:59,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda06e0e93e23147241a96 HTTP/1.1" 200 None 2026-03-22 19:30:59,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:30:59,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:30:59,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:30:59,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda06e0e93e23147241a96 HTTP/1.1" 200 None 2026-03-22 19:30:59,980 - lava_callback - ERROR - Error processing callback for node 69bda06e0e93e23147241a96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49708 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:31:40,022 - lava_callback - INFO - Received callback for node 69bda0980e93e23147241aa2 INFO: 172.20.0.1:42702 - "POST /node/69bda0980e93e23147241aa2 HTTP/1.0" 202 Accepted 2026-03-22 19:31:40,027 - lava_callback - INFO - Starting async job submit for node 69bda0980e93e23147241aa2 2026-03-22 19:31:40,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:31:40,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda0980e93e23147241aa2 HTTP/1.1" 200 None 2026-03-22 19:31:40,043 - lava_callback - DEBUG - Processing node 69bda0980e93e23147241aa2: baseline-arm-clabbe 2026-03-22 19:31:40,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda0980e93e23147241aa2 2026-03-22 19:31:40,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:31:40,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda0980e93e23147241aa2 HTTP/1.1" 200 None 2026-03-22 19:31:40,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:31:40,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:31:40,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:31:40,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda0980e93e23147241aa2 HTTP/1.1" 200 None 2026-03-22 19:31:40,096 - lava_callback - ERROR - Error processing callback for node 69bda0980e93e23147241aa2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36520 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:32:40,025 - lava_callback - INFO - Received callback for node 69bda0c20e93e23147241ab3 INFO: 172.20.0.1:34064 - "POST /node/69bda0c20e93e23147241ab3 HTTP/1.0" 202 Accepted 2026-03-22 19:32:40,031 - lava_callback - INFO - Starting async job submit for node 69bda0c20e93e23147241ab3 2026-03-22 19:32:40,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:32:40,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda0c20e93e23147241ab3 HTTP/1.1" 200 None 2026-03-22 19:32:40,048 - lava_callback - DEBUG - Processing node 69bda0c20e93e23147241ab3: baseline-arm-clabbe 2026-03-22 19:32:40,048 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda0c20e93e23147241ab3 2026-03-22 19:32:40,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:32:40,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda0c20e93e23147241ab3 HTTP/1.1" 200 None 2026-03-22 19:32:40,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:32:40,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:32:40,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:32:40,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda0c20e93e23147241ab3 HTTP/1.1" 200 None 2026-03-22 19:32:40,142 - lava_callback - ERROR - Error processing callback for node 69bda0c20e93e23147241ab3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57126 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:33:20,001 - lava_callback - INFO - Received callback for node 69bda0ec0e93e23147241abf INFO: 172.20.0.1:49574 - "POST /node/69bda0ec0e93e23147241abf HTTP/1.0" 202 Accepted 2026-03-22 19:33:20,007 - lava_callback - INFO - Starting async job submit for node 69bda0ec0e93e23147241abf 2026-03-22 19:33:20,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:33:20,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda0ec0e93e23147241abf HTTP/1.1" 200 None 2026-03-22 19:33:20,031 - lava_callback - DEBUG - Processing node 69bda0ec0e93e23147241abf: baseline-arm-clabbe 2026-03-22 19:33:20,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda0ec0e93e23147241abf 2026-03-22 19:33:20,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:33:20,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda0ec0e93e23147241abf HTTP/1.1" 200 None 2026-03-22 19:33:20,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:33:20,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:33:20,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:33:20,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda0ec0e93e23147241abf HTTP/1.1" 200 None 2026-03-22 19:33:20,089 - lava_callback - ERROR - Error processing callback for node 69bda0ec0e93e23147241abf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55706 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:34:00,059 - lava_callback - INFO - Received callback for node 69bda1160e93e23147241acb INFO: 172.20.0.1:52108 - "POST /node/69bda1160e93e23147241acb HTTP/1.0" 202 Accepted 2026-03-22 19:34:00,064 - lava_callback - INFO - Starting async job submit for node 69bda1160e93e23147241acb 2026-03-22 19:34:00,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:34:00,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda1160e93e23147241acb HTTP/1.1" 200 None 2026-03-22 19:34:00,085 - lava_callback - DEBUG - Processing node 69bda1160e93e23147241acb: baseline-arm-clabbe 2026-03-22 19:34:00,085 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda1160e93e23147241acb 2026-03-22 19:34:00,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:34:00,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda1160e93e23147241acb HTTP/1.1" 200 None 2026-03-22 19:34:00,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:34:00,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:34:00,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:34:00,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda1160e93e23147241acb HTTP/1.1" 200 None 2026-03-22 19:34:00,169 - lava_callback - ERROR - Error processing callback for node 69bda1160e93e23147241acb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46234 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:34:40,074 - lava_callback - INFO - Received callback for node 69bda13f0e93e23147241aff INFO: 172.20.0.1:39068 - "POST /node/69bda13f0e93e23147241aff HTTP/1.0" 202 Accepted 2026-03-22 19:34:40,080 - lava_callback - INFO - Starting async job submit for node 69bda13f0e93e23147241aff 2026-03-22 19:34:40,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:34:40,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda13f0e93e23147241aff HTTP/1.1" 200 None 2026-03-22 19:34:40,095 - lava_callback - DEBUG - Processing node 69bda13f0e93e23147241aff: baseline-arm-clabbe 2026-03-22 19:34:40,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda13f0e93e23147241aff 2026-03-22 19:34:40,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:34:40,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda13f0e93e23147241aff HTTP/1.1" 200 None 2026-03-22 19:34:40,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:34:40,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:34:40,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:34:40,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda13f0e93e23147241aff HTTP/1.1" 200 None 2026-03-22 19:34:40,178 - lava_callback - ERROR - Error processing callback for node 69bda13f0e93e23147241aff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55340 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:35:17,585 - lava_callback - INFO - Received callback for node 69bda1660e93e23147241b1f INFO: 172.20.0.1:55342 - "POST /node/69bda1660e93e23147241b1f HTTP/1.0" 202 Accepted 2026-03-22 19:35:17,590 - lava_callback - INFO - Starting async job submit for node 69bda1660e93e23147241b1f 2026-03-22 19:35:17,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:35:17,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda1660e93e23147241b1f HTTP/1.1" 200 None 2026-03-22 19:35:17,610 - lava_callback - DEBUG - Processing node 69bda1660e93e23147241b1f: baseline-arm-baylibre 2026-03-22 19:35:17,610 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda1660e93e23147241b1f 2026-03-22 19:35:17,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:35:17,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda1660e93e23147241b1f HTTP/1.1" 200 None 2026-03-22 19:35:17,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:35:17,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:35:17,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:35:17,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda1660e93e23147241b1f HTTP/1.1" 200 None 2026-03-22 19:35:17,688 - lava_callback - ERROR - Error processing callback for node 69bda1660e93e23147241b1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37330 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:35:57,274 - lava_callback - INFO - Received callback for node 69bda1900e93e23147241b41 INFO: 172.20.0.1:37336 - "POST /node/69bda1900e93e23147241b41 HTTP/1.0" 202 Accepted 2026-03-22 19:35:57,279 - lava_callback - INFO - Starting async job submit for node 69bda1900e93e23147241b41 2026-03-22 19:35:57,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:35:57,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda1900e93e23147241b41 HTTP/1.1" 200 None 2026-03-22 19:35:57,296 - lava_callback - DEBUG - Processing node 69bda1900e93e23147241b41: baseline-arm-baylibre 2026-03-22 19:35:57,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda1900e93e23147241b41 2026-03-22 19:35:57,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:35:57,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda1900e93e23147241b41 HTTP/1.1" 200 None 2026-03-22 19:35:57,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:35:57,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:35:57,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:35:57,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda1900e93e23147241b41 HTTP/1.1" 200 None 2026-03-22 19:35:57,359 - lava_callback - ERROR - Error processing callback for node 69bda1900e93e23147241b41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57346 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:36:40,196 - lava_callback - INFO - Received callback for node 69bda1bc0e93e23147241b8a INFO: 172.20.0.1:52700 - "POST /node/69bda1bc0e93e23147241b8a HTTP/1.0" 202 Accepted 2026-03-22 19:36:40,202 - lava_callback - INFO - Starting async job submit for node 69bda1bc0e93e23147241b8a 2026-03-22 19:36:40,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:36:40,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda1bc0e93e23147241b8a HTTP/1.1" 200 None 2026-03-22 19:36:40,241 - lava_callback - DEBUG - Processing node 69bda1bc0e93e23147241b8a: baseline-arm-clabbe 2026-03-22 19:36:40,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda1bc0e93e23147241b8a 2026-03-22 19:36:40,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:36:40,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda1bc0e93e23147241b8a HTTP/1.1" 200 None 2026-03-22 19:36:40,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:36:40,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:36:40,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:36:40,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda1bc0e93e23147241b8a HTTP/1.1" 200 None 2026-03-22 19:36:40,304 - lava_callback - ERROR - Error processing callback for node 69bda1bc0e93e23147241b8a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44852 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:37:17,368 - lava_callback - INFO - Received callback for node 69bda1e40e93e23147241ba6 INFO: 172.20.0.1:44858 - "POST /node/69bda1e40e93e23147241ba6 HTTP/1.0" 202 Accepted 2026-03-22 19:37:17,373 - lava_callback - INFO - Starting async job submit for node 69bda1e40e93e23147241ba6 2026-03-22 19:37:17,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:37:17,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda1e40e93e23147241ba6 HTTP/1.1" 200 None 2026-03-22 19:37:17,413 - lava_callback - DEBUG - Processing node 69bda1e40e93e23147241ba6: baseline-arm-baylibre 2026-03-22 19:37:17,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda1e40e93e23147241ba6 2026-03-22 19:37:17,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:37:17,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda1e40e93e23147241ba6 HTTP/1.1" 200 None 2026-03-22 19:37:17,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:37:17,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:37:17,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:37:17,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda1e40e93e23147241ba6 HTTP/1.1" 200 None 2026-03-22 19:37:17,548 - lava_callback - ERROR - Error processing callback for node 69bda1e40e93e23147241ba6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58788 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:39:20,320 - lava_callback - INFO - Received callback for node 69bda2610e93e23147241bf6 INFO: 172.20.0.1:57900 - "POST /node/69bda2610e93e23147241bf6 HTTP/1.0" 202 Accepted 2026-03-22 19:39:20,326 - lava_callback - INFO - Starting async job submit for node 69bda2610e93e23147241bf6 2026-03-22 19:39:20,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:39:20,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda2610e93e23147241bf6 HTTP/1.1" 200 None 2026-03-22 19:39:20,366 - lava_callback - DEBUG - Processing node 69bda2610e93e23147241bf6: baseline-arm-clabbe 2026-03-22 19:39:20,366 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda2610e93e23147241bf6 2026-03-22 19:39:20,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:39:20,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda2610e93e23147241bf6 HTTP/1.1" 200 None 2026-03-22 19:39:20,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:39:20,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:39:20,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:39:20,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda2610e93e23147241bf6 HTTP/1.1" 200 None 2026-03-22 19:39:20,453 - lava_callback - ERROR - Error processing callback for node 69bda2610e93e23147241bf6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50494 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:40:00,287 - lava_callback - INFO - Received callback for node 69bda2870e93e23147241c23 INFO: 172.20.0.1:52344 - "POST /node/69bda2870e93e23147241c23 HTTP/1.0" 202 Accepted 2026-03-22 19:40:00,292 - lava_callback - INFO - Starting async job submit for node 69bda2870e93e23147241c23 2026-03-22 19:40:00,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:40:00,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda2870e93e23147241c23 HTTP/1.1" 200 None 2026-03-22 19:40:00,309 - lava_callback - DEBUG - Processing node 69bda2870e93e23147241c23: baseline-arm-clabbe 2026-03-22 19:40:00,309 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda2870e93e23147241c23 2026-03-22 19:40:00,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:40:00,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda2870e93e23147241c23 HTTP/1.1" 200 None 2026-03-22 19:40:00,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:40:00,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:40:00,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:40:00,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda2870e93e23147241c23 HTTP/1.1" 200 None 2026-03-22 19:40:00,414 - lava_callback - ERROR - Error processing callback for node 69bda2870e93e23147241c23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40078 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:40:40,369 - lava_callback - INFO - Received callback for node 69bda2b20e93e23147241c44 INFO: 172.20.0.1:40616 - "POST /node/69bda2b20e93e23147241c44 HTTP/1.0" 202 Accepted 2026-03-22 19:40:40,374 - lava_callback - INFO - Starting async job submit for node 69bda2b20e93e23147241c44 2026-03-22 19:40:40,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:40:40,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda2b20e93e23147241c44 HTTP/1.1" 200 None 2026-03-22 19:40:40,395 - lava_callback - DEBUG - Processing node 69bda2b20e93e23147241c44: baseline-arm-clabbe 2026-03-22 19:40:40,395 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda2b20e93e23147241c44 2026-03-22 19:40:40,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:40:40,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda2b20e93e23147241c44 HTTP/1.1" 200 None 2026-03-22 19:40:40,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:40:40,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:40:40,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:40:40,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda2b20e93e23147241c44 HTTP/1.1" 200 None 2026-03-22 19:40:40,477 - lava_callback - ERROR - Error processing callback for node 69bda2b20e93e23147241c44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42992 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:41:20,347 - lava_callback - INFO - Received callback for node 69bda2dc0e93e23147241c66 INFO: 172.20.0.1:56890 - "POST /node/69bda2dc0e93e23147241c66 HTTP/1.0" 202 Accepted 2026-03-22 19:41:20,353 - lava_callback - INFO - Starting async job submit for node 69bda2dc0e93e23147241c66 2026-03-22 19:41:20,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:41:20,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda2dc0e93e23147241c66 HTTP/1.1" 200 None 2026-03-22 19:41:20,370 - lava_callback - DEBUG - Processing node 69bda2dc0e93e23147241c66: baseline-arm-clabbe 2026-03-22 19:41:20,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda2dc0e93e23147241c66 2026-03-22 19:41:20,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:41:20,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda2dc0e93e23147241c66 HTTP/1.1" 200 None 2026-03-22 19:41:20,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:41:20,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:41:20,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:41:20,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda2dc0e93e23147241c66 HTTP/1.1" 200 None 2026-03-22 19:41:20,455 - lava_callback - ERROR - Error processing callback for node 69bda2dc0e93e23147241c66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43056 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:42:00,438 - lava_callback - INFO - Received callback for node 69bda3050e93e23147241cb1 INFO: 172.20.0.1:53836 - "POST /node/69bda3050e93e23147241cb1 HTTP/1.0" 202 Accepted 2026-03-22 19:42:00,444 - lava_callback - INFO - Starting async job submit for node 69bda3050e93e23147241cb1 2026-03-22 19:42:00,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:42:00,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda3050e93e23147241cb1 HTTP/1.1" 200 None 2026-03-22 19:42:00,463 - lava_callback - DEBUG - Processing node 69bda3050e93e23147241cb1: baseline-arm-clabbe 2026-03-22 19:42:00,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda3050e93e23147241cb1 2026-03-22 19:42:00,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:42:00,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda3050e93e23147241cb1 HTTP/1.1" 200 None 2026-03-22 19:42:00,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:42:00,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:42:00,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:42:00,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda3050e93e23147241cb1 HTTP/1.1" 200 None 2026-03-22 19:42:00,524 - lava_callback - ERROR - Error processing callback for node 69bda3050e93e23147241cb1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43910 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:44:20,513 - lava_callback - INFO - Received callback for node 69bda3810e93e23147241cf2 INFO: 172.20.0.1:44148 - "POST /node/69bda3810e93e23147241cf2 HTTP/1.0" 202 Accepted 2026-03-22 19:44:20,518 - lava_callback - INFO - Starting async job submit for node 69bda3810e93e23147241cf2 2026-03-22 19:44:20,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:44:20,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda3810e93e23147241cf2 HTTP/1.1" 200 None 2026-03-22 19:44:20,534 - lava_callback - DEBUG - Processing node 69bda3810e93e23147241cf2: baseline-arm-clabbe 2026-03-22 19:44:20,534 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda3810e93e23147241cf2 2026-03-22 19:44:20,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:44:20,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda3810e93e23147241cf2 HTTP/1.1" 200 None 2026-03-22 19:44:20,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:44:20,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:44:20,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:44:20,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda3810e93e23147241cf2 HTTP/1.1" 200 None 2026-03-22 19:44:20,671 - lava_callback - ERROR - Error processing callback for node 69bda3810e93e23147241cf2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33704 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:45:00,505 - lava_callback - INFO - Received callback for node 69bda3ac0e93e23147241d0a INFO: 172.20.0.1:57004 - "POST /node/69bda3ac0e93e23147241d0a HTTP/1.0" 202 Accepted 2026-03-22 19:45:00,511 - lava_callback - INFO - Starting async job submit for node 69bda3ac0e93e23147241d0a 2026-03-22 19:45:00,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:45:00,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda3ac0e93e23147241d0a HTTP/1.1" 200 None 2026-03-22 19:45:00,535 - lava_callback - DEBUG - Processing node 69bda3ac0e93e23147241d0a: baseline-arm-clabbe 2026-03-22 19:45:00,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda3ac0e93e23147241d0a 2026-03-22 19:45:00,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:45:00,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda3ac0e93e23147241d0a HTTP/1.1" 200 None 2026-03-22 19:45:00,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:45:00,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-22 19:45:00,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:45:00,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda3ac0e93e23147241d0a HTTP/1.1" 200 None 2026-03-22 19:45:00,595 - lava_callback - ERROR - Error processing callback for node 69bda3ac0e93e23147241d0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:55:18,387 - lava_callback - INFO - Received callback for node 69bda6180e93e23147241e2f INFO: 172.20.0.1:52142 - "POST /node/69bda6180e93e23147241e2f HTTP/1.0" 202 Accepted 2026-03-22 19:55:18,393 - lava_callback - INFO - Starting async job submit for node 69bda6180e93e23147241e2f 2026-03-22 19:55:18,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:55:18,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda6180e93e23147241e2f HTTP/1.1" 200 None 2026-03-22 19:55:18,413 - lava_callback - DEBUG - Processing node 69bda6180e93e23147241e2f: baseline-arm64-baylibre 2026-03-22 19:55:18,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda6180e93e23147241e2f 2026-03-22 19:55:18,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:55:18,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda6180e93e23147241e2f HTTP/1.1" 200 None 2026-03-22 19:55:18,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:55:18,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-22 19:55:18,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:55:18,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda6180e93e23147241e2f HTTP/1.1" 200 None 2026-03-22 19:55:18,474 - lava_callback - ERROR - Error processing callback for node 69bda6180e93e23147241e2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45442 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:56:41,093 - lava_callback - INFO - Received callback for node 69bda66f0e93e23147241e41 INFO: 172.20.0.1:53172 - "POST /node/69bda66f0e93e23147241e41 HTTP/1.0" 202 Accepted 2026-03-22 19:56:41,099 - lava_callback - INFO - Starting async job submit for node 69bda66f0e93e23147241e41 2026-03-22 19:56:41,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:56:41,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda66f0e93e23147241e41 HTTP/1.1" 200 None 2026-03-22 19:56:41,120 - lava_callback - DEBUG - Processing node 69bda66f0e93e23147241e41: baseline-arm64-clabbe 2026-03-22 19:56:41,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda66f0e93e23147241e41 2026-03-22 19:56:41,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:56:41,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda66f0e93e23147241e41 HTTP/1.1" 200 None 2026-03-22 19:56:41,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:56:41,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-22 19:56:41,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:56:41,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda66f0e93e23147241e41 HTTP/1.1" 200 None 2026-03-22 19:56:41,172 - lava_callback - ERROR - Error processing callback for node 69bda66f0e93e23147241e41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54842 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:58:01,050 - lava_callback - INFO - Received callback for node 69bda6c10e93e23147241e5c INFO: 172.20.0.1:57342 - "POST /node/69bda6c10e93e23147241e5c HTTP/1.0" 202 Accepted 2026-03-22 19:58:01,056 - lava_callback - INFO - Starting async job submit for node 69bda6c10e93e23147241e5c 2026-03-22 19:58:01,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:58:01,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda6c10e93e23147241e5c HTTP/1.1" 200 None 2026-03-22 19:58:01,073 - lava_callback - DEBUG - Processing node 69bda6c10e93e23147241e5c: baseline-arm64-clabbe 2026-03-22 19:58:01,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda6c10e93e23147241e5c 2026-03-22 19:58:01,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:58:01,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda6c10e93e23147241e5c HTTP/1.1" 200 None 2026-03-22 19:58:01,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:58:01,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-22 19:58:01,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:58:01,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda6c10e93e23147241e5c HTTP/1.1" 200 None 2026-03-22 19:58:01,153 - lava_callback - ERROR - Error processing callback for node 69bda6c10e93e23147241e5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33236 - "GET / HTTP/1.0" 200 OK 2026-03-22 19:59:21,132 - lava_callback - INFO - Received callback for node 69bda7110e93e23147241e61 INFO: 172.20.0.1:34780 - "POST /node/69bda7110e93e23147241e61 HTTP/1.0" 202 Accepted 2026-03-22 19:59:21,138 - lava_callback - INFO - Starting async job submit for node 69bda7110e93e23147241e61 2026-03-22 19:59:21,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:59:21,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda7110e93e23147241e61 HTTP/1.1" 200 None 2026-03-22 19:59:21,153 - lava_callback - DEBUG - Processing node 69bda7110e93e23147241e61: baseline-arm64-clabbe 2026-03-22 19:59:21,154 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda7110e93e23147241e61 2026-03-22 19:59:21,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:59:21,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda7110e93e23147241e61 HTTP/1.1" 200 None 2026-03-22 19:59:21,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:59:21,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-22 19:59:21,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 19:59:21,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda7110e93e23147241e61 HTTP/1.1" 200 None 2026-03-22 19:59:21,214 - lava_callback - ERROR - Error processing callback for node 69bda7110e93e23147241e61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:01:01,154 - lava_callback - INFO - Received callback for node 69bda7670e93e23147241e77 INFO: 172.20.0.1:49158 - "POST /node/69bda7670e93e23147241e77 HTTP/1.0" 202 Accepted 2026-03-22 20:01:01,159 - lava_callback - INFO - Starting async job submit for node 69bda7670e93e23147241e77 2026-03-22 20:01:01,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:01:01,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda7670e93e23147241e77 HTTP/1.1" 200 None 2026-03-22 20:01:01,175 - lava_callback - DEBUG - Processing node 69bda7670e93e23147241e77: baseline-arm64-clabbe 2026-03-22 20:01:01,175 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda7670e93e23147241e77 2026-03-22 20:01:01,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:01:01,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda7670e93e23147241e77 HTTP/1.1" 200 None 2026-03-22 20:01:01,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:01:01,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-22 20:01:01,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:01:01,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda7670e93e23147241e77 HTTP/1.1" 200 None 2026-03-22 20:01:01,249 - lava_callback - ERROR - Error processing callback for node 69bda7670e93e23147241e77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39254 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:02:21,267 - lava_callback - INFO - Received callback for node 69bda7bb0e93e23147241e90 INFO: 172.20.0.1:38880 - "POST /node/69bda7bb0e93e23147241e90 HTTP/1.0" 202 Accepted 2026-03-22 20:02:21,273 - lava_callback - INFO - Starting async job submit for node 69bda7bb0e93e23147241e90 2026-03-22 20:02:21,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:02:21,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda7bb0e93e23147241e90 HTTP/1.1" 200 None 2026-03-22 20:02:21,289 - lava_callback - DEBUG - Processing node 69bda7bb0e93e23147241e90: baseline-arm64-clabbe 2026-03-22 20:02:21,289 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda7bb0e93e23147241e90 2026-03-22 20:02:21,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:02:21,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda7bb0e93e23147241e90 HTTP/1.1" 200 None 2026-03-22 20:02:21,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:02:21,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-22 20:02:21,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:02:21,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda7bb0e93e23147241e90 HTTP/1.1" 200 None 2026-03-22 20:02:21,351 - lava_callback - ERROR - Error processing callback for node 69bda7bb0e93e23147241e90: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33892 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:03:41,318 - lava_callback - INFO - Received callback for node 69bda80e0e93e23147241ead INFO: 172.20.0.1:59734 - "POST /node/69bda80e0e93e23147241ead HTTP/1.0" 202 Accepted 2026-03-22 20:03:41,323 - lava_callback - INFO - Starting async job submit for node 69bda80e0e93e23147241ead 2026-03-22 20:03:41,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:03:41,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda80e0e93e23147241ead HTTP/1.1" 200 None 2026-03-22 20:03:41,349 - lava_callback - DEBUG - Processing node 69bda80e0e93e23147241ead: baseline-arm64-clabbe 2026-03-22 20:03:41,349 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bda80e0e93e23147241ead 2026-03-22 20:03:41,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:03:41,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bda80e0e93e23147241ead HTTP/1.1" 200 None 2026-03-22 20:03:41,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:03:41,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-22 20:03:41,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:03:41,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bda80e0e93e23147241ead HTTP/1.1" 200 None 2026-03-22 20:03:41,477 - lava_callback - ERROR - Error processing callback for node 69bda80e0e93e23147241ead: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33402 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:42:01,905 - lava_callback - INFO - Received callback for node 69bdb1090e93e23147241efb INFO: 172.20.0.1:56650 - "POST /node/69bdb1090e93e23147241efb HTTP/1.0" 202 Accepted 2026-03-22 20:42:01,910 - lava_callback - INFO - Starting async job submit for node 69bdb1090e93e23147241efb 2026-03-22 20:42:01,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:42:01,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb1090e93e23147241efb HTTP/1.1" 200 None 2026-03-22 20:42:01,956 - lava_callback - DEBUG - Processing node 69bdb1090e93e23147241efb: baseline-arm-baylibre 2026-03-22 20:42:01,956 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb1090e93e23147241efb 2026-03-22 20:42:01,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:42:01,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb1090e93e23147241efb HTTP/1.1" 200 None 2026-03-22 20:42:01,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:42:02,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:42:02,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:42:02,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb1090e93e23147241efb HTTP/1.1" 200 None 2026-03-22 20:42:02,036 - lava_callback - ERROR - Error processing callback for node 69bdb1090e93e23147241efb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37670 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:42:40,984 - lava_callback - INFO - Received callback for node 69bdb13c0e93e23147241f08 INFO: 172.20.0.1:54654 - "POST /node/69bdb13c0e93e23147241f08 HTTP/1.0" 202 Accepted 2026-03-22 20:42:40,990 - lava_callback - INFO - Starting async job submit for node 69bdb13c0e93e23147241f08 2026-03-22 20:42:40,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:42:41,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb13c0e93e23147241f08 HTTP/1.1" 200 None 2026-03-22 20:42:41,006 - lava_callback - DEBUG - Processing node 69bdb13c0e93e23147241f08: baseline-arm-baylibre 2026-03-22 20:42:41,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb13c0e93e23147241f08 2026-03-22 20:42:41,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:42:41,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb13c0e93e23147241f08 HTTP/1.1" 200 None 2026-03-22 20:42:41,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:42:41,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:42:41,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:42:41,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb13c0e93e23147241f08 HTTP/1.1" 200 None 2026-03-22 20:42:41,084 - lava_callback - ERROR - Error processing callback for node 69bdb13c0e93e23147241f08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45954 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:43:21,030 - lava_callback - INFO - Received callback for node 69bdb1660e93e23147241f1d INFO: 172.20.0.1:41846 - "POST /node/69bdb1660e93e23147241f1d HTTP/1.0" 202 Accepted 2026-03-22 20:43:21,035 - lava_callback - INFO - Starting async job submit for node 69bdb1660e93e23147241f1d 2026-03-22 20:43:21,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:43:21,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb1660e93e23147241f1d HTTP/1.1" 200 None 2026-03-22 20:43:21,066 - lava_callback - DEBUG - Processing node 69bdb1660e93e23147241f1d: baseline-arm-baylibre 2026-03-22 20:43:21,066 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb1660e93e23147241f1d 2026-03-22 20:43:21,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:43:21,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb1660e93e23147241f1d HTTP/1.1" 200 None 2026-03-22 20:43:21,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:43:21,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:43:21,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:43:21,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb1660e93e23147241f1d HTTP/1.1" 200 None 2026-03-22 20:43:21,160 - lava_callback - ERROR - Error processing callback for node 69bdb1660e93e23147241f1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55890 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:45:01,111 - lava_callback - INFO - Received callback for node 69bdb1ba0e93e23147241f39 INFO: 172.20.0.1:50520 - "POST /node/69bdb1ba0e93e23147241f39 HTTP/1.0" 202 Accepted 2026-03-22 20:45:01,117 - lava_callback - INFO - Starting async job submit for node 69bdb1ba0e93e23147241f39 2026-03-22 20:45:01,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:45:01,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb1ba0e93e23147241f39 HTTP/1.1" 200 None 2026-03-22 20:45:01,133 - lava_callback - DEBUG - Processing node 69bdb1ba0e93e23147241f39: baseline-arm-baylibre 2026-03-22 20:45:01,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb1ba0e93e23147241f39 2026-03-22 20:45:01,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:45:01,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb1ba0e93e23147241f39 HTTP/1.1" 200 None 2026-03-22 20:45:01,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:45:01,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:45:01,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:45:01,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb1ba0e93e23147241f39 HTTP/1.1" 200 None 2026-03-22 20:45:01,195 - lava_callback - ERROR - Error processing callback for node 69bdb1ba0e93e23147241f39: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34362 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:46:21,232 - lava_callback - INFO - Received callback for node 69bdb20e0e93e23147241f8f INFO: 172.20.0.1:39374 - "POST /node/69bdb20e0e93e23147241f8f HTTP/1.0" 202 Accepted 2026-03-22 20:46:21,238 - lava_callback - INFO - Starting async job submit for node 69bdb20e0e93e23147241f8f 2026-03-22 20:46:21,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:46:21,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb20e0e93e23147241f8f HTTP/1.1" 200 None 2026-03-22 20:46:21,254 - lava_callback - DEBUG - Processing node 69bdb20e0e93e23147241f8f: baseline-arm-baylibre 2026-03-22 20:46:21,255 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb20e0e93e23147241f8f 2026-03-22 20:46:21,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:46:21,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb20e0e93e23147241f8f HTTP/1.1" 200 None 2026-03-22 20:46:21,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:46:21,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:46:21,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:46:21,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb20e0e93e23147241f8f HTTP/1.1" 200 None 2026-03-22 20:46:21,304 - lava_callback - ERROR - Error processing callback for node 69bdb20e0e93e23147241f8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45118 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:47:01,211 - lava_callback - INFO - Received callback for node 69bdb2380e93e23147241fa2 INFO: 172.20.0.1:56456 - "POST /node/69bdb2380e93e23147241fa2 HTTP/1.0" 202 Accepted 2026-03-22 20:47:01,217 - lava_callback - INFO - Starting async job submit for node 69bdb2380e93e23147241fa2 2026-03-22 20:47:01,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:47:01,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb2380e93e23147241fa2 HTTP/1.1" 200 None 2026-03-22 20:47:01,232 - lava_callback - DEBUG - Processing node 69bdb2380e93e23147241fa2: baseline-arm-baylibre 2026-03-22 20:47:01,232 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb2380e93e23147241fa2 2026-03-22 20:47:01,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:47:01,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb2380e93e23147241fa2 HTTP/1.1" 200 None 2026-03-22 20:47:01,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:47:01,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:47:01,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:47:01,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb2380e93e23147241fa2 HTTP/1.1" 200 None 2026-03-22 20:47:01,298 - lava_callback - ERROR - Error processing callback for node 69bdb2380e93e23147241fa2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40864 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:49:01,397 - lava_callback - INFO - Received callback for node 69bdb2b30e93e23147241fdb INFO: 172.20.0.1:47576 - "POST /node/69bdb2b30e93e23147241fdb HTTP/1.0" 202 Accepted 2026-03-22 20:49:01,403 - lava_callback - INFO - Starting async job submit for node 69bdb2b30e93e23147241fdb 2026-03-22 20:49:01,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:49:01,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb2b30e93e23147241fdb HTTP/1.1" 200 None 2026-03-22 20:49:01,424 - lava_callback - DEBUG - Processing node 69bdb2b30e93e23147241fdb: baseline-arm-baylibre 2026-03-22 20:49:01,424 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb2b30e93e23147241fdb 2026-03-22 20:49:01,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:49:01,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb2b30e93e23147241fdb HTTP/1.1" 200 None 2026-03-22 20:49:01,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:49:01,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:49:01,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:49:01,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb2b30e93e23147241fdb HTTP/1.1" 200 None 2026-03-22 20:49:01,481 - lava_callback - ERROR - Error processing callback for node 69bdb2b30e93e23147241fdb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42566 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:49:41,341 - lava_callback - INFO - Received callback for node 69bdb2e00e93e23147241fe8 INFO: 172.20.0.1:58726 - "POST /node/69bdb2e00e93e23147241fe8 HTTP/1.0" 202 Accepted 2026-03-22 20:49:41,346 - lava_callback - INFO - Starting async job submit for node 69bdb2e00e93e23147241fe8 2026-03-22 20:49:41,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:49:41,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb2e00e93e23147241fe8 HTTP/1.1" 200 None 2026-03-22 20:49:41,384 - lava_callback - DEBUG - Processing node 69bdb2e00e93e23147241fe8: baseline-arm-baylibre 2026-03-22 20:49:41,385 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb2e00e93e23147241fe8 2026-03-22 20:49:41,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:49:41,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb2e00e93e23147241fe8 HTTP/1.1" 200 None 2026-03-22 20:49:41,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:49:41,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-22 20:49:41,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:49:41,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb2e00e93e23147241fe8 HTTP/1.1" 200 None 2026-03-22 20:49:41,434 - lava_callback - ERROR - Error processing callback for node 69bdb2e00e93e23147241fe8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57496 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:50:23,305 - lava_callback - INFO - Received callback for node 69bdb30c0e93e23147241ffe INFO: 172.20.0.1:55574 - "POST /node/69bdb30c0e93e23147241ffe HTTP/1.0" 202 Accepted 2026-03-22 20:50:23,310 - lava_callback - INFO - Starting async job submit for node 69bdb30c0e93e23147241ffe 2026-03-22 20:50:23,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:50:23,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb30c0e93e23147241ffe HTTP/1.1" 200 None 2026-03-22 20:50:23,327 - lava_callback - DEBUG - Processing node 69bdb30c0e93e23147241ffe: baseline-arm-clabbe 2026-03-22 20:50:23,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb30c0e93e23147241ffe 2026-03-22 20:50:23,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:50:23,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb30c0e93e23147241ffe HTTP/1.1" 200 None 2026-03-22 20:50:23,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:50:23,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:50:23,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:50:23,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb30c0e93e23147241ffe HTTP/1.1" 200 None 2026-03-22 20:50:23,386 - lava_callback - ERROR - Error processing callback for node 69bdb30c0e93e23147241ffe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54654 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:51:23,382 - lava_callback - INFO - Received callback for node 69bdb3360e93e2314724200f INFO: 172.20.0.1:47340 - "POST /node/69bdb3360e93e2314724200f HTTP/1.0" 202 Accepted 2026-03-22 20:51:23,388 - lava_callback - INFO - Starting async job submit for node 69bdb3360e93e2314724200f 2026-03-22 20:51:23,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:51:23,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb3360e93e2314724200f HTTP/1.1" 200 None 2026-03-22 20:51:23,404 - lava_callback - DEBUG - Processing node 69bdb3360e93e2314724200f: baseline-arm-clabbe 2026-03-22 20:51:23,404 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb3360e93e2314724200f 2026-03-22 20:51:23,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:51:23,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb3360e93e2314724200f HTTP/1.1" 200 None 2026-03-22 20:51:23,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:51:23,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:51:23,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:51:23,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb3360e93e2314724200f HTTP/1.1" 200 None 2026-03-22 20:51:23,470 - lava_callback - ERROR - Error processing callback for node 69bdb3360e93e2314724200f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36316 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:53:23,443 - lava_callback - INFO - Received callback for node 69bdb3b10e93e23147242045 INFO: 172.20.0.1:34590 - "POST /node/69bdb3b10e93e23147242045 HTTP/1.0" 202 Accepted 2026-03-22 20:53:23,448 - lava_callback - INFO - Starting async job submit for node 69bdb3b10e93e23147242045 2026-03-22 20:53:23,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:53:23,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb3b10e93e23147242045 HTTP/1.1" 200 None 2026-03-22 20:53:23,464 - lava_callback - DEBUG - Processing node 69bdb3b10e93e23147242045: baseline-arm-clabbe 2026-03-22 20:53:23,465 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb3b10e93e23147242045 2026-03-22 20:53:23,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:53:23,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb3b10e93e23147242045 HTTP/1.1" 200 None 2026-03-22 20:53:23,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:53:23,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:53:23,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:53:23,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb3b10e93e23147242045 HTTP/1.1" 200 None 2026-03-22 20:53:23,523 - lava_callback - ERROR - Error processing callback for node 69bdb3b10e93e23147242045: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37792 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:54:03,428 - lava_callback - INFO - Received callback for node 69bdb3db0e93e23147242055 INFO: 172.20.0.1:41258 - "POST /node/69bdb3db0e93e23147242055 HTTP/1.0" 202 Accepted 2026-03-22 20:54:03,434 - lava_callback - INFO - Starting async job submit for node 69bdb3db0e93e23147242055 2026-03-22 20:54:03,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:54:03,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb3db0e93e23147242055 HTTP/1.1" 200 None 2026-03-22 20:54:03,450 - lava_callback - DEBUG - Processing node 69bdb3db0e93e23147242055: baseline-arm-clabbe 2026-03-22 20:54:03,450 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb3db0e93e23147242055 2026-03-22 20:54:03,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:54:03,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb3db0e93e23147242055 HTTP/1.1" 200 None 2026-03-22 20:54:03,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:54:03,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-22 20:54:03,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:54:03,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb3db0e93e23147242055 HTTP/1.1" 200 None 2026-03-22 20:54:03,506 - lava_callback - ERROR - Error processing callback for node 69bdb3db0e93e23147242055: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54574 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:54:43,470 - lava_callback - INFO - Received callback for node 69bdb4050e93e2314724206e INFO: 172.20.0.1:51192 - "POST /node/69bdb4050e93e2314724206e HTTP/1.0" 202 Accepted 2026-03-22 20:54:43,476 - lava_callback - INFO - Starting async job submit for node 69bdb4050e93e2314724206e 2026-03-22 20:54:43,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:54:43,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb4050e93e2314724206e HTTP/1.1" 200 None 2026-03-22 20:54:43,492 - lava_callback - DEBUG - Processing node 69bdb4050e93e2314724206e: baseline-arm-clabbe 2026-03-22 20:54:43,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb4050e93e2314724206e 2026-03-22 20:54:43,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:54:43,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb4050e93e2314724206e HTTP/1.1" 200 None 2026-03-22 20:54:43,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:54:43,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:54:43,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:54:43,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb4050e93e2314724206e HTTP/1.1" 200 None 2026-03-22 20:54:43,554 - lava_callback - ERROR - Error processing callback for node 69bdb4050e93e2314724206e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55530 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:55:23,452 - lava_callback - INFO - Received callback for node 69bdb42f0e93e2314724207d INFO: 172.20.0.1:46276 - "POST /node/69bdb42f0e93e2314724207d HTTP/1.0" 202 Accepted 2026-03-22 20:55:23,457 - lava_callback - INFO - Starting async job submit for node 69bdb42f0e93e2314724207d 2026-03-22 20:55:23,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:55:23,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb42f0e93e2314724207d HTTP/1.1" 200 None 2026-03-22 20:55:23,473 - lava_callback - DEBUG - Processing node 69bdb42f0e93e2314724207d: baseline-arm-clabbe 2026-03-22 20:55:23,473 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb42f0e93e2314724207d 2026-03-22 20:55:23,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:55:23,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb42f0e93e2314724207d HTTP/1.1" 200 None 2026-03-22 20:55:23,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:55:23,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-22 20:55:23,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:55:23,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb42f0e93e2314724207d HTTP/1.1" 200 None 2026-03-22 20:55:23,539 - lava_callback - ERROR - Error processing callback for node 69bdb42f0e93e2314724207d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54306 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:56:03,602 - lava_callback - INFO - Received callback for node 69bdb4590e93e23147242095 INFO: 172.20.0.1:51896 - "POST /node/69bdb4590e93e23147242095 HTTP/1.0" 202 Accepted 2026-03-22 20:56:03,608 - lava_callback - INFO - Starting async job submit for node 69bdb4590e93e23147242095 2026-03-22 20:56:03,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:56:03,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb4590e93e23147242095 HTTP/1.1" 200 None 2026-03-22 20:56:03,646 - lava_callback - DEBUG - Processing node 69bdb4590e93e23147242095: baseline-arm-clabbe 2026-03-22 20:56:03,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb4590e93e23147242095 2026-03-22 20:56:03,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:56:03,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb4590e93e23147242095 HTTP/1.1" 200 None 2026-03-22 20:56:03,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:56:03,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-22 20:56:03,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:56:03,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb4590e93e23147242095 HTTP/1.1" 200 None 2026-03-22 20:56:03,700 - lava_callback - ERROR - Error processing callback for node 69bdb4590e93e23147242095: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43368 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:56:43,544 - lava_callback - INFO - Received callback for node 69bdb4830e93e231472420a4 INFO: 172.20.0.1:53532 - "POST /node/69bdb4830e93e231472420a4 HTTP/1.0" 202 Accepted 2026-03-22 20:56:43,552 - lava_callback - INFO - Starting async job submit for node 69bdb4830e93e231472420a4 2026-03-22 20:56:43,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:56:43,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb4830e93e231472420a4 HTTP/1.1" 200 None 2026-03-22 20:56:43,571 - lava_callback - DEBUG - Processing node 69bdb4830e93e231472420a4: baseline-arm-clabbe 2026-03-22 20:56:43,571 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb4830e93e231472420a4 2026-03-22 20:56:43,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:56:43,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb4830e93e231472420a4 HTTP/1.1" 200 None 2026-03-22 20:56:43,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:56:43,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:56:43,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:56:43,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb4830e93e231472420a4 HTTP/1.1" 200 None 2026-03-22 20:56:43,630 - lava_callback - ERROR - Error processing callback for node 69bdb4830e93e231472420a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46834 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:57:23,844 - lava_callback - INFO - Received callback for node 69bdb4ad0e93e231472420b3 INFO: 172.20.0.1:51030 - "POST /node/69bdb4ad0e93e231472420b3 HTTP/1.0" 202 Accepted 2026-03-22 20:57:23,849 - lava_callback - INFO - Starting async job submit for node 69bdb4ad0e93e231472420b3 2026-03-22 20:57:23,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:57:23,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb4ad0e93e231472420b3 HTTP/1.1" 200 None 2026-03-22 20:57:23,866 - lava_callback - DEBUG - Processing node 69bdb4ad0e93e231472420b3: baseline-arm-clabbe 2026-03-22 20:57:23,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb4ad0e93e231472420b3 2026-03-22 20:57:23,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:57:23,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb4ad0e93e231472420b3 HTTP/1.1" 200 None 2026-03-22 20:57:23,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:57:23,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-22 20:57:23,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:57:23,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb4ad0e93e231472420b3 HTTP/1.1" 200 None 2026-03-22 20:57:23,918 - lava_callback - ERROR - Error processing callback for node 69bdb4ad0e93e231472420b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51806 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:58:03,616 - lava_callback - INFO - Received callback for node 69bdb4d80e93e231472420c5 INFO: 172.20.0.1:34580 - "POST /node/69bdb4d80e93e231472420c5 HTTP/1.0" 202 Accepted 2026-03-22 20:58:03,622 - lava_callback - INFO - Starting async job submit for node 69bdb4d80e93e231472420c5 2026-03-22 20:58:03,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:58:03,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb4d80e93e231472420c5 HTTP/1.1" 200 None 2026-03-22 20:58:03,637 - lava_callback - DEBUG - Processing node 69bdb4d80e93e231472420c5: baseline-arm-clabbe 2026-03-22 20:58:03,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb4d80e93e231472420c5 2026-03-22 20:58:03,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:58:03,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb4d80e93e231472420c5 HTTP/1.1" 200 None 2026-03-22 20:58:03,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:58:03,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-22 20:58:03,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:58:03,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb4d80e93e231472420c5 HTTP/1.1" 200 None 2026-03-22 20:58:03,701 - lava_callback - ERROR - Error processing callback for node 69bdb4d80e93e231472420c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:42060 - "GET / HTTP/1.0" 200 OK 2026-03-22 20:59:03,681 - lava_callback - INFO - Received callback for node 69bdb5020e93e231472420d6 INFO: 172.20.0.1:33116 - "POST /node/69bdb5020e93e231472420d6 HTTP/1.0" 202 Accepted 2026-03-22 20:59:03,686 - lava_callback - INFO - Starting async job submit for node 69bdb5020e93e231472420d6 2026-03-22 20:59:03,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:59:03,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb5020e93e231472420d6 HTTP/1.1" 200 None 2026-03-22 20:59:03,712 - lava_callback - DEBUG - Processing node 69bdb5020e93e231472420d6: baseline-arm-clabbe 2026-03-22 20:59:03,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb5020e93e231472420d6 2026-03-22 20:59:03,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:59:03,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb5020e93e231472420d6 HTTP/1.1" 200 None 2026-03-22 20:59:03,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:59:03,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 20:59:03,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 20:59:03,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb5020e93e231472420d6 HTTP/1.1" 200 None 2026-03-22 20:59:03,772 - lava_callback - ERROR - Error processing callback for node 69bdb5020e93e231472420d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44910 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:00:23,876 - lava_callback - INFO - Received callback for node 69bdb5560e93e231472420f7 INFO: 172.20.0.1:60458 - "POST /node/69bdb5560e93e231472420f7 HTTP/1.0" 202 Accepted 2026-03-22 21:00:23,882 - lava_callback - INFO - Starting async job submit for node 69bdb5560e93e231472420f7 2026-03-22 21:00:23,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:00:23,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb5560e93e231472420f7 HTTP/1.1" 200 None 2026-03-22 21:00:23,897 - lava_callback - DEBUG - Processing node 69bdb5560e93e231472420f7: baseline-arm-clabbe 2026-03-22 21:00:23,898 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb5560e93e231472420f7 2026-03-22 21:00:23,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:00:23,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb5560e93e231472420f7 HTTP/1.1" 200 None 2026-03-22 21:00:23,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:00:23,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 21:00:23,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:00:23,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb5560e93e231472420f7 HTTP/1.1" 200 None 2026-03-22 21:00:23,957 - lava_callback - ERROR - Error processing callback for node 69bdb5560e93e231472420f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42752 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:01:42,165 - lava_callback - INFO - Received callback for node 69bdb5b00e93e23147242114 INFO: 172.20.0.1:37650 - "POST /node/69bdb5b00e93e23147242114 HTTP/1.0" 202 Accepted 2026-03-22 21:01:42,171 - lava_callback - INFO - Starting async job submit for node 69bdb5b00e93e23147242114 2026-03-22 21:01:42,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:01:42,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb5b00e93e23147242114 HTTP/1.1" 200 None 2026-03-22 21:01:42,187 - lava_callback - DEBUG - Processing node 69bdb5b00e93e23147242114: baseline-arm64-baylibre 2026-03-22 21:01:42,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb5b00e93e23147242114 2026-03-22 21:01:42,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:01:42,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb5b00e93e23147242114 HTTP/1.1" 200 None 2026-03-22 21:01:42,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:01:42,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:01:42,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:01:42,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb5b00e93e23147242114 HTTP/1.1" 200 None 2026-03-22 21:01:42,249 - lava_callback - ERROR - Error processing callback for node 69bdb5b00e93e23147242114: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:03:03,886 - lava_callback - INFO - Received callback for node 69bdb5fb0e93e23147242121 INFO: 172.20.0.1:58454 - "POST /node/69bdb5fb0e93e23147242121 HTTP/1.0" 202 Accepted 2026-03-22 21:03:03,892 - lava_callback - INFO - Starting async job submit for node 69bdb5fb0e93e23147242121 2026-03-22 21:03:03,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:03:03,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb5fb0e93e23147242121 HTTP/1.1" 200 None 2026-03-22 21:03:03,910 - lava_callback - DEBUG - Processing node 69bdb5fb0e93e23147242121: baseline-arm-clabbe 2026-03-22 21:03:03,910 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb5fb0e93e23147242121 2026-03-22 21:03:03,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:03:03,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb5fb0e93e23147242121 HTTP/1.1" 200 None 2026-03-22 21:03:03,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:03:03,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 21:03:03,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:03:03,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb5fb0e93e23147242121 HTTP/1.1" 200 None 2026-03-22 21:03:03,989 - lava_callback - ERROR - Error processing callback for node 69bdb5fb0e93e23147242121: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35090 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:03:43,823 - lava_callback - INFO - Received callback for node 69bdb6230e93e23147242158 INFO: 172.20.0.1:45072 - "POST /node/69bdb6230e93e23147242158 HTTP/1.0" 202 Accepted 2026-03-22 21:03:43,829 - lava_callback - INFO - Starting async job submit for node 69bdb6230e93e23147242158 2026-03-22 21:03:43,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:03:43,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb6230e93e23147242158 HTTP/1.1" 200 None 2026-03-22 21:03:43,846 - lava_callback - DEBUG - Processing node 69bdb6230e93e23147242158: baseline-arm-clabbe 2026-03-22 21:03:43,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb6230e93e23147242158 2026-03-22 21:03:43,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:03:43,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb6230e93e23147242158 HTTP/1.1" 200 None 2026-03-22 21:03:43,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:03:43,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 21:03:43,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:03:43,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb6230e93e23147242158 HTTP/1.1" 200 None 2026-03-22 21:03:43,906 - lava_callback - ERROR - Error processing callback for node 69bdb6230e93e23147242158: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49136 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:05:42,325 - lava_callback - INFO - Received callback for node 69bdb6a00e93e2314724219b INFO: 172.20.0.1:59142 - "POST /node/69bdb6a00e93e2314724219b HTTP/1.0" 202 Accepted 2026-03-22 21:05:42,331 - lava_callback - INFO - Starting async job submit for node 69bdb6a00e93e2314724219b 2026-03-22 21:05:42,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:05:42,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb6a00e93e2314724219b HTTP/1.1" 200 None 2026-03-22 21:05:42,348 - lava_callback - DEBUG - Processing node 69bdb6a00e93e2314724219b: baseline-arm64-baylibre 2026-03-22 21:05:42,348 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb6a00e93e2314724219b 2026-03-22 21:05:42,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:05:42,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb6a00e93e2314724219b HTTP/1.1" 200 None 2026-03-22 21:05:42,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:05:42,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:05:42,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:05:42,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb6a00e93e2314724219b HTTP/1.1" 200 None 2026-03-22 21:05:42,399 - lava_callback - ERROR - Error processing callback for node 69bdb6a00e93e2314724219b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57056 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:07:02,391 - lava_callback - INFO - Received callback for node 69bdb6f20e93e231472421aa INFO: 172.20.0.1:50548 - "POST /node/69bdb6f20e93e231472421aa HTTP/1.0" 202 Accepted 2026-03-22 21:07:02,397 - lava_callback - INFO - Starting async job submit for node 69bdb6f20e93e231472421aa 2026-03-22 21:07:02,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:07:02,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb6f20e93e231472421aa HTTP/1.1" 200 None 2026-03-22 21:07:02,417 - lava_callback - DEBUG - Processing node 69bdb6f20e93e231472421aa: baseline-arm64-baylibre 2026-03-22 21:07:02,418 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb6f20e93e231472421aa 2026-03-22 21:07:02,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:07:02,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb6f20e93e231472421aa HTTP/1.1" 200 None 2026-03-22 21:07:02,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:07:02,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:07:02,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:07:02,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb6f20e93e231472421aa HTTP/1.1" 200 None 2026-03-22 21:07:02,475 - lava_callback - ERROR - Error processing callback for node 69bdb6f20e93e231472421aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37102 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:11:02,481 - lava_callback - INFO - Received callback for node 69bdb7e10e93e231472421d1 INFO: 172.20.0.1:33646 - "POST /node/69bdb7e10e93e231472421d1 HTTP/1.0" 202 Accepted 2026-03-22 21:11:02,486 - lava_callback - INFO - Starting async job submit for node 69bdb7e10e93e231472421d1 2026-03-22 21:11:02,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:11:02,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb7e10e93e231472421d1 HTTP/1.1" 200 None 2026-03-22 21:11:02,502 - lava_callback - DEBUG - Processing node 69bdb7e10e93e231472421d1: baseline-arm64-baylibre 2026-03-22 21:11:02,502 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb7e10e93e231472421d1 2026-03-22 21:11:02,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:11:02,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb7e10e93e231472421d1 HTTP/1.1" 200 None 2026-03-22 21:11:02,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:11:02,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:11:02,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:11:02,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb7e10e93e231472421d1 HTTP/1.1" 200 None 2026-03-22 21:11:02,556 - lava_callback - ERROR - Error processing callback for node 69bdb7e10e93e231472421d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55566 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:13:42,677 - lava_callback - INFO - Received callback for node 69bdb87e0e93e231472421ee INFO: 172.20.0.1:59526 - "POST /node/69bdb87e0e93e231472421ee HTTP/1.0" 202 Accepted 2026-03-22 21:13:42,687 - lava_callback - INFO - Starting async job submit for node 69bdb87e0e93e231472421ee 2026-03-22 21:13:42,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:13:42,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb87e0e93e231472421ee HTTP/1.1" 200 None 2026-03-22 21:13:42,720 - lava_callback - DEBUG - Processing node 69bdb87e0e93e231472421ee: baseline-arm64-baylibre 2026-03-22 21:13:42,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb87e0e93e231472421ee 2026-03-22 21:13:42,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:13:42,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb87e0e93e231472421ee HTTP/1.1" 200 None 2026-03-22 21:13:42,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:13:42,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:13:42,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:13:42,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb87e0e93e231472421ee HTTP/1.1" 200 None 2026-03-22 21:13:42,812 - lava_callback - ERROR - Error processing callback for node 69bdb87e0e93e231472421ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36428 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:16:22,800 - lava_callback - INFO - Received callback for node 69bdb91b0e93e2314724220d INFO: 172.20.0.1:35248 - "POST /node/69bdb91b0e93e2314724220d HTTP/1.0" 202 Accepted 2026-03-22 21:16:22,805 - lava_callback - INFO - Starting async job submit for node 69bdb91b0e93e2314724220d 2026-03-22 21:16:22,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:16:22,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb91b0e93e2314724220d HTTP/1.1" 200 None 2026-03-22 21:16:22,822 - lava_callback - DEBUG - Processing node 69bdb91b0e93e2314724220d: baseline-arm64-baylibre 2026-03-22 21:16:22,822 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb91b0e93e2314724220d 2026-03-22 21:16:22,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:16:22,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb91b0e93e2314724220d HTTP/1.1" 200 None 2026-03-22 21:16:22,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:16:22,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:16:22,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:16:22,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb91b0e93e2314724220d HTTP/1.1" 200 None 2026-03-22 21:16:22,883 - lava_callback - ERROR - Error processing callback for node 69bdb91b0e93e2314724220d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36948 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:17:44,427 - lava_callback - INFO - Received callback for node 69bdb9660e93e23147242223 INFO: 172.20.0.1:36958 - "POST /node/69bdb9660e93e23147242223 HTTP/1.0" 202 Accepted 2026-03-22 21:17:44,432 - lava_callback - INFO - Starting async job submit for node 69bdb9660e93e23147242223 2026-03-22 21:17:44,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:17:44,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb9660e93e23147242223 HTTP/1.1" 200 None 2026-03-22 21:17:44,454 - lava_callback - DEBUG - Processing node 69bdb9660e93e23147242223: baseline-arm-clabbe 2026-03-22 21:17:44,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb9660e93e23147242223 2026-03-22 21:17:44,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:17:44,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb9660e93e23147242223 HTTP/1.1" 200 None 2026-03-22 21:17:44,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:17:44,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 21:17:44,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:17:44,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb9660e93e23147242223 HTTP/1.1" 200 None 2026-03-22 21:17:44,510 - lava_callback - ERROR - Error processing callback for node 69bdb9660e93e23147242223: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43078 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:18:23,075 - lava_callback - INFO - Received callback for node 69bdb9980e93e2314724222c INFO: 172.20.0.1:35584 - "POST /node/69bdb9980e93e2314724222c HTTP/1.0" 202 Accepted 2026-03-22 21:18:23,080 - lava_callback - INFO - Starting async job submit for node 69bdb9980e93e2314724222c 2026-03-22 21:18:23,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:18:23,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb9980e93e2314724222c HTTP/1.1" 200 None 2026-03-22 21:18:23,096 - lava_callback - DEBUG - Processing node 69bdb9980e93e2314724222c: baseline-arm64-baylibre 2026-03-22 21:18:23,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb9980e93e2314724222c 2026-03-22 21:18:23,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:18:23,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb9980e93e2314724222c HTTP/1.1" 200 None 2026-03-22 21:18:23,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:18:23,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:18:23,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:18:23,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb9980e93e2314724222c HTTP/1.1" 200 None 2026-03-22 21:18:23,158 - lava_callback - ERROR - Error processing callback for node 69bdb9980e93e2314724222c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57934 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:19:42,958 - lava_callback - INFO - Received callback for node 69bdb9e90e93e2314724223c INFO: 172.20.0.1:47606 - "POST /node/69bdb9e90e93e2314724223c HTTP/1.0" 202 Accepted 2026-03-22 21:19:42,965 - lava_callback - INFO - Starting async job submit for node 69bdb9e90e93e2314724223c 2026-03-22 21:19:42,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:19:42,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb9e90e93e2314724223c HTTP/1.1" 200 None 2026-03-22 21:19:42,985 - lava_callback - DEBUG - Processing node 69bdb9e90e93e2314724223c: baseline-arm64-baylibre 2026-03-22 21:19:42,985 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdb9e90e93e2314724223c 2026-03-22 21:19:42,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:19:43,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdb9e90e93e2314724223c HTTP/1.1" 200 None 2026-03-22 21:19:43,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:19:43,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-22 21:19:43,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:19:43,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdb9e90e93e2314724223c HTTP/1.1" 200 None 2026-03-22 21:19:43,060 - lava_callback - ERROR - Error processing callback for node 69bdb9e90e93e2314724223c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37172 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:21:03,066 - lava_callback - INFO - Received callback for node 69bdba370e93e2314724224d INFO: 172.20.0.1:53932 - "POST /node/69bdba370e93e2314724224d HTTP/1.0" 202 Accepted 2026-03-22 21:21:03,072 - lava_callback - INFO - Starting async job submit for node 69bdba370e93e2314724224d 2026-03-22 21:21:03,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:21:03,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdba370e93e2314724224d HTTP/1.1" 200 None 2026-03-22 21:21:03,091 - lava_callback - DEBUG - Processing node 69bdba370e93e2314724224d: baseline-arm64-baylibre 2026-03-22 21:21:03,091 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdba370e93e2314724224d 2026-03-22 21:21:03,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:21:03,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdba370e93e2314724224d HTTP/1.1" 200 None 2026-03-22 21:21:03,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:21:03,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:21:03,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:21:03,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdba370e93e2314724224d HTTP/1.1" 200 None 2026-03-22 21:21:03,142 - lava_callback - ERROR - Error processing callback for node 69bdba370e93e2314724224d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41366 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:22:24,728 - lava_callback - INFO - Received callback for node 69bdba830e93e2314724225d INFO: 172.20.0.1:41786 - "POST /node/69bdba830e93e2314724225d HTTP/1.0" 202 Accepted 2026-03-22 21:22:24,733 - lava_callback - INFO - Starting async job submit for node 69bdba830e93e2314724225d 2026-03-22 21:22:24,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:22:24,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdba830e93e2314724225d HTTP/1.1" 200 None 2026-03-22 21:22:24,753 - lava_callback - DEBUG - Processing node 69bdba830e93e2314724225d: baseline-arm-clabbe 2026-03-22 21:22:24,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdba830e93e2314724225d 2026-03-22 21:22:24,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:22:24,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdba830e93e2314724225d HTTP/1.1" 200 None 2026-03-22 21:22:24,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:22:24,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 21:22:24,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:22:24,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdba830e93e2314724225d HTTP/1.1" 200 None 2026-03-22 21:22:24,805 - lava_callback - ERROR - Error processing callback for node 69bdba830e93e2314724225d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43376 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:23:24,730 - lava_callback - INFO - Received callback for node 69bdbabd0e93e23147242274 INFO: 172.20.0.1:57536 - "POST /node/69bdbabd0e93e23147242274 HTTP/1.0" 202 Accepted 2026-03-22 21:23:24,736 - lava_callback - INFO - Starting async job submit for node 69bdbabd0e93e23147242274 2026-03-22 21:23:24,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:23:24,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbabd0e93e23147242274 HTTP/1.1" 200 None 2026-03-22 21:23:24,752 - lava_callback - DEBUG - Processing node 69bdbabd0e93e23147242274: baseline-arm64-clabbe 2026-03-22 21:23:24,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbabd0e93e23147242274 2026-03-22 21:23:24,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:23:24,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbabd0e93e23147242274 HTTP/1.1" 200 None 2026-03-22 21:23:24,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:23:24,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-22 21:23:24,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:23:24,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbabd0e93e23147242274 HTTP/1.1" 200 None 2026-03-22 21:23:24,805 - lava_callback - ERROR - Error processing callback for node 69bdbabd0e93e23147242274: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55844 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:24:43,271 - lava_callback - INFO - Received callback for node 69bdbb070e93e23147242289 INFO: 172.20.0.1:60778 - "POST /node/69bdbb070e93e23147242289 HTTP/1.0" 202 Accepted 2026-03-22 21:24:43,276 - lava_callback - INFO - Starting async job submit for node 69bdbb070e93e23147242289 2026-03-22 21:24:43,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:24:43,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbb070e93e23147242289 HTTP/1.1" 200 None 2026-03-22 21:24:43,295 - lava_callback - DEBUG - Processing node 69bdbb070e93e23147242289: baseline-arm64-baylibre 2026-03-22 21:24:43,295 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbb070e93e23147242289 2026-03-22 21:24:43,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:24:43,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbb070e93e23147242289 HTTP/1.1" 200 None 2026-03-22 21:24:43,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:24:43,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:24:43,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:24:43,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbb070e93e23147242289 HTTP/1.1" 200 None 2026-03-22 21:24:43,355 - lava_callback - ERROR - Error processing callback for node 69bdbb070e93e23147242289: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50914 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:26:03,397 - lava_callback - INFO - Received callback for node 69bdbb590e93e2314724228e INFO: 172.20.0.1:51056 - "POST /node/69bdbb590e93e2314724228e HTTP/1.0" 202 Accepted 2026-03-22 21:26:03,403 - lava_callback - INFO - Starting async job submit for node 69bdbb590e93e2314724228e 2026-03-22 21:26:03,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:26:03,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbb590e93e2314724228e HTTP/1.1" 200 None 2026-03-22 21:26:03,419 - lava_callback - DEBUG - Processing node 69bdbb590e93e2314724228e: baseline-arm64-baylibre 2026-03-22 21:26:03,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbb590e93e2314724228e 2026-03-22 21:26:03,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:26:03,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbb590e93e2314724228e HTTP/1.1" 200 None 2026-03-22 21:26:03,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:26:03,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:26:03,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:26:03,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbb590e93e2314724228e HTTP/1.1" 200 None 2026-03-22 21:26:03,477 - lava_callback - ERROR - Error processing callback for node 69bdbb590e93e2314724228e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58094 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:27:24,916 - lava_callback - INFO - Received callback for node 69bdbbb10e93e231472422a2 INFO: 172.20.0.1:48080 - "POST /node/69bdbbb10e93e231472422a2 HTTP/1.0" 202 Accepted 2026-03-22 21:27:24,922 - lava_callback - INFO - Starting async job submit for node 69bdbbb10e93e231472422a2 2026-03-22 21:27:24,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:27:24,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbbb10e93e231472422a2 HTTP/1.1" 200 None 2026-03-22 21:27:24,938 - lava_callback - DEBUG - Processing node 69bdbbb10e93e231472422a2: baseline-arm64-clabbe 2026-03-22 21:27:24,939 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbbb10e93e231472422a2 2026-03-22 21:27:24,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:27:24,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbbb10e93e231472422a2 HTTP/1.1" 200 None 2026-03-22 21:27:24,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:27:24,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-22 21:27:24,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:27:25,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbbb10e93e231472422a2 HTTP/1.1" 200 None 2026-03-22 21:27:25,007 - lava_callback - ERROR - Error processing callback for node 69bdbbb10e93e231472422a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36964 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:28:44,916 - lava_callback - INFO - Received callback for node 69bdbc040e93e231472422b5 INFO: 172.20.0.1:36980 - "POST /node/69bdbc040e93e231472422b5 HTTP/1.0" 202 Accepted 2026-03-22 21:28:44,922 - lava_callback - INFO - Starting async job submit for node 69bdbc040e93e231472422b5 2026-03-22 21:28:44,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:28:44,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbc040e93e231472422b5 HTTP/1.1" 200 None 2026-03-22 21:28:44,937 - lava_callback - DEBUG - Processing node 69bdbc040e93e231472422b5: baseline-arm64-clabbe 2026-03-22 21:28:44,938 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbc040e93e231472422b5 2026-03-22 21:28:44,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:28:44,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbc040e93e231472422b5 HTTP/1.1" 200 None 2026-03-22 21:28:44,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:28:44,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:28:44,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:28:44,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbc040e93e231472422b5 HTTP/1.1" 200 None 2026-03-22 21:28:44,991 - lava_callback - ERROR - Error processing callback for node 69bdbc040e93e231472422b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48780 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:30:05,119 - lava_callback - INFO - Received callback for node 69bdbc550e93e231472422c5 INFO: 172.20.0.1:56296 - "POST /node/69bdbc550e93e231472422c5 HTTP/1.0" 202 Accepted 2026-03-22 21:30:05,124 - lava_callback - INFO - Starting async job submit for node 69bdbc550e93e231472422c5 2026-03-22 21:30:05,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:30:05,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbc550e93e231472422c5 HTTP/1.1" 200 None 2026-03-22 21:30:05,141 - lava_callback - DEBUG - Processing node 69bdbc550e93e231472422c5: baseline-arm64-clabbe 2026-03-22 21:30:05,142 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbc550e93e231472422c5 2026-03-22 21:30:05,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:30:05,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbc550e93e231472422c5 HTTP/1.1" 200 None 2026-03-22 21:30:05,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:30:05,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-22 21:30:05,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:30:05,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbc550e93e231472422c5 HTTP/1.1" 200 None 2026-03-22 21:30:05,197 - lava_callback - ERROR - Error processing callback for node 69bdbc550e93e231472422c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47914 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34384 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:31:25,043 - lava_callback - INFO - Received callback for node 69bdbca60e93e231472422d0 INFO: 172.20.0.1:40764 - "POST /node/69bdbca60e93e231472422d0 HTTP/1.0" 202 Accepted 2026-03-22 21:31:25,048 - lava_callback - INFO - Starting async job submit for node 69bdbca60e93e231472422d0 2026-03-22 21:31:25,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:31:25,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbca60e93e231472422d0 HTTP/1.1" 200 None 2026-03-22 21:31:25,065 - lava_callback - DEBUG - Processing node 69bdbca60e93e231472422d0: baseline-arm64-clabbe 2026-03-22 21:31:25,065 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbca60e93e231472422d0 2026-03-22 21:31:25,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:31:25,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbca60e93e231472422d0 HTTP/1.1" 200 None 2026-03-22 21:31:25,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:31:25,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-22 21:31:25,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:31:25,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbca60e93e231472422d0 HTTP/1.1" 200 None 2026-03-22 21:31:25,126 - lava_callback - ERROR - Error processing callback for node 69bdbca60e93e231472422d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39374 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:32:43,821 - lava_callback - INFO - Received callback for node 69bdbcf20e93e231472422de INFO: 172.20.0.1:40608 - "POST /node/69bdbcf20e93e231472422de HTTP/1.0" 202 Accepted 2026-03-22 21:32:43,827 - lava_callback - INFO - Starting async job submit for node 69bdbcf20e93e231472422de 2026-03-22 21:32:43,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:32:43,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbcf20e93e231472422de HTTP/1.1" 200 None 2026-03-22 21:32:43,845 - lava_callback - DEBUG - Processing node 69bdbcf20e93e231472422de: baseline-arm64-baylibre 2026-03-22 21:32:43,845 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbcf20e93e231472422de 2026-03-22 21:32:43,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:32:43,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbcf20e93e231472422de HTTP/1.1" 200 None 2026-03-22 21:32:43,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:32:43,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:32:43,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:32:43,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbcf20e93e231472422de HTTP/1.1" 200 None 2026-03-22 21:32:43,899 - lava_callback - ERROR - Error processing callback for node 69bdbcf20e93e231472422de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43504 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:34:05,122 - lava_callback - INFO - Received callback for node 69bdbd490e93e231472422f1 INFO: 172.20.0.1:36996 - "POST /node/69bdbd490e93e231472422f1 HTTP/1.0" 202 Accepted 2026-03-22 21:34:05,127 - lava_callback - INFO - Starting async job submit for node 69bdbd490e93e231472422f1 2026-03-22 21:34:05,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:34:05,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbd490e93e231472422f1 HTTP/1.1" 200 None 2026-03-22 21:34:05,144 - lava_callback - DEBUG - Processing node 69bdbd490e93e231472422f1: baseline-arm64-clabbe 2026-03-22 21:34:05,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbd490e93e231472422f1 2026-03-22 21:34:05,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:34:05,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbd490e93e231472422f1 HTTP/1.1" 200 None 2026-03-22 21:34:05,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:34:05,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:34:05,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:34:05,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbd490e93e231472422f1 HTTP/1.1" 200 None 2026-03-22 21:34:05,207 - lava_callback - ERROR - Error processing callback for node 69bdbd490e93e231472422f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40762 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:35:45,247 - lava_callback - INFO - Received callback for node 69bdbd9c0e93e23147242303 INFO: 172.20.0.1:40766 - "POST /node/69bdbd9c0e93e23147242303 HTTP/1.0" 202 Accepted 2026-03-22 21:35:45,253 - lava_callback - INFO - Starting async job submit for node 69bdbd9c0e93e23147242303 2026-03-22 21:35:45,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:35:45,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbd9c0e93e23147242303 HTTP/1.1" 200 None 2026-03-22 21:35:45,269 - lava_callback - DEBUG - Processing node 69bdbd9c0e93e23147242303: baseline-arm64-clabbe 2026-03-22 21:35:45,269 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbd9c0e93e23147242303 2026-03-22 21:35:45,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:35:45,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbd9c0e93e23147242303 HTTP/1.1" 200 None 2026-03-22 21:35:45,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:35:45,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-22 21:35:45,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:35:45,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbd9c0e93e23147242303 HTTP/1.1" 200 None 2026-03-22 21:35:45,325 - lava_callback - ERROR - Error processing callback for node 69bdbd9c0e93e23147242303: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41176 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:37:05,253 - lava_callback - INFO - Received callback for node 69bdbded0e93e2314724230b INFO: 172.20.0.1:39330 - "POST /node/69bdbded0e93e2314724230b HTTP/1.0" 202 Accepted 2026-03-22 21:37:05,258 - lava_callback - INFO - Starting async job submit for node 69bdbded0e93e2314724230b 2026-03-22 21:37:05,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:37:05,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbded0e93e2314724230b HTTP/1.1" 200 None 2026-03-22 21:37:05,278 - lava_callback - DEBUG - Processing node 69bdbded0e93e2314724230b: baseline-arm64-clabbe 2026-03-22 21:37:05,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbded0e93e2314724230b 2026-03-22 21:37:05,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:37:05,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbded0e93e2314724230b HTTP/1.1" 200 None 2026-03-22 21:37:05,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:37:05,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-22 21:37:05,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:37:05,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbded0e93e2314724230b HTTP/1.1" 200 None 2026-03-22 21:37:05,330 - lava_callback - ERROR - Error processing callback for node 69bdbded0e93e2314724230b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40108 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:38:04,128 - lava_callback - INFO - Received callback for node 69bdbe350e93e2314724231d INFO: 172.20.0.1:47860 - "POST /node/69bdbe350e93e2314724231d HTTP/1.0" 202 Accepted 2026-03-22 21:38:04,133 - lava_callback - INFO - Starting async job submit for node 69bdbe350e93e2314724231d 2026-03-22 21:38:04,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:38:04,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbe350e93e2314724231d HTTP/1.1" 200 None 2026-03-22 21:38:04,149 - lava_callback - DEBUG - Processing node 69bdbe350e93e2314724231d: baseline-arm64-baylibre 2026-03-22 21:38:04,149 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbe350e93e2314724231d 2026-03-22 21:38:04,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:38:04,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbe350e93e2314724231d HTTP/1.1" 200 None 2026-03-22 21:38:04,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:38:04,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:38:04,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:38:04,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbe350e93e2314724231d HTTP/1.1" 200 None 2026-03-22 21:38:04,202 - lava_callback - ERROR - Error processing callback for node 69bdbe350e93e2314724231d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59056 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:39:25,322 - lava_callback - INFO - Received callback for node 69bdbe7f0e93e23147242328 INFO: 172.20.0.1:52452 - "POST /node/69bdbe7f0e93e23147242328 HTTP/1.0" 202 Accepted 2026-03-22 21:39:25,327 - lava_callback - INFO - Starting async job submit for node 69bdbe7f0e93e23147242328 2026-03-22 21:39:25,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:39:25,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbe7f0e93e23147242328 HTTP/1.1" 200 None 2026-03-22 21:39:25,343 - lava_callback - DEBUG - Processing node 69bdbe7f0e93e23147242328: baseline-arm-clabbe 2026-03-22 21:39:25,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbe7f0e93e23147242328 2026-03-22 21:39:25,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:39:25,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbe7f0e93e23147242328 HTTP/1.1" 200 None 2026-03-22 21:39:25,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:39:25,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 21:39:25,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:39:25,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbe7f0e93e23147242328 HTTP/1.1" 200 None 2026-03-22 21:39:25,397 - lava_callback - ERROR - Error processing callback for node 69bdbe7f0e93e23147242328: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56830 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:40:25,412 - lava_callback - INFO - Received callback for node 69bdbeb80e93e2314724235e INFO: 172.20.0.1:60166 - "POST /node/69bdbeb80e93e2314724235e HTTP/1.0" 202 Accepted 2026-03-22 21:40:25,417 - lava_callback - INFO - Starting async job submit for node 69bdbeb80e93e2314724235e 2026-03-22 21:40:25,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:40:25,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbeb80e93e2314724235e HTTP/1.1" 200 None 2026-03-22 21:40:25,433 - lava_callback - DEBUG - Processing node 69bdbeb80e93e2314724235e: baseline-arm64-clabbe 2026-03-22 21:40:25,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbeb80e93e2314724235e 2026-03-22 21:40:25,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:40:25,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbeb80e93e2314724235e HTTP/1.1" 200 None 2026-03-22 21:40:25,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:40:25,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:40:25,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:40:25,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbeb80e93e2314724235e HTTP/1.1" 200 None 2026-03-22 21:40:25,517 - lava_callback - ERROR - Error processing callback for node 69bdbeb80e93e2314724235e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36134 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:41:25,440 - lava_callback - INFO - Received callback for node 69bdbef50e93e23147242368 INFO: 172.20.0.1:36622 - "POST /node/69bdbef50e93e23147242368 HTTP/1.0" 202 Accepted 2026-03-22 21:41:25,445 - lava_callback - INFO - Starting async job submit for node 69bdbef50e93e23147242368 2026-03-22 21:41:25,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:41:25,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbef50e93e23147242368 HTTP/1.1" 200 None 2026-03-22 21:41:25,461 - lava_callback - DEBUG - Processing node 69bdbef50e93e23147242368: baseline-arm-clabbe 2026-03-22 21:41:25,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbef50e93e23147242368 2026-03-22 21:41:25,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:41:25,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbef50e93e23147242368 HTTP/1.1" 200 None 2026-03-22 21:41:25,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:41:25,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 21:41:25,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:41:25,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbef50e93e23147242368 HTTP/1.1" 200 None 2026-03-22 21:41:25,520 - lava_callback - ERROR - Error processing callback for node 69bdbef50e93e23147242368: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58574 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:42:25,475 - lava_callback - INFO - Received callback for node 69bdbf2d0e93e2314724237e INFO: 172.20.0.1:37240 - "POST /node/69bdbf2d0e93e2314724237e HTTP/1.0" 202 Accepted 2026-03-22 21:42:25,480 - lava_callback - INFO - Starting async job submit for node 69bdbf2d0e93e2314724237e 2026-03-22 21:42:25,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:42:25,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbf2d0e93e2314724237e HTTP/1.1" 200 None 2026-03-22 21:42:25,497 - lava_callback - DEBUG - Processing node 69bdbf2d0e93e2314724237e: baseline-arm64-clabbe 2026-03-22 21:42:25,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbf2d0e93e2314724237e 2026-03-22 21:42:25,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:42:25,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbf2d0e93e2314724237e HTTP/1.1" 200 None 2026-03-22 21:42:25,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:42:25,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:42:25,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:42:25,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbf2d0e93e2314724237e HTTP/1.1" 200 None 2026-03-22 21:42:25,558 - lava_callback - ERROR - Error processing callback for node 69bdbf2d0e93e2314724237e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48016 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:43:25,476 - lava_callback - INFO - Received callback for node 69bdbf6d0e93e23147242386 INFO: 172.20.0.1:60868 - "POST /node/69bdbf6d0e93e23147242386 HTTP/1.0" 202 Accepted 2026-03-22 21:43:25,482 - lava_callback - INFO - Starting async job submit for node 69bdbf6d0e93e23147242386 2026-03-22 21:43:25,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:43:25,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbf6d0e93e23147242386 HTTP/1.1" 200 None 2026-03-22 21:43:25,497 - lava_callback - DEBUG - Processing node 69bdbf6d0e93e23147242386: baseline-arm-clabbe 2026-03-22 21:43:25,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbf6d0e93e23147242386 2026-03-22 21:43:25,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:43:25,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbf6d0e93e23147242386 HTTP/1.1" 200 None 2026-03-22 21:43:25,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:43:25,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 21:43:25,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:43:25,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbf6d0e93e23147242386 HTTP/1.1" 200 None 2026-03-22 21:43:25,555 - lava_callback - ERROR - Error processing callback for node 69bdbf6d0e93e23147242386: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42216 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:44:05,456 - lava_callback - INFO - Received callback for node 69bdbf970e93e23147242395 INFO: 172.20.0.1:49734 - "POST /node/69bdbf970e93e23147242395 HTTP/1.0" 202 Accepted 2026-03-22 21:44:05,461 - lava_callback - INFO - Starting async job submit for node 69bdbf970e93e23147242395 2026-03-22 21:44:05,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:44:05,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbf970e93e23147242395 HTTP/1.1" 200 None 2026-03-22 21:44:05,511 - lava_callback - DEBUG - Processing node 69bdbf970e93e23147242395: baseline-arm-clabbe 2026-03-22 21:44:05,511 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdbf970e93e23147242395 2026-03-22 21:44:05,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:44:05,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdbf970e93e23147242395 HTTP/1.1" 200 None 2026-03-22 21:44:05,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:44:05,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-22 21:44:05,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:44:05,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdbf970e93e23147242395 HTTP/1.1" 200 None 2026-03-22 21:44:05,607 - lava_callback - ERROR - Error processing callback for node 69bdbf970e93e23147242395: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49036 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:46:05,575 - lava_callback - INFO - Received callback for node 69bdc0110e93e231472423b0 INFO: 172.20.0.1:51130 - "POST /node/69bdc0110e93e231472423b0 HTTP/1.0" 202 Accepted 2026-03-22 21:46:05,581 - lava_callback - INFO - Starting async job submit for node 69bdc0110e93e231472423b0 2026-03-22 21:46:05,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:46:05,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc0110e93e231472423b0 HTTP/1.1" 200 None 2026-03-22 21:46:05,597 - lava_callback - DEBUG - Processing node 69bdc0110e93e231472423b0: baseline-arm64-clabbe 2026-03-22 21:46:05,597 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc0110e93e231472423b0 2026-03-22 21:46:05,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:46:05,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc0110e93e231472423b0 HTTP/1.1" 200 None 2026-03-22 21:46:05,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:46:05,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:46:05,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:46:05,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc0110e93e231472423b0 HTTP/1.1" 200 None 2026-03-22 21:46:05,658 - lava_callback - ERROR - Error processing callback for node 69bdc0110e93e231472423b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:47:25,651 - lava_callback - INFO - Received callback for node 69bdc0630e93e231472423b8 INFO: 172.20.0.1:55206 - "POST /node/69bdc0630e93e231472423b8 HTTP/1.0" 202 Accepted 2026-03-22 21:47:25,656 - lava_callback - INFO - Starting async job submit for node 69bdc0630e93e231472423b8 2026-03-22 21:47:25,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:47:25,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc0630e93e231472423b8 HTTP/1.1" 200 None 2026-03-22 21:47:25,673 - lava_callback - DEBUG - Processing node 69bdc0630e93e231472423b8: baseline-arm64-clabbe 2026-03-22 21:47:25,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc0630e93e231472423b8 2026-03-22 21:47:25,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:47:25,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc0630e93e231472423b8 HTTP/1.1" 200 None 2026-03-22 21:47:25,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:47:25,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:47:25,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:47:25,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc0630e93e231472423b8 HTTP/1.1" 200 None 2026-03-22 21:47:25,741 - lava_callback - ERROR - Error processing callback for node 69bdc0630e93e231472423b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57134 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:48:44,670 - lava_callback - INFO - Received callback for node 69bdc0ab0e93e231472423c4 INFO: 172.20.0.1:37886 - "POST /node/69bdc0ab0e93e231472423c4 HTTP/1.0" 202 Accepted 2026-03-22 21:48:44,676 - lava_callback - INFO - Starting async job submit for node 69bdc0ab0e93e231472423c4 2026-03-22 21:48:44,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:48:44,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc0ab0e93e231472423c4 HTTP/1.1" 200 None 2026-03-22 21:48:44,692 - lava_callback - DEBUG - Processing node 69bdc0ab0e93e231472423c4: baseline-arm64-baylibre 2026-03-22 21:48:44,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc0ab0e93e231472423c4 2026-03-22 21:48:44,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:48:44,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc0ab0e93e231472423c4 HTTP/1.1" 200 None 2026-03-22 21:48:44,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:37902 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:48:44,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:48:44,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:48:44,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc0ab0e93e231472423c4 HTTP/1.1" 200 None 2026-03-22 21:48:44,758 - lava_callback - ERROR - Error processing callback for node 69bdc0ab0e93e231472423c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55726 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:50:04,713 - lava_callback - INFO - Received callback for node 69bdc0ff0e93e231472423cc INFO: 172.20.0.1:51506 - "POST /node/69bdc0ff0e93e231472423cc HTTP/1.0" 202 Accepted 2026-03-22 21:50:04,718 - lava_callback - INFO - Starting async job submit for node 69bdc0ff0e93e231472423cc 2026-03-22 21:50:04,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:50:04,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc0ff0e93e231472423cc HTTP/1.1" 200 None 2026-03-22 21:50:04,742 - lava_callback - DEBUG - Processing node 69bdc0ff0e93e231472423cc: baseline-arm64-baylibre 2026-03-22 21:50:04,743 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc0ff0e93e231472423cc 2026-03-22 21:50:04,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:50:04,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc0ff0e93e231472423cc HTTP/1.1" 200 None 2026-03-22 21:50:04,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:50:04,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:50:04,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:50:04,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc0ff0e93e231472423cc HTTP/1.1" 200 None 2026-03-22 21:50:04,816 - lava_callback - ERROR - Error processing callback for node 69bdc0ff0e93e231472423cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:51:45,845 - lava_callback - INFO - Received callback for node 69bdc15b0e93e231472423db INFO: 172.20.0.1:33172 - "POST /node/69bdc15b0e93e231472423db HTTP/1.0" 202 Accepted 2026-03-22 21:51:45,850 - lava_callback - INFO - Starting async job submit for node 69bdc15b0e93e231472423db 2026-03-22 21:51:45,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:51:45,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc15b0e93e231472423db HTTP/1.1" 200 None 2026-03-22 21:51:45,867 - lava_callback - DEBUG - Processing node 69bdc15b0e93e231472423db: baseline-arm64-clabbe 2026-03-22 21:51:45,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc15b0e93e231472423db 2026-03-22 21:51:45,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:51:45,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc15b0e93e231472423db HTTP/1.1" 200 None 2026-03-22 21:51:45,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:51:45,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:51:45,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:51:45,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc15b0e93e231472423db HTTP/1.1" 200 None 2026-03-22 21:51:45,959 - lava_callback - ERROR - Error processing callback for node 69bdc15b0e93e231472423db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56760 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:53:05,944 - lava_callback - INFO - Received callback for node 69bdc1b00e93e231472423ec INFO: 172.20.0.1:60068 - "POST /node/69bdc1b00e93e231472423ec HTTP/1.0" 202 Accepted 2026-03-22 21:53:05,950 - lava_callback - INFO - Starting async job submit for node 69bdc1b00e93e231472423ec 2026-03-22 21:53:05,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:53:05,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc1b00e93e231472423ec HTTP/1.1" 200 None 2026-03-22 21:53:05,975 - lava_callback - DEBUG - Processing node 69bdc1b00e93e231472423ec: baseline-arm64-clabbe 2026-03-22 21:53:05,975 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc1b00e93e231472423ec 2026-03-22 21:53:05,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:53:05,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc1b00e93e231472423ec HTTP/1.1" 200 None 2026-03-22 21:53:05,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:53:06,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:53:06,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:53:06,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc1b00e93e231472423ec HTTP/1.1" 200 None 2026-03-22 21:53:06,080 - lava_callback - ERROR - Error processing callback for node 69bdc1b00e93e231472423ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39932 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:54:04,989 - lava_callback - INFO - Received callback for node 69bdc1f80e93e231472423fd INFO: 172.20.0.1:33098 - "POST /node/69bdc1f80e93e231472423fd HTTP/1.0" 202 Accepted 2026-03-22 21:54:04,995 - lava_callback - INFO - Starting async job submit for node 69bdc1f80e93e231472423fd 2026-03-22 21:54:04,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:54:05,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc1f80e93e231472423fd HTTP/1.1" 200 None 2026-03-22 21:54:05,025 - lava_callback - DEBUG - Processing node 69bdc1f80e93e231472423fd: baseline-arm64-baylibre 2026-03-22 21:54:05,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc1f80e93e231472423fd 2026-03-22 21:54:05,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:54:05,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc1f80e93e231472423fd HTTP/1.1" 200 None 2026-03-22 21:54:05,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:54:05,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:54:05,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:54:05,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc1f80e93e231472423fd HTTP/1.1" 200 None 2026-03-22 21:54:05,127 - lava_callback - ERROR - Error processing callback for node 69bdc1f80e93e231472423fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39778 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:55:46,052 - lava_callback - INFO - Received callback for node 69bdc2520e93e23147242410 INFO: 172.20.0.1:39786 - "POST /node/69bdc2520e93e23147242410 HTTP/1.0" 202 Accepted 2026-03-22 21:55:46,060 - lava_callback - INFO - Starting async job submit for node 69bdc2520e93e23147242410 2026-03-22 21:55:46,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:55:46,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc2520e93e23147242410 HTTP/1.1" 200 None 2026-03-22 21:55:46,081 - lava_callback - DEBUG - Processing node 69bdc2520e93e23147242410: baseline-arm64-clabbe 2026-03-22 21:55:46,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc2520e93e23147242410 2026-03-22 21:55:46,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:55:46,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc2520e93e23147242410 HTTP/1.1" 200 None 2026-03-22 21:55:46,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:55:46,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:55:46,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:55:46,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc2520e93e23147242410 HTTP/1.1" 200 None 2026-03-22 21:55:46,251 - lava_callback - ERROR - Error processing callback for node 69bdc2520e93e23147242410: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32926 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:57:05,138 - lava_callback - INFO - Received callback for node 69bdc29d0e93e23147242415 INFO: 172.20.0.1:42538 - "POST /node/69bdc29d0e93e23147242415 HTTP/1.0" 202 Accepted 2026-03-22 21:57:05,144 - lava_callback - INFO - Starting async job submit for node 69bdc29d0e93e23147242415 2026-03-22 21:57:05,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:57:05,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc29d0e93e23147242415 HTTP/1.1" 200 None 2026-03-22 21:57:05,160 - lava_callback - DEBUG - Processing node 69bdc29d0e93e23147242415: baseline-arm64-baylibre 2026-03-22 21:57:05,160 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc29d0e93e23147242415 2026-03-22 21:57:05,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:57:05,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc29d0e93e23147242415 HTTP/1.1" 200 None 2026-03-22 21:57:05,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:57:05,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:57:05,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:57:05,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc29d0e93e23147242415 HTTP/1.1" 200 None 2026-03-22 21:57:05,219 - lava_callback - ERROR - Error processing callback for node 69bdc29d0e93e23147242415: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56304 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:58:25,350 - lava_callback - INFO - Received callback for node 69bdc2ec0e93e23147242426 INFO: 172.20.0.1:46586 - "POST /node/69bdc2ec0e93e23147242426 HTTP/1.0" 202 Accepted 2026-03-22 21:58:25,356 - lava_callback - INFO - Starting async job submit for node 69bdc2ec0e93e23147242426 2026-03-22 21:58:25,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:58:25,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc2ec0e93e23147242426 HTTP/1.1" 200 None 2026-03-22 21:58:25,372 - lava_callback - DEBUG - Processing node 69bdc2ec0e93e23147242426: baseline-arm64-baylibre 2026-03-22 21:58:25,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc2ec0e93e23147242426 2026-03-22 21:58:25,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:58:25,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc2ec0e93e23147242426 HTTP/1.1" 200 None 2026-03-22 21:58:25,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:58:25,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:58:25,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:58:25,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc2ec0e93e23147242426 HTTP/1.1" 200 None 2026-03-22 21:58:25,427 - lava_callback - ERROR - Error processing callback for node 69bdc2ec0e93e23147242426: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53376 - "GET / HTTP/1.0" 200 OK 2026-03-22 21:59:46,219 - lava_callback - INFO - Received callback for node 69bdc3430e93e2314724243b INFO: 172.20.0.1:53378 - "POST /node/69bdc3430e93e2314724243b HTTP/1.0" 202 Accepted 2026-03-22 21:59:46,225 - lava_callback - INFO - Starting async job submit for node 69bdc3430e93e2314724243b 2026-03-22 21:59:46,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:59:46,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc3430e93e2314724243b HTTP/1.1" 200 None 2026-03-22 21:59:46,242 - lava_callback - DEBUG - Processing node 69bdc3430e93e2314724243b: baseline-arm64-clabbe 2026-03-22 21:59:46,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc3430e93e2314724243b 2026-03-22 21:59:46,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:59:46,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc3430e93e2314724243b HTTP/1.1" 200 None 2026-03-22 21:59:46,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:59:46,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 21:59:46,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 21:59:46,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc3430e93e2314724243b HTTP/1.1" 200 None 2026-03-22 21:59:46,305 - lava_callback - ERROR - Error processing callback for node 69bdc3430e93e2314724243b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52326 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:01:05,351 - lava_callback - INFO - Received callback for node 69bdc38e0e93e23147242443 INFO: 172.20.0.1:44942 - "POST /node/69bdc38e0e93e23147242443 HTTP/1.0" 202 Accepted 2026-03-22 22:01:05,356 - lava_callback - INFO - Starting async job submit for node 69bdc38e0e93e23147242443 2026-03-22 22:01:05,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:01:05,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc38e0e93e23147242443 HTTP/1.1" 200 None 2026-03-22 22:01:05,397 - lava_callback - DEBUG - Processing node 69bdc38e0e93e23147242443: baseline-arm64-baylibre 2026-03-22 22:01:05,397 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc38e0e93e23147242443 2026-03-22 22:01:05,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:01:05,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc38e0e93e23147242443 HTTP/1.1" 200 None 2026-03-22 22:01:05,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:01:05,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 22:01:05,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:01:05,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc38e0e93e23147242443 HTTP/1.1" 200 None 2026-03-22 22:01:05,453 - lava_callback - ERROR - Error processing callback for node 69bdc38e0e93e23147242443: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60336 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:02:26,339 - lava_callback - INFO - Received callback for node 69bdc3e60e93e2314724245e INFO: 172.20.0.1:49910 - "POST /node/69bdc3e60e93e2314724245e HTTP/1.0" 202 Accepted 2026-03-22 22:02:26,345 - lava_callback - INFO - Starting async job submit for node 69bdc3e60e93e2314724245e 2026-03-22 22:02:26,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:02:26,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc3e60e93e2314724245e HTTP/1.1" 200 None 2026-03-22 22:02:26,361 - lava_callback - DEBUG - Processing node 69bdc3e60e93e2314724245e: baseline-arm64-clabbe 2026-03-22 22:02:26,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc3e60e93e2314724245e 2026-03-22 22:02:26,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:02:26,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc3e60e93e2314724245e HTTP/1.1" 200 None 2026-03-22 22:02:26,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:02:26,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 22:02:26,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:02:26,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc3e60e93e2314724245e HTTP/1.1" 200 None 2026-03-22 22:02:26,467 - lava_callback - ERROR - Error processing callback for node 69bdc3e60e93e2314724245e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55440 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:03:46,376 - lava_callback - INFO - Received callback for node 69bdc4370e93e2314724247b INFO: 172.20.0.1:55446 - "POST /node/69bdc4370e93e2314724247b HTTP/1.0" 202 Accepted 2026-03-22 22:03:46,382 - lava_callback - INFO - Starting async job submit for node 69bdc4370e93e2314724247b 2026-03-22 22:03:46,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:03:46,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc4370e93e2314724247b HTTP/1.1" 200 None 2026-03-22 22:03:46,407 - lava_callback - DEBUG - Processing node 69bdc4370e93e2314724247b: baseline-arm64-clabbe 2026-03-22 22:03:46,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc4370e93e2314724247b 2026-03-22 22:03:46,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:03:46,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc4370e93e2314724247b HTTP/1.1" 200 None 2026-03-22 22:03:46,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:03:46,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 22:03:46,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:03:46,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc4370e93e2314724247b HTTP/1.1" 200 None 2026-03-22 22:03:46,520 - lava_callback - ERROR - Error processing callback for node 69bdc4370e93e2314724247b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54028 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:05:06,483 - lava_callback - INFO - Received callback for node 69bdc48a0e93e2314724248e INFO: 172.20.0.1:37040 - "POST /node/69bdc48a0e93e2314724248e HTTP/1.0" 202 Accepted 2026-03-22 22:05:06,489 - lava_callback - INFO - Starting async job submit for node 69bdc48a0e93e2314724248e 2026-03-22 22:05:06,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:05:06,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc48a0e93e2314724248e HTTP/1.1" 200 None 2026-03-22 22:05:06,508 - lava_callback - DEBUG - Processing node 69bdc48a0e93e2314724248e: baseline-arm64-clabbe 2026-03-22 22:05:06,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc48a0e93e2314724248e 2026-03-22 22:05:06,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:37056 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:05:06,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc48a0e93e2314724248e HTTP/1.1" 200 None 2026-03-22 22:05:06,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:05:06,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 22:05:06,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:05:06,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc48a0e93e2314724248e HTTP/1.1" 200 None 2026-03-22 22:05:06,589 - lava_callback - ERROR - Error processing callback for node 69bdc48a0e93e2314724248e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44962 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:06:26,438 - lava_callback - INFO - Received callback for node 69bdc4dc0e93e23147242495 INFO: 172.20.0.1:45518 - "POST /node/69bdc4dc0e93e23147242495 HTTP/1.0" 202 Accepted 2026-03-22 22:06:26,444 - lava_callback - INFO - Starting async job submit for node 69bdc4dc0e93e23147242495 2026-03-22 22:06:26,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:06:26,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc4dc0e93e23147242495 HTTP/1.1" 200 None 2026-03-22 22:06:26,499 - lava_callback - DEBUG - Processing node 69bdc4dc0e93e23147242495: baseline-arm64-clabbe 2026-03-22 22:06:26,499 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc4dc0e93e23147242495 2026-03-22 22:06:26,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:45520 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:06:26,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc4dc0e93e23147242495 HTTP/1.1" 200 None 2026-03-22 22:06:26,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:06:26,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 22:06:26,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:06:26,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc4dc0e93e23147242495 HTTP/1.1" 200 None 2026-03-22 22:06:26,591 - lava_callback - ERROR - Error processing callback for node 69bdc4dc0e93e23147242495: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58504 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:07:46,477 - lava_callback - INFO - Received callback for node 69bdc52e0e93e231472424a5 INFO: 172.20.0.1:58506 - "POST /node/69bdc52e0e93e231472424a5 HTTP/1.0" 202 Accepted 2026-03-22 22:07:46,483 - lava_callback - INFO - Starting async job submit for node 69bdc52e0e93e231472424a5 2026-03-22 22:07:46,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:58516 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:07:46,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc52e0e93e231472424a5 HTTP/1.1" 200 None 2026-03-22 22:07:46,518 - lava_callback - DEBUG - Processing node 69bdc52e0e93e231472424a5: baseline-arm64-clabbe 2026-03-22 22:07:46,519 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc52e0e93e231472424a5 2026-03-22 22:07:46,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:07:46,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc52e0e93e231472424a5 HTTP/1.1" 200 None 2026-03-22 22:07:46,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:07:46,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 22:07:46,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:07:46,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc52e0e93e231472424a5 HTTP/1.1" 200 None 2026-03-22 22:07:46,689 - lava_callback - ERROR - Error processing callback for node 69bdc52e0e93e231472424a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41688 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:09:26,590 - lava_callback - INFO - Received callback for node 69bdc5800e93e231472424b3 INFO: 172.20.0.1:41704 - "POST /node/69bdc5800e93e231472424b3 HTTP/1.0" 202 Accepted 2026-03-22 22:09:26,595 - lava_callback - INFO - Starting async job submit for node 69bdc5800e93e231472424b3 2026-03-22 22:09:26,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:09:26,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc5800e93e231472424b3 HTTP/1.1" 200 None 2026-03-22 22:09:26,629 - lava_callback - DEBUG - Processing node 69bdc5800e93e231472424b3: baseline-arm64-clabbe 2026-03-22 22:09:26,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc5800e93e231472424b3 2026-03-22 22:09:26,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:09:26,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc5800e93e231472424b3 HTTP/1.1" 200 None 2026-03-22 22:09:26,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:09:26,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 22:09:26,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:09:26,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc5800e93e231472424b3 HTTP/1.1" 200 None 2026-03-22 22:09:26,704 - lava_callback - ERROR - Error processing callback for node 69bdc5800e93e231472424b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59744 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:10:46,637 - lava_callback - INFO - Received callback for node 69bdc5d20e93e231472424bd INFO: 172.20.0.1:59752 - "POST /node/69bdc5d20e93e231472424bd HTTP/1.0" 202 Accepted 2026-03-22 22:10:46,643 - lava_callback - INFO - Starting async job submit for node 69bdc5d20e93e231472424bd 2026-03-22 22:10:46,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:10:46,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc5d20e93e231472424bd HTTP/1.1" 200 None 2026-03-22 22:10:46,672 - lava_callback - DEBUG - Processing node 69bdc5d20e93e231472424bd: baseline-arm64-clabbe 2026-03-22 22:10:46,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc5d20e93e231472424bd 2026-03-22 22:10:46,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:10:46,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc5d20e93e231472424bd HTTP/1.1" 200 None 2026-03-22 22:10:46,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:10:46,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 22:10:46,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:10:46,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc5d20e93e231472424bd HTTP/1.1" 200 None 2026-03-22 22:10:46,739 - lava_callback - ERROR - Error processing callback for node 69bdc5d20e93e231472424bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35874 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:12:06,700 - lava_callback - INFO - Received callback for node 69bdc6220e93e231472424d0 INFO: 172.20.0.1:35884 - "POST /node/69bdc6220e93e231472424d0 HTTP/1.0" 202 Accepted 2026-03-22 22:12:06,706 - lava_callback - INFO - Starting async job submit for node 69bdc6220e93e231472424d0 2026-03-22 22:12:06,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:12:06,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc6220e93e231472424d0 HTTP/1.1" 200 None 2026-03-22 22:12:06,726 - lava_callback - DEBUG - Processing node 69bdc6220e93e231472424d0: baseline-arm64-clabbe 2026-03-22 22:12:06,726 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc6220e93e231472424d0 2026-03-22 22:12:06,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:12:06,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc6220e93e231472424d0 HTTP/1.1" 200 None 2026-03-22 22:12:06,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:12:06,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-22 22:12:06,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:12:06,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc6220e93e231472424d0 HTTP/1.1" 200 None 2026-03-22 22:12:06,806 - lava_callback - ERROR - Error processing callback for node 69bdc6220e93e231472424d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38770 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:13:06,074 - lava_callback - INFO - Received callback for node 69bdc66b0e93e231472424e7 INFO: 172.20.0.1:45878 - "POST /node/69bdc66b0e93e231472424e7 HTTP/1.0" 202 Accepted 2026-03-22 22:13:06,079 - lava_callback - INFO - Starting async job submit for node 69bdc66b0e93e231472424e7 2026-03-22 22:13:06,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:13:06,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc66b0e93e231472424e7 HTTP/1.1" 200 None 2026-03-22 22:13:06,109 - lava_callback - DEBUG - Processing node 69bdc66b0e93e231472424e7: baseline-arm64-baylibre 2026-03-22 22:13:06,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc66b0e93e231472424e7 2026-03-22 22:13:06,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:13:06,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc66b0e93e231472424e7 HTTP/1.1" 200 None 2026-03-22 22:13:06,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:13:06,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:13:06,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:13:06,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc66b0e93e231472424e7 HTTP/1.1" 200 None 2026-03-22 22:13:06,213 - lava_callback - ERROR - Error processing callback for node 69bdc66b0e93e231472424e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57464 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:15:46,118 - lava_callback - INFO - Received callback for node 69bdc70a0e93e2314724250a INFO: 172.20.0.1:57478 - "POST /node/69bdc70a0e93e2314724250a HTTP/1.0" 202 Accepted 2026-03-22 22:15:46,123 - lava_callback - INFO - Starting async job submit for node 69bdc70a0e93e2314724250a 2026-03-22 22:15:46,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:15:46,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc70a0e93e2314724250a HTTP/1.1" 200 None 2026-03-22 22:15:46,139 - lava_callback - DEBUG - Processing node 69bdc70a0e93e2314724250a: baseline-arm64-baylibre 2026-03-22 22:15:46,139 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc70a0e93e2314724250a 2026-03-22 22:15:46,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:15:46,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc70a0e93e2314724250a HTTP/1.1" 200 None 2026-03-22 22:15:46,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:15:46,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:15:46,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:15:46,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc70a0e93e2314724250a HTTP/1.1" 200 None 2026-03-22 22:15:46,191 - lava_callback - ERROR - Error processing callback for node 69bdc70a0e93e2314724250a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53566 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:17:06,603 - lava_callback - INFO - Received callback for node 69bdc7590e93e23147242523 INFO: 172.20.0.1:53582 - "POST /node/69bdc7590e93e23147242523 HTTP/1.0" 202 Accepted 2026-03-22 22:17:06,608 - lava_callback - INFO - Starting async job submit for node 69bdc7590e93e23147242523 2026-03-22 22:17:06,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:17:06,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc7590e93e23147242523 HTTP/1.1" 200 None 2026-03-22 22:17:06,653 - lava_callback - DEBUG - Processing node 69bdc7590e93e23147242523: baseline-arm64-baylibre 2026-03-22 22:17:06,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc7590e93e23147242523 2026-03-22 22:17:06,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:17:06,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc7590e93e23147242523 HTTP/1.1" 200 None 2026-03-22 22:17:06,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:17:06,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:17:06,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:17:06,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc7590e93e23147242523 HTTP/1.1" 200 None 2026-03-22 22:17:06,709 - lava_callback - ERROR - Error processing callback for node 69bdc7590e93e23147242523: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59824 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:18:26,619 - lava_callback - INFO - Received callback for node 69bdc7ac0e93e23147242537 INFO: 172.20.0.1:59838 - "POST /node/69bdc7ac0e93e23147242537 HTTP/1.0" 202 Accepted 2026-03-22 22:18:26,625 - lava_callback - INFO - Starting async job submit for node 69bdc7ac0e93e23147242537 2026-03-22 22:18:26,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:18:26,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc7ac0e93e23147242537 HTTP/1.1" 200 None 2026-03-22 22:18:26,662 - lava_callback - DEBUG - Processing node 69bdc7ac0e93e23147242537: baseline-arm64-baylibre 2026-03-22 22:18:26,663 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc7ac0e93e23147242537 2026-03-22 22:18:26,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:18:26,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc7ac0e93e23147242537 HTTP/1.1" 200 None 2026-03-22 22:18:26,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:18:26,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:18:26,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:18:26,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc7ac0e93e23147242537 HTTP/1.1" 200 None 2026-03-22 22:18:26,734 - lava_callback - ERROR - Error processing callback for node 69bdc7ac0e93e23147242537: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54116 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:19:46,311 - lava_callback - INFO - Received callback for node 69bdc7ff0e93e2314724255a INFO: 172.20.0.1:54120 - "POST /node/69bdc7ff0e93e2314724255a HTTP/1.0" 202 Accepted 2026-03-22 22:19:46,317 - lava_callback - INFO - Starting async job submit for node 69bdc7ff0e93e2314724255a 2026-03-22 22:19:46,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:19:46,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc7ff0e93e2314724255a HTTP/1.1" 200 None 2026-03-22 22:19:46,342 - lava_callback - DEBUG - Processing node 69bdc7ff0e93e2314724255a: baseline-arm64-baylibre 2026-03-22 22:19:46,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc7ff0e93e2314724255a 2026-03-22 22:19:46,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:19:46,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc7ff0e93e2314724255a HTTP/1.1" 200 None 2026-03-22 22:19:46,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:19:46,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:19:46,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:19:46,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc7ff0e93e2314724255a HTTP/1.1" 200 None 2026-03-22 22:19:46,426 - lava_callback - ERROR - Error processing callback for node 69bdc7ff0e93e2314724255a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49370 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:21:26,382 - lava_callback - INFO - Received callback for node 69bdc8510e93e23147242575 INFO: 172.20.0.1:60410 - "POST /node/69bdc8510e93e23147242575 HTTP/1.0" 202 Accepted 2026-03-22 22:21:26,387 - lava_callback - INFO - Starting async job submit for node 69bdc8510e93e23147242575 2026-03-22 22:21:26,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:21:26,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc8510e93e23147242575 HTTP/1.1" 200 None 2026-03-22 22:21:26,404 - lava_callback - DEBUG - Processing node 69bdc8510e93e23147242575: baseline-arm64-baylibre 2026-03-22 22:21:26,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc8510e93e23147242575 2026-03-22 22:21:26,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:21:26,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc8510e93e23147242575 HTTP/1.1" 200 None 2026-03-22 22:21:26,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:21:26,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:21:26,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:21:26,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc8510e93e23147242575 HTTP/1.1" 200 None 2026-03-22 22:21:26,458 - lava_callback - ERROR - Error processing callback for node 69bdc8510e93e23147242575: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33786 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:22:47,187 - lava_callback - INFO - Received callback for node 69bdc8a90e93e23147242597 INFO: 172.20.0.1:33800 - "POST /node/69bdc8a90e93e23147242597 HTTP/1.0" 202 Accepted 2026-03-22 22:22:47,192 - lava_callback - INFO - Starting async job submit for node 69bdc8a90e93e23147242597 2026-03-22 22:22:47,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:22:47,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc8a90e93e23147242597 HTTP/1.1" 200 None 2026-03-22 22:22:47,249 - lava_callback - DEBUG - Processing node 69bdc8a90e93e23147242597: baseline-arm64-clabbe 2026-03-22 22:22:47,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc8a90e93e23147242597 2026-03-22 22:22:47,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:22:47,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc8a90e93e23147242597 HTTP/1.1" 200 None 2026-03-22 22:22:47,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:22:47,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:22:47,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:22:47,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc8a90e93e23147242597 HTTP/1.1" 200 None 2026-03-22 22:22:47,333 - lava_callback - ERROR - Error processing callback for node 69bdc8a90e93e23147242597: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41708 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:24:06,598 - lava_callback - INFO - Received callback for node 69bdc8f30e93e231472425a5 INFO: 172.20.0.1:41718 - "POST /node/69bdc8f30e93e231472425a5 HTTP/1.0" 202 Accepted 2026-03-22 22:24:06,604 - lava_callback - INFO - Starting async job submit for node 69bdc8f30e93e231472425a5 2026-03-22 22:24:06,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:24:06,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc8f30e93e231472425a5 HTTP/1.1" 200 None 2026-03-22 22:24:06,629 - lava_callback - DEBUG - Processing node 69bdc8f30e93e231472425a5: baseline-arm64-baylibre 2026-03-22 22:24:06,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc8f30e93e231472425a5 2026-03-22 22:24:06,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:24:06,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc8f30e93e231472425a5 HTTP/1.1" 200 None 2026-03-22 22:24:06,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:24:06,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:24:06,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:24:06,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc8f30e93e231472425a5 HTTP/1.1" 200 None 2026-03-22 22:24:06,686 - lava_callback - ERROR - Error processing callback for node 69bdc8f30e93e231472425a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:25:27,343 - lava_callback - INFO - Received callback for node 69bdc94c0e93e231472425bc INFO: 172.20.0.1:43872 - "POST /node/69bdc94c0e93e231472425bc HTTP/1.0" 202 Accepted 2026-03-22 22:25:27,349 - lava_callback - INFO - Starting async job submit for node 69bdc94c0e93e231472425bc 2026-03-22 22:25:27,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:25:27,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc94c0e93e231472425bc HTTP/1.1" 200 None 2026-03-22 22:25:27,374 - lava_callback - DEBUG - Processing node 69bdc94c0e93e231472425bc: baseline-arm64-clabbe 2026-03-22 22:25:27,374 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc94c0e93e231472425bc 2026-03-22 22:25:27,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:25:27,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc94c0e93e231472425bc HTTP/1.1" 200 None 2026-03-22 22:25:27,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:25:27,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:25:27,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:25:27,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc94c0e93e231472425bc HTTP/1.1" 200 None 2026-03-22 22:25:27,436 - lava_callback - ERROR - Error processing callback for node 69bdc94c0e93e231472425bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46530 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:28:07,397 - lava_callback - INFO - Received callback for node 69bdc9f30e93e231472425dc INFO: 172.20.0.1:46532 - "POST /node/69bdc9f30e93e231472425dc HTTP/1.0" 202 Accepted 2026-03-22 22:28:07,402 - lava_callback - INFO - Starting async job submit for node 69bdc9f30e93e231472425dc 2026-03-22 22:28:07,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:28:07,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc9f30e93e231472425dc HTTP/1.1" 200 None 2026-03-22 22:28:07,447 - lava_callback - DEBUG - Processing node 69bdc9f30e93e231472425dc: baseline-arm64-clabbe 2026-03-22 22:28:07,447 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdc9f30e93e231472425dc 2026-03-22 22:28:07,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:28:07,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdc9f30e93e231472425dc HTTP/1.1" 200 None 2026-03-22 22:28:07,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:28:07,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:28:07,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:28:07,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdc9f30e93e231472425dc HTTP/1.1" 200 None 2026-03-22 22:28:07,548 - lava_callback - ERROR - Error processing callback for node 69bdc9f30e93e231472425dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58610 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:29:47,438 - lava_callback - INFO - Received callback for node 69bdca480e93e231472425f4 INFO: 172.20.0.1:58622 - "POST /node/69bdca480e93e231472425f4 HTTP/1.0" 202 Accepted 2026-03-22 22:29:47,444 - lava_callback - INFO - Starting async job submit for node 69bdca480e93e231472425f4 2026-03-22 22:29:47,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:29:47,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdca480e93e231472425f4 HTTP/1.1" 200 None 2026-03-22 22:29:47,468 - lava_callback - DEBUG - Processing node 69bdca480e93e231472425f4: baseline-arm64-clabbe 2026-03-22 22:29:47,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdca480e93e231472425f4 2026-03-22 22:29:47,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:29:47,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdca480e93e231472425f4 HTTP/1.1" 200 None 2026-03-22 22:29:47,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:29:47,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:29:47,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:29:47,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdca480e93e231472425f4 HTTP/1.1" 200 None 2026-03-22 22:29:47,535 - lava_callback - ERROR - Error processing callback for node 69bdca480e93e231472425f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57800 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:31:07,496 - lava_callback - INFO - Received callback for node 69bdca9b0e93e231472425fc INFO: 172.20.0.1:57814 - "POST /node/69bdca9b0e93e231472425fc HTTP/1.0" 202 Accepted 2026-03-22 22:31:07,502 - lava_callback - INFO - Starting async job submit for node 69bdca9b0e93e231472425fc 2026-03-22 22:31:07,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:31:07,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdca9b0e93e231472425fc HTTP/1.1" 200 None 2026-03-22 22:31:07,538 - lava_callback - DEBUG - Processing node 69bdca9b0e93e231472425fc: baseline-arm64-clabbe 2026-03-22 22:31:07,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdca9b0e93e231472425fc 2026-03-22 22:31:07,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:31:07,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdca9b0e93e231472425fc HTTP/1.1" 200 None 2026-03-22 22:31:07,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:31:07,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:31:07,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:31:07,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdca9b0e93e231472425fc HTTP/1.1" 200 None 2026-03-22 22:31:07,613 - lava_callback - ERROR - Error processing callback for node 69bdca9b0e93e231472425fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34366 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:32:27,601 - lava_callback - INFO - Received callback for node 69bdcaef0e93e23147242610 INFO: 172.20.0.1:34370 - "POST /node/69bdcaef0e93e23147242610 HTTP/1.0" 202 Accepted 2026-03-22 22:32:27,607 - lava_callback - INFO - Starting async job submit for node 69bdcaef0e93e23147242610 2026-03-22 22:32:27,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:32:27,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcaef0e93e23147242610 HTTP/1.1" 200 None 2026-03-22 22:32:27,623 - lava_callback - DEBUG - Processing node 69bdcaef0e93e23147242610: baseline-arm64-clabbe 2026-03-22 22:32:27,623 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcaef0e93e23147242610 2026-03-22 22:32:27,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:32:27,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcaef0e93e23147242610 HTTP/1.1" 200 None 2026-03-22 22:32:27,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:32:27,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:32:27,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:32:27,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcaef0e93e23147242610 HTTP/1.1" 200 None 2026-03-22 22:32:27,679 - lava_callback - ERROR - Error processing callback for node 69bdcaef0e93e23147242610: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42456 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:33:47,640 - lava_callback - INFO - Received callback for node 69bdcb430e93e23147242623 INFO: 172.20.0.1:42464 - "POST /node/69bdcb430e93e23147242623 HTTP/1.0" 202 Accepted 2026-03-22 22:33:47,646 - lava_callback - INFO - Starting async job submit for node 69bdcb430e93e23147242623 2026-03-22 22:33:47,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:33:47,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcb430e93e23147242623 HTTP/1.1" 200 None 2026-03-22 22:33:47,662 - lava_callback - DEBUG - Processing node 69bdcb430e93e23147242623: baseline-arm64-clabbe 2026-03-22 22:33:47,662 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcb430e93e23147242623 2026-03-22 22:33:47,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:33:47,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcb430e93e23147242623 HTTP/1.1" 200 None 2026-03-22 22:33:47,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:33:47,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:33:47,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:33:47,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcb430e93e23147242623 HTTP/1.1" 200 None 2026-03-22 22:33:47,716 - lava_callback - ERROR - Error processing callback for node 69bdcb430e93e23147242623: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52606 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:35:07,741 - lava_callback - INFO - Received callback for node 69bdcb970e93e23147242637 INFO: 172.20.0.1:52614 - "POST /node/69bdcb970e93e23147242637 HTTP/1.0" 202 Accepted 2026-03-22 22:35:07,747 - lava_callback - INFO - Starting async job submit for node 69bdcb970e93e23147242637 2026-03-22 22:35:07,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:35:07,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcb970e93e23147242637 HTTP/1.1" 200 None 2026-03-22 22:35:07,771 - lava_callback - DEBUG - Processing node 69bdcb970e93e23147242637: baseline-arm64-clabbe 2026-03-22 22:35:07,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcb970e93e23147242637 2026-03-22 22:35:07,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:35:07,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcb970e93e23147242637 HTTP/1.1" 200 None 2026-03-22 22:35:07,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:35:07,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:35:07,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:35:07,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcb970e93e23147242637 HTTP/1.1" 200 None 2026-03-22 22:35:07,868 - lava_callback - ERROR - Error processing callback for node 69bdcb970e93e23147242637: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34986 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:36:47,760 - lava_callback - INFO - Received callback for node 69bdcbea0e93e2314724264c INFO: 172.20.0.1:35002 - "POST /node/69bdcbea0e93e2314724264c HTTP/1.0" 202 Accepted 2026-03-22 22:36:47,766 - lava_callback - INFO - Starting async job submit for node 69bdcbea0e93e2314724264c 2026-03-22 22:36:47,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:36:47,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcbea0e93e2314724264c HTTP/1.1" 200 None 2026-03-22 22:36:47,784 - lava_callback - DEBUG - Processing node 69bdcbea0e93e2314724264c: baseline-arm64-clabbe 2026-03-22 22:36:47,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcbea0e93e2314724264c 2026-03-22 22:36:47,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:36:47,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcbea0e93e2314724264c HTTP/1.1" 200 None 2026-03-22 22:36:47,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:36:47,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:36:47,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:36:47,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcbea0e93e2314724264c HTTP/1.1" 200 None 2026-03-22 22:36:47,884 - lava_callback - ERROR - Error processing callback for node 69bdcbea0e93e2314724264c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40312 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:38:07,840 - lava_callback - INFO - Received callback for node 69bdcc3e0e93e2314724265a INFO: 172.20.0.1:40324 - "POST /node/69bdcc3e0e93e2314724265a HTTP/1.0" 202 Accepted 2026-03-22 22:38:07,846 - lava_callback - INFO - Starting async job submit for node 69bdcc3e0e93e2314724265a 2026-03-22 22:38:07,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:38:07,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcc3e0e93e2314724265a HTTP/1.1" 200 None 2026-03-22 22:38:07,862 - lava_callback - DEBUG - Processing node 69bdcc3e0e93e2314724265a: baseline-arm64-clabbe 2026-03-22 22:38:07,862 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcc3e0e93e2314724265a 2026-03-22 22:38:07,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:38:07,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcc3e0e93e2314724265a HTTP/1.1" 200 None 2026-03-22 22:38:07,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:38:07,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:38:07,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:38:07,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcc3e0e93e2314724265a HTTP/1.1" 200 None 2026-03-22 22:38:07,917 - lava_callback - ERROR - Error processing callback for node 69bdcc3e0e93e2314724265a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53470 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:39:27,868 - lava_callback - INFO - Received callback for node 69bdcc910e93e23147242674 INFO: 172.20.0.1:53478 - "POST /node/69bdcc910e93e23147242674 HTTP/1.0" 202 Accepted 2026-03-22 22:39:27,873 - lava_callback - INFO - Starting async job submit for node 69bdcc910e93e23147242674 2026-03-22 22:39:27,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:39:27,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcc910e93e23147242674 HTTP/1.1" 200 None 2026-03-22 22:39:27,890 - lava_callback - DEBUG - Processing node 69bdcc910e93e23147242674: baseline-arm64-clabbe 2026-03-22 22:39:27,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcc910e93e23147242674 2026-03-22 22:39:27,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:39:27,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcc910e93e23147242674 HTTP/1.1" 200 None 2026-03-22 22:39:27,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:39:27,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:39:27,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:39:27,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcc910e93e23147242674 HTTP/1.1" 200 None 2026-03-22 22:39:27,955 - lava_callback - ERROR - Error processing callback for node 69bdcc910e93e23147242674: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57198 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:40:47,925 - lava_callback - INFO - Received callback for node 69bdcce30e93e2314724268e INFO: 172.20.0.1:57206 - "POST /node/69bdcce30e93e2314724268e HTTP/1.0" 202 Accepted 2026-03-22 22:40:47,931 - lava_callback - INFO - Starting async job submit for node 69bdcce30e93e2314724268e 2026-03-22 22:40:47,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:40:47,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcce30e93e2314724268e HTTP/1.1" 200 None 2026-03-22 22:40:47,947 - lava_callback - DEBUG - Processing node 69bdcce30e93e2314724268e: baseline-arm64-clabbe 2026-03-22 22:40:47,947 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcce30e93e2314724268e 2026-03-22 22:40:47,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:40:47,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcce30e93e2314724268e HTTP/1.1" 200 None 2026-03-22 22:40:47,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:40:47,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-22 22:40:47,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:40:47,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcce30e93e2314724268e HTTP/1.1" 200 None 2026-03-22 22:40:47,999 - lava_callback - ERROR - Error processing callback for node 69bdcce30e93e2314724268e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49174 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:42:47,572 - lava_callback - INFO - Received callback for node 69bdcd600e93e231472426b8 INFO: 172.20.0.1:49184 - "POST /node/69bdcd600e93e231472426b8 HTTP/1.0" 202 Accepted 2026-03-22 22:42:47,578 - lava_callback - INFO - Starting async job submit for node 69bdcd600e93e231472426b8 2026-03-22 22:42:47,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:42:47,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcd600e93e231472426b8 HTTP/1.1" 200 None 2026-03-22 22:42:47,594 - lava_callback - DEBUG - Processing node 69bdcd600e93e231472426b8: baseline-arm-baylibre 2026-03-22 22:42:47,594 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcd600e93e231472426b8 2026-03-22 22:42:47,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:42:47,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcd600e93e231472426b8 HTTP/1.1" 200 None 2026-03-22 22:42:47,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:42:47,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 22:42:47,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:42:47,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcd600e93e231472426b8 HTTP/1.1" 200 None 2026-03-22 22:42:47,654 - lava_callback - ERROR - Error processing callback for node 69bdcd600e93e231472426b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58610 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:43:27,537 - lava_callback - INFO - Received callback for node 69bdcd8b0e93e231472426cc INFO: 172.20.0.1:58626 - "POST /node/69bdcd8b0e93e231472426cc HTTP/1.0" 202 Accepted 2026-03-22 22:43:27,543 - lava_callback - INFO - Starting async job submit for node 69bdcd8b0e93e231472426cc 2026-03-22 22:43:27,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:43:27,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcd8b0e93e231472426cc HTTP/1.1" 200 None 2026-03-22 22:43:27,559 - lava_callback - DEBUG - Processing node 69bdcd8b0e93e231472426cc: baseline-arm-baylibre 2026-03-22 22:43:27,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcd8b0e93e231472426cc 2026-03-22 22:43:27,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:43:27,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcd8b0e93e231472426cc HTTP/1.1" 200 None 2026-03-22 22:43:27,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:43:27,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 22:43:27,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:43:27,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcd8b0e93e231472426cc HTTP/1.1" 200 None 2026-03-22 22:43:27,617 - lava_callback - ERROR - Error processing callback for node 69bdcd8b0e93e231472426cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38162 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:44:07,860 - lava_callback - INFO - Received callback for node 69bdcdb50e93e23147242711 INFO: 172.20.0.1:38172 - "POST /node/69bdcdb50e93e23147242711 HTTP/1.0" 202 Accepted 2026-03-22 22:44:07,866 - lava_callback - INFO - Starting async job submit for node 69bdcdb50e93e23147242711 2026-03-22 22:44:07,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:44:07,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcdb50e93e23147242711 HTTP/1.1" 200 None 2026-03-22 22:44:07,881 - lava_callback - DEBUG - Processing node 69bdcdb50e93e23147242711: baseline-arm-baylibre 2026-03-22 22:44:07,881 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcdb50e93e23147242711 2026-03-22 22:44:07,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:44:07,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcdb50e93e23147242711 HTTP/1.1" 200 None 2026-03-22 22:44:07,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:44:07,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:44:07,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:44:07,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcdb50e93e23147242711 HTTP/1.1" 200 None 2026-03-22 22:44:07,947 - lava_callback - ERROR - Error processing callback for node 69bdcdb50e93e23147242711: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45710 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:45:07,656 - lava_callback - INFO - Received callback for node 69bdcde00e93e23147242735 INFO: 172.20.0.1:45716 - "POST /node/69bdcde00e93e23147242735 HTTP/1.0" 202 Accepted 2026-03-22 22:45:07,661 - lava_callback - INFO - Starting async job submit for node 69bdcde00e93e23147242735 2026-03-22 22:45:07,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:45:07,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcde00e93e23147242735 HTTP/1.1" 200 None 2026-03-22 22:45:07,677 - lava_callback - DEBUG - Processing node 69bdcde00e93e23147242735: baseline-arm-baylibre 2026-03-22 22:45:07,677 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcde00e93e23147242735 2026-03-22 22:45:07,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:45:07,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcde00e93e23147242735 HTTP/1.1" 200 None 2026-03-22 22:45:07,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:45:07,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:45:07,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:45:07,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcde00e93e23147242735 HTTP/1.1" 200 None 2026-03-22 22:45:07,735 - lava_callback - ERROR - Error processing callback for node 69bdcde00e93e23147242735: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47966 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:45:47,739 - lava_callback - INFO - Received callback for node 69bdce0a0e93e23147242781 INFO: 172.20.0.1:47974 - "POST /node/69bdce0a0e93e23147242781 HTTP/1.0" 202 Accepted 2026-03-22 22:45:47,744 - lava_callback - INFO - Starting async job submit for node 69bdce0a0e93e23147242781 2026-03-22 22:45:47,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:45:47,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdce0a0e93e23147242781 HTTP/1.1" 200 None 2026-03-22 22:45:47,760 - lava_callback - DEBUG - Processing node 69bdce0a0e93e23147242781: baseline-arm-baylibre 2026-03-22 22:45:47,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdce0a0e93e23147242781 2026-03-22 22:45:47,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:45:47,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdce0a0e93e23147242781 HTTP/1.1" 200 None 2026-03-22 22:45:47,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:45:47,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:45:47,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:45:47,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdce0a0e93e23147242781 HTTP/1.1" 200 None 2026-03-22 22:45:47,817 - lava_callback - ERROR - Error processing callback for node 69bdce0a0e93e23147242781: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34264 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:46:27,617 - lava_callback - INFO - Received callback for node 69bdce330e93e231472427a3 INFO: 172.20.0.1:34272 - "POST /node/69bdce330e93e231472427a3 HTTP/1.0" 202 Accepted 2026-03-22 22:46:27,623 - lava_callback - INFO - Starting async job submit for node 69bdce330e93e231472427a3 2026-03-22 22:46:27,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:46:27,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdce330e93e231472427a3 HTTP/1.1" 200 None 2026-03-22 22:46:27,638 - lava_callback - DEBUG - Processing node 69bdce330e93e231472427a3: baseline-arm-baylibre 2026-03-22 22:46:27,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdce330e93e231472427a3 2026-03-22 22:46:27,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:46:27,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdce330e93e231472427a3 HTTP/1.1" 200 None 2026-03-22 22:46:27,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:46:27,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:46:27,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:46:27,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdce330e93e231472427a3 HTTP/1.1" 200 None 2026-03-22 22:46:27,690 - lava_callback - ERROR - Error processing callback for node 69bdce330e93e231472427a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55638 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:47:07,770 - lava_callback - INFO - Received callback for node 69bdce5e0e93e231472427c5 INFO: 172.20.0.1:55652 - "POST /node/69bdce5e0e93e231472427c5 HTTP/1.0" 202 Accepted 2026-03-22 22:47:07,775 - lava_callback - INFO - Starting async job submit for node 69bdce5e0e93e231472427c5 2026-03-22 22:47:07,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:47:07,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdce5e0e93e231472427c5 HTTP/1.1" 200 None 2026-03-22 22:47:07,791 - lava_callback - DEBUG - Processing node 69bdce5e0e93e231472427c5: baseline-arm-baylibre 2026-03-22 22:47:07,791 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdce5e0e93e231472427c5 2026-03-22 22:47:07,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:47:07,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdce5e0e93e231472427c5 HTTP/1.1" 200 None 2026-03-22 22:47:07,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:47:07,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:47:07,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:47:07,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdce5e0e93e231472427c5 HTTP/1.1" 200 None 2026-03-22 22:47:07,850 - lava_callback - ERROR - Error processing callback for node 69bdce5e0e93e231472427c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44006 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:47:47,733 - lava_callback - INFO - Received callback for node 69bdce880e93e23147242811 INFO: 172.20.0.1:44012 - "POST /node/69bdce880e93e23147242811 HTTP/1.0" 202 Accepted 2026-03-22 22:47:47,738 - lava_callback - INFO - Starting async job submit for node 69bdce880e93e23147242811 2026-03-22 22:47:47,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:47:47,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdce880e93e23147242811 HTTP/1.1" 200 None 2026-03-22 22:47:47,754 - lava_callback - DEBUG - Processing node 69bdce880e93e23147242811: baseline-arm-baylibre 2026-03-22 22:47:47,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdce880e93e23147242811 2026-03-22 22:47:47,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:47:47,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdce880e93e23147242811 HTTP/1.1" 200 None 2026-03-22 22:47:47,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:47:47,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:47:47,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:47:47,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdce880e93e23147242811 HTTP/1.1" 200 None 2026-03-22 22:47:47,840 - lava_callback - ERROR - Error processing callback for node 69bdce880e93e23147242811: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48270 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:48:27,868 - lava_callback - INFO - Received callback for node 69bdceb20e93e23147242835 INFO: 172.20.0.1:48284 - "POST /node/69bdceb20e93e23147242835 HTTP/1.0" 202 Accepted 2026-03-22 22:48:27,874 - lava_callback - INFO - Starting async job submit for node 69bdceb20e93e23147242835 2026-03-22 22:48:27,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:48:27,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdceb20e93e23147242835 HTTP/1.1" 200 None 2026-03-22 22:48:27,889 - lava_callback - DEBUG - Processing node 69bdceb20e93e23147242835: baseline-arm-baylibre 2026-03-22 22:48:27,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdceb20e93e23147242835 2026-03-22 22:48:27,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:48:27,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdceb20e93e23147242835 HTTP/1.1" 200 None 2026-03-22 22:48:27,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:48:27,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:48:27,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:48:27,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdceb20e93e23147242835 HTTP/1.1" 200 None 2026-03-22 22:48:27,974 - lava_callback - ERROR - Error processing callback for node 69bdceb20e93e23147242835: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35846 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:49:07,831 - lava_callback - INFO - Received callback for node 69bdcedb0e93e2314724287e INFO: 172.20.0.1:35854 - "POST /node/69bdcedb0e93e2314724287e HTTP/1.0" 202 Accepted 2026-03-22 22:49:07,837 - lava_callback - INFO - Starting async job submit for node 69bdcedb0e93e2314724287e 2026-03-22 22:49:07,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:49:07,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcedb0e93e2314724287e HTTP/1.1" 200 None 2026-03-22 22:49:07,863 - lava_callback - DEBUG - Processing node 69bdcedb0e93e2314724287e: baseline-arm-baylibre 2026-03-22 22:49:07,863 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcedb0e93e2314724287e 2026-03-22 22:49:07,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:49:07,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcedb0e93e2314724287e HTTP/1.1" 200 None 2026-03-22 22:49:07,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:49:07,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 22:49:07,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:49:07,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcedb0e93e2314724287e HTTP/1.1" 200 None 2026-03-22 22:49:07,916 - lava_callback - ERROR - Error processing callback for node 69bdcedb0e93e2314724287e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58118 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:49:47,944 - lava_callback - INFO - Received callback for node 69bdcf080e93e23147242899 INFO: 172.20.0.1:58122 - "POST /node/69bdcf080e93e23147242899 HTTP/1.0" 202 Accepted 2026-03-22 22:49:47,949 - lava_callback - INFO - Starting async job submit for node 69bdcf080e93e23147242899 2026-03-22 22:49:47,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:49:47,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcf080e93e23147242899 HTTP/1.1" 200 None 2026-03-22 22:49:47,965 - lava_callback - DEBUG - Processing node 69bdcf080e93e23147242899: baseline-arm-baylibre 2026-03-22 22:49:47,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcf080e93e23147242899 2026-03-22 22:49:47,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:49:47,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcf080e93e23147242899 HTTP/1.1" 200 None 2026-03-22 22:49:47,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:49:47,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:49:47,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:49:48,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcf080e93e23147242899 HTTP/1.1" 200 None 2026-03-22 22:49:48,016 - lava_callback - ERROR - Error processing callback for node 69bdcf080e93e23147242899: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34002 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:50:27,893 - lava_callback - INFO - Received callback for node 69bdcf310e93e231472428c9 INFO: 172.20.0.1:34014 - "POST /node/69bdcf310e93e231472428c9 HTTP/1.0" 202 Accepted 2026-03-22 22:50:27,899 - lava_callback - INFO - Starting async job submit for node 69bdcf310e93e231472428c9 2026-03-22 22:50:27,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:50:27,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcf310e93e231472428c9 HTTP/1.1" 200 None 2026-03-22 22:50:27,915 - lava_callback - DEBUG - Processing node 69bdcf310e93e231472428c9: baseline-arm-baylibre 2026-03-22 22:50:27,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcf310e93e231472428c9 2026-03-22 22:50:27,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:50:27,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcf310e93e231472428c9 HTTP/1.1" 200 None 2026-03-22 22:50:27,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:50:27,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:50:27,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:50:27,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcf310e93e231472428c9 HTTP/1.1" 200 None 2026-03-22 22:50:27,977 - lava_callback - ERROR - Error processing callback for node 69bdcf310e93e231472428c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36736 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:51:28,062 - lava_callback - INFO - Received callback for node 69bdcf5d0e93e2314724290c INFO: 172.20.0.1:36746 - "POST /node/69bdcf5d0e93e2314724290c HTTP/1.0" 202 Accepted 2026-03-22 22:51:28,068 - lava_callback - INFO - Starting async job submit for node 69bdcf5d0e93e2314724290c 2026-03-22 22:51:28,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:51:28,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcf5d0e93e2314724290c HTTP/1.1" 200 None 2026-03-22 22:51:28,084 - lava_callback - DEBUG - Processing node 69bdcf5d0e93e2314724290c: baseline-arm-baylibre 2026-03-22 22:51:28,084 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcf5d0e93e2314724290c 2026-03-22 22:51:28,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:51:28,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcf5d0e93e2314724290c HTTP/1.1" 200 None 2026-03-22 22:51:28,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:51:28,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:51:28,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:51:28,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcf5d0e93e2314724290c HTTP/1.1" 200 None 2026-03-22 22:51:28,141 - lava_callback - ERROR - Error processing callback for node 69bdcf5d0e93e2314724290c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46450 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:52:08,598 - lava_callback - INFO - Received callback for node 69bdcf8a0e93e2314724294e INFO: 172.20.0.1:46462 - "POST /node/69bdcf8a0e93e2314724294e HTTP/1.0" 202 Accepted 2026-03-22 22:52:08,603 - lava_callback - INFO - Starting async job submit for node 69bdcf8a0e93e2314724294e 2026-03-22 22:52:08,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:52:08,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcf8a0e93e2314724294e HTTP/1.1" 200 None 2026-03-22 22:52:08,622 - lava_callback - DEBUG - Processing node 69bdcf8a0e93e2314724294e: baseline-arm-clabbe 2026-03-22 22:52:08,623 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcf8a0e93e2314724294e 2026-03-22 22:52:08,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:52:08,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcf8a0e93e2314724294e HTTP/1.1" 200 None 2026-03-22 22:52:08,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:52:08,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:52:08,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:52:08,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcf8a0e93e2314724294e HTTP/1.1" 200 None 2026-03-22 22:52:08,694 - lava_callback - ERROR - Error processing callback for node 69bdcf8a0e93e2314724294e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35842 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:52:48,104 - lava_callback - INFO - Received callback for node 69bdcfaf0e93e23147242998 INFO: 172.20.0.1:35846 - "POST /node/69bdcfaf0e93e23147242998 HTTP/1.0" 202 Accepted 2026-03-22 22:52:48,110 - lava_callback - INFO - Starting async job submit for node 69bdcfaf0e93e23147242998 2026-03-22 22:52:48,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:52:48,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcfaf0e93e23147242998 HTTP/1.1" 200 None 2026-03-22 22:52:48,126 - lava_callback - DEBUG - Processing node 69bdcfaf0e93e23147242998: baseline-arm-baylibre 2026-03-22 22:52:48,126 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcfaf0e93e23147242998 2026-03-22 22:52:48,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:52:48,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcfaf0e93e23147242998 HTTP/1.1" 200 None 2026-03-22 22:52:48,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:52:48,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 22:52:48,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:52:48,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcfaf0e93e23147242998 HTTP/1.1" 200 None 2026-03-22 22:52:48,182 - lava_callback - ERROR - Error processing callback for node 69bdcfaf0e93e23147242998: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38354 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:53:28,047 - lava_callback - INFO - Received callback for node 69bdcfdb0e93e231472429b6 INFO: 172.20.0.1:38356 - "POST /node/69bdcfdb0e93e231472429b6 HTTP/1.0" 202 Accepted 2026-03-22 22:53:28,053 - lava_callback - INFO - Starting async job submit for node 69bdcfdb0e93e231472429b6 2026-03-22 22:53:28,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:53:28,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcfdb0e93e231472429b6 HTTP/1.1" 200 None 2026-03-22 22:53:28,073 - lava_callback - DEBUG - Processing node 69bdcfdb0e93e231472429b6: baseline-arm-baylibre 2026-03-22 22:53:28,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdcfdb0e93e231472429b6 2026-03-22 22:53:28,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:53:28,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdcfdb0e93e231472429b6 HTTP/1.1" 200 None 2026-03-22 22:53:28,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:53:28,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:53:28,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:53:28,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdcfdb0e93e231472429b6 HTTP/1.1" 200 None 2026-03-22 22:53:28,126 - lava_callback - ERROR - Error processing callback for node 69bdcfdb0e93e231472429b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37306 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:54:08,563 - lava_callback - INFO - Received callback for node 69bdd0080e93e231472429fa INFO: 172.20.0.1:37308 - "POST /node/69bdd0080e93e231472429fa HTTP/1.0" 202 Accepted 2026-03-22 22:54:08,569 - lava_callback - INFO - Starting async job submit for node 69bdd0080e93e231472429fa 2026-03-22 22:54:08,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:54:08,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0080e93e231472429fa HTTP/1.1" 200 None 2026-03-22 22:54:08,585 - lava_callback - DEBUG - Processing node 69bdd0080e93e231472429fa: baseline-arm-clabbe 2026-03-22 22:54:08,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd0080e93e231472429fa 2026-03-22 22:54:08,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:54:08,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0080e93e231472429fa HTTP/1.1" 200 None 2026-03-22 22:54:08,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:54:08,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:54:08,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:54:08,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd0080e93e231472429fa HTTP/1.1" 200 None 2026-03-22 22:54:08,639 - lava_callback - ERROR - Error processing callback for node 69bdd0080e93e231472429fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52108 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:54:48,535 - lava_callback - INFO - Received callback for node 69bdd0320e93e23147242a1a INFO: 172.20.0.1:52118 - "POST /node/69bdd0320e93e23147242a1a HTTP/1.0" 202 Accepted 2026-03-22 22:54:48,542 - lava_callback - INFO - Starting async job submit for node 69bdd0320e93e23147242a1a 2026-03-22 22:54:48,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:54:48,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0320e93e23147242a1a HTTP/1.1" 200 None 2026-03-22 22:54:48,559 - lava_callback - DEBUG - Processing node 69bdd0320e93e23147242a1a: baseline-arm-clabbe 2026-03-22 22:54:48,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd0320e93e23147242a1a 2026-03-22 22:54:48,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:54:48,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0320e93e23147242a1a HTTP/1.1" 200 None 2026-03-22 22:54:48,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:54:48,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:54:48,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:54:48,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd0320e93e23147242a1a HTTP/1.1" 200 None 2026-03-22 22:54:48,625 - lava_callback - ERROR - Error processing callback for node 69bdd0320e93e23147242a1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59708 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:55:28,410 - lava_callback - INFO - Received callback for node 69bdd05a0e93e23147242a3a INFO: 172.20.0.1:59714 - "POST /node/69bdd05a0e93e23147242a3a HTTP/1.0" 202 Accepted 2026-03-22 22:55:28,416 - lava_callback - INFO - Starting async job submit for node 69bdd05a0e93e23147242a3a 2026-03-22 22:55:28,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:55:28,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd05a0e93e23147242a3a HTTP/1.1" 200 None 2026-03-22 22:55:28,438 - lava_callback - DEBUG - Processing node 69bdd05a0e93e23147242a3a: baseline-arm-baylibre 2026-03-22 22:55:28,439 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd05a0e93e23147242a3a 2026-03-22 22:55:28,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:55:28,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd05a0e93e23147242a3a HTTP/1.1" 200 None 2026-03-22 22:55:28,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:55:28,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:55:28,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:55:28,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd05a0e93e23147242a3a HTTP/1.1" 200 None 2026-03-22 22:55:28,490 - lava_callback - ERROR - Error processing callback for node 69bdd05a0e93e23147242a3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36952 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:56:08,698 - lava_callback - INFO - Received callback for node 69bdd0860e93e23147242a7f INFO: 172.20.0.1:36962 - "POST /node/69bdd0860e93e23147242a7f HTTP/1.0" 202 Accepted 2026-03-22 22:56:08,703 - lava_callback - INFO - Starting async job submit for node 69bdd0860e93e23147242a7f 2026-03-22 22:56:08,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:56:08,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0860e93e23147242a7f HTTP/1.1" 200 None 2026-03-22 22:56:08,721 - lava_callback - DEBUG - Processing node 69bdd0860e93e23147242a7f: baseline-arm-clabbe 2026-03-22 22:56:08,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd0860e93e23147242a7f 2026-03-22 22:56:08,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:56:08,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0860e93e23147242a7f HTTP/1.1" 200 None 2026-03-22 22:56:08,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:56:08,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:56:08,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:56:08,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd0860e93e23147242a7f HTTP/1.1" 200 None 2026-03-22 22:56:08,803 - lava_callback - ERROR - Error processing callback for node 69bdd0860e93e23147242a7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46098 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:57:08,667 - lava_callback - INFO - Received callback for node 69bdd0b00e93e23147242a9c INFO: 172.20.0.1:46106 - "POST /node/69bdd0b00e93e23147242a9c HTTP/1.0" 202 Accepted 2026-03-22 22:57:08,674 - lava_callback - INFO - Starting async job submit for node 69bdd0b00e93e23147242a9c 2026-03-22 22:57:08,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:57:08,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0b00e93e23147242a9c HTTP/1.1" 200 None 2026-03-22 22:57:08,690 - lava_callback - DEBUG - Processing node 69bdd0b00e93e23147242a9c: baseline-arm-clabbe 2026-03-22 22:57:08,690 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd0b00e93e23147242a9c 2026-03-22 22:57:08,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:57:08,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0b00e93e23147242a9c HTTP/1.1" 200 None 2026-03-22 22:57:08,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:57:08,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:57:08,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:57:08,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd0b00e93e23147242a9c HTTP/1.1" 200 None 2026-03-22 22:57:08,748 - lava_callback - ERROR - Error processing callback for node 69bdd0b00e93e23147242a9c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53154 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:57:28,322 - lava_callback - INFO - Received callback for node 69bdd0d50e93e23147242aca INFO: 172.20.0.1:53158 - "POST /node/69bdd0d50e93e23147242aca HTTP/1.0" 202 Accepted 2026-03-22 22:57:28,327 - lava_callback - INFO - Starting async job submit for node 69bdd0d50e93e23147242aca 2026-03-22 22:57:28,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:57:28,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0d50e93e23147242aca HTTP/1.1" 200 None 2026-03-22 22:57:28,343 - lava_callback - DEBUG - Processing node 69bdd0d50e93e23147242aca: baseline-arm-baylibre 2026-03-22 22:57:28,343 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd0d50e93e23147242aca 2026-03-22 22:57:28,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:57:28,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0d50e93e23147242aca HTTP/1.1" 200 None 2026-03-22 22:57:28,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:57:28,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 22:57:28,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:57:28,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd0d50e93e23147242aca HTTP/1.1" 200 None 2026-03-22 22:57:28,418 - lava_callback - ERROR - Error processing callback for node 69bdd0d50e93e23147242aca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57104 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:58:28,331 - lava_callback - INFO - Received callback for node 69bdd0ff0e93e23147242b0d INFO: 172.20.0.1:57106 - "POST /node/69bdd0ff0e93e23147242b0d HTTP/1.0" 202 Accepted 2026-03-22 22:58:28,336 - lava_callback - INFO - Starting async job submit for node 69bdd0ff0e93e23147242b0d 2026-03-22 22:58:28,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:58:28,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0ff0e93e23147242b0d HTTP/1.1" 200 None 2026-03-22 22:58:28,357 - lava_callback - DEBUG - Processing node 69bdd0ff0e93e23147242b0d: baseline-arm-baylibre 2026-03-22 22:58:28,357 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd0ff0e93e23147242b0d 2026-03-22 22:58:28,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:58:28,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd0ff0e93e23147242b0d HTTP/1.1" 200 None 2026-03-22 22:58:28,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:58:28,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 22:58:28,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:58:28,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd0ff0e93e23147242b0d HTTP/1.1" 200 None 2026-03-22 22:58:28,420 - lava_callback - ERROR - Error processing callback for node 69bdd0ff0e93e23147242b0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59910 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:59:08,392 - lava_callback - INFO - Received callback for node 69bdd12b0e93e23147242b41 INFO: 172.20.0.1:59916 - "POST /node/69bdd12b0e93e23147242b41 HTTP/1.0" 202 Accepted 2026-03-22 22:59:08,398 - lava_callback - INFO - Starting async job submit for node 69bdd12b0e93e23147242b41 2026-03-22 22:59:08,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:59:08,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd12b0e93e23147242b41 HTTP/1.1" 200 None 2026-03-22 22:59:08,414 - lava_callback - DEBUG - Processing node 69bdd12b0e93e23147242b41: baseline-arm-baylibre 2026-03-22 22:59:08,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd12b0e93e23147242b41 2026-03-22 22:59:08,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:59:08,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd12b0e93e23147242b41 HTTP/1.1" 200 None 2026-03-22 22:59:08,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:59:08,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 22:59:08,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:59:08,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd12b0e93e23147242b41 HTTP/1.1" 200 None 2026-03-22 22:59:08,475 - lava_callback - ERROR - Error processing callback for node 69bdd12b0e93e23147242b41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53086 - "GET / HTTP/1.0" 200 OK 2026-03-22 22:59:48,830 - lava_callback - INFO - Received callback for node 69bdd1580e93e23147242b87 INFO: 172.20.0.1:53098 - "POST /node/69bdd1580e93e23147242b87 HTTP/1.0" 202 Accepted 2026-03-22 22:59:48,835 - lava_callback - INFO - Starting async job submit for node 69bdd1580e93e23147242b87 2026-03-22 22:59:48,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:59:48,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1580e93e23147242b87 HTTP/1.1" 200 None 2026-03-22 22:59:48,852 - lava_callback - DEBUG - Processing node 69bdd1580e93e23147242b87: baseline-arm-clabbe 2026-03-22 22:59:48,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd1580e93e23147242b87 2026-03-22 22:59:48,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:59:48,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1580e93e23147242b87 HTTP/1.1" 200 None 2026-03-22 22:59:48,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:59:48,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 22:59:48,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 22:59:48,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd1580e93e23147242b87 HTTP/1.1" 200 None 2026-03-22 22:59:48,902 - lava_callback - ERROR - Error processing callback for node 69bdd1580e93e23147242b87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53556 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:00:28,740 - lava_callback - INFO - Received callback for node 69bdd1790e93e23147242b9f INFO: 172.20.0.1:53568 - "POST /node/69bdd1790e93e23147242b9f HTTP/1.0" 202 Accepted 2026-03-22 23:00:28,746 - lava_callback - INFO - Starting async job submit for node 69bdd1790e93e23147242b9f 2026-03-22 23:00:28,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:00:28,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1790e93e23147242b9f HTTP/1.1" 200 None 2026-03-22 23:00:28,762 - lava_callback - DEBUG - Processing node 69bdd1790e93e23147242b9f: baseline-arm-clabbe 2026-03-22 23:00:28,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd1790e93e23147242b9f 2026-03-22 23:00:28,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:00:28,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1790e93e23147242b9f HTTP/1.1" 200 None 2026-03-22 23:00:28,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:00:28,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 23:00:28,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:00:28,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd1790e93e23147242b9f HTTP/1.1" 200 None 2026-03-22 23:00:28,819 - lava_callback - ERROR - Error processing callback for node 69bdd1790e93e23147242b9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41442 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:00:48,500 - lava_callback - INFO - Received callback for node 69bdd1940e93e23147242baa INFO: 172.20.0.1:41452 - "POST /node/69bdd1940e93e23147242baa HTTP/1.0" 202 Accepted 2026-03-22 23:00:48,505 - lava_callback - INFO - Starting async job submit for node 69bdd1940e93e23147242baa 2026-03-22 23:00:48,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:00:48,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1940e93e23147242baa HTTP/1.1" 200 None 2026-03-22 23:00:48,522 - lava_callback - DEBUG - Processing node 69bdd1940e93e23147242baa: baseline-arm-baylibre 2026-03-22 23:00:48,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd1940e93e23147242baa 2026-03-22 23:00:48,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:00:48,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1940e93e23147242baa HTTP/1.1" 200 None 2026-03-22 23:00:48,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:00:48,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:00:48,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:00:48,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd1940e93e23147242baa HTTP/1.1" 200 None 2026-03-22 23:00:48,591 - lava_callback - ERROR - Error processing callback for node 69bdd1940e93e23147242baa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43172 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:01:08,834 - lava_callback - INFO - Received callback for node 69bdd1b10e93e23147242bb3 INFO: 172.20.0.1:43182 - "POST /node/69bdd1b10e93e23147242bb3 HTTP/1.0" 202 Accepted 2026-03-22 23:01:08,839 - lava_callback - INFO - Starting async job submit for node 69bdd1b10e93e23147242bb3 2026-03-22 23:01:08,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:01:08,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1b10e93e23147242bb3 HTTP/1.1" 200 None 2026-03-22 23:01:08,857 - lava_callback - DEBUG - Processing node 69bdd1b10e93e23147242bb3: baseline-arm-clabbe 2026-03-22 23:01:08,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd1b10e93e23147242bb3 2026-03-22 23:01:08,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:01:08,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1b10e93e23147242bb3 HTTP/1.1" 200 None 2026-03-22 23:01:08,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:01:08,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:01:08,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:01:08,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd1b10e93e23147242bb3 HTTP/1.1" 200 None 2026-03-22 23:01:08,914 - lava_callback - ERROR - Error processing callback for node 69bdd1b10e93e23147242bb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37250 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:01:48,851 - lava_callback - INFO - Received callback for node 69bdd1cd0e93e23147242bbf INFO: 172.20.0.1:37260 - "POST /node/69bdd1cd0e93e23147242bbf HTTP/1.0" 202 Accepted 2026-03-22 23:01:48,856 - lava_callback - INFO - Starting async job submit for node 69bdd1cd0e93e23147242bbf 2026-03-22 23:01:48,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:01:48,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1cd0e93e23147242bbf HTTP/1.1" 200 None 2026-03-22 23:01:48,878 - lava_callback - DEBUG - Processing node 69bdd1cd0e93e23147242bbf: baseline-arm-clabbe 2026-03-22 23:01:48,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd1cd0e93e23147242bbf 2026-03-22 23:01:48,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:01:48,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1cd0e93e23147242bbf HTTP/1.1" 200 None 2026-03-22 23:01:48,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:01:48,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 23:01:48,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:01:48,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd1cd0e93e23147242bbf HTTP/1.1" 200 None 2026-03-22 23:01:48,932 - lava_callback - ERROR - Error processing callback for node 69bdd1cd0e93e23147242bbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41432 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:02:08,865 - lava_callback - INFO - Received callback for node 69bdd1eb0e93e23147242bc9 INFO: 172.20.0.1:41438 - "POST /node/69bdd1eb0e93e23147242bc9 HTTP/1.0" 202 Accepted 2026-03-22 23:02:08,870 - lava_callback - INFO - Starting async job submit for node 69bdd1eb0e93e23147242bc9 2026-03-22 23:02:08,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:02:08,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1eb0e93e23147242bc9 HTTP/1.1" 200 None 2026-03-22 23:02:08,886 - lava_callback - DEBUG - Processing node 69bdd1eb0e93e23147242bc9: baseline-arm-clabbe 2026-03-22 23:02:08,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd1eb0e93e23147242bc9 2026-03-22 23:02:08,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:02:08,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd1eb0e93e23147242bc9 HTTP/1.1" 200 None 2026-03-22 23:02:08,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:02:08,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:02:08,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:02:08,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd1eb0e93e23147242bc9 HTTP/1.1" 200 None 2026-03-22 23:02:08,949 - lava_callback - ERROR - Error processing callback for node 69bdd1eb0e93e23147242bc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44286 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:02:48,867 - lava_callback - INFO - Received callback for node 69bdd2070e93e23147242bd2 INFO: 172.20.0.1:44294 - "POST /node/69bdd2070e93e23147242bd2 HTTP/1.0" 202 Accepted 2026-03-22 23:02:48,872 - lava_callback - INFO - Starting async job submit for node 69bdd2070e93e23147242bd2 2026-03-22 23:02:48,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:02:48,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd2070e93e23147242bd2 HTTP/1.1" 200 None 2026-03-22 23:02:48,891 - lava_callback - DEBUG - Processing node 69bdd2070e93e23147242bd2: baseline-arm-clabbe 2026-03-22 23:02:48,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd2070e93e23147242bd2 2026-03-22 23:02:48,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:02:48,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd2070e93e23147242bd2 HTTP/1.1" 200 None 2026-03-22 23:02:48,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:02:48,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:02:48,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:02:48,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd2070e93e23147242bd2 HTTP/1.1" 200 None 2026-03-22 23:02:48,942 - lava_callback - ERROR - Error processing callback for node 69bdd2070e93e23147242bd2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54942 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:03:08,917 - lava_callback - INFO - Received callback for node 69bdd2220e93e23147242bdb INFO: 172.20.0.1:54956 - "POST /node/69bdd2220e93e23147242bdb HTTP/1.0" 202 Accepted 2026-03-22 23:03:08,922 - lava_callback - INFO - Starting async job submit for node 69bdd2220e93e23147242bdb 2026-03-22 23:03:08,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:03:08,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd2220e93e23147242bdb HTTP/1.1" 200 None 2026-03-22 23:03:08,953 - lava_callback - DEBUG - Processing node 69bdd2220e93e23147242bdb: baseline-arm-clabbe 2026-03-22 23:03:08,953 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd2220e93e23147242bdb 2026-03-22 23:03:08,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:03:08,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd2220e93e23147242bdb HTTP/1.1" 200 None 2026-03-22 23:03:08,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:03:09,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 23:03:09,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:03:09,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd2220e93e23147242bdb HTTP/1.1" 200 None 2026-03-22 23:03:09,040 - lava_callback - ERROR - Error processing callback for node 69bdd2220e93e23147242bdb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49440 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:03:48,930 - lava_callback - INFO - Received callback for node 69bdd23f0e93e23147242be4 INFO: 172.20.0.1:49452 - "POST /node/69bdd23f0e93e23147242be4 HTTP/1.0" 202 Accepted 2026-03-22 23:03:48,936 - lava_callback - INFO - Starting async job submit for node 69bdd23f0e93e23147242be4 2026-03-22 23:03:48,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:03:48,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd23f0e93e23147242be4 HTTP/1.1" 200 None 2026-03-22 23:03:48,956 - lava_callback - DEBUG - Processing node 69bdd23f0e93e23147242be4: baseline-arm-clabbe 2026-03-22 23:03:48,957 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd23f0e93e23147242be4 2026-03-22 23:03:48,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:03:48,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd23f0e93e23147242be4 HTTP/1.1" 200 None 2026-03-22 23:03:48,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:03:48,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad09221a77312e2220ee1 HTTP/1.1" 200 None 2026-03-22 23:03:48,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:03:49,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd23f0e93e23147242be4 HTTP/1.1" 200 None 2026-03-22 23:03:49,012 - lava_callback - ERROR - Error processing callback for node 69bdd23f0e93e23147242be4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51784 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:04:08,973 - lava_callback - INFO - Received callback for node 69bdd25b0e93e23147242be9 INFO: 172.20.0.1:51800 - "POST /node/69bdd25b0e93e23147242be9 HTTP/1.0" 202 Accepted 2026-03-22 23:04:08,979 - lava_callback - INFO - Starting async job submit for node 69bdd25b0e93e23147242be9 2026-03-22 23:04:08,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:08,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd25b0e93e23147242be9 HTTP/1.1" 200 None 2026-03-22 23:04:08,994 - lava_callback - DEBUG - Processing node 69bdd25b0e93e23147242be9: baseline-arm-clabbe 2026-03-22 23:04:08,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd25b0e93e23147242be9 2026-03-22 23:04:08,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:09,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd25b0e93e23147242be9 HTTP/1.1" 200 None 2026-03-22 23:04:09,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:09,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:04:09,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:09,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd25b0e93e23147242be9 HTTP/1.1" 200 None 2026-03-22 23:04:09,059 - lava_callback - ERROR - Error processing callback for node 69bdd25b0e93e23147242be9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58436 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:04:28,968 - lava_callback - INFO - Received callback for node 69bdd2780e93e23147242bf4 INFO: 172.20.0.1:58438 - "POST /node/69bdd2780e93e23147242bf4 HTTP/1.0" 202 Accepted 2026-03-22 23:04:28,973 - lava_callback - INFO - Starting async job submit for node 69bdd2780e93e23147242bf4 2026-03-22 23:04:28,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:28,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd2780e93e23147242bf4 HTTP/1.1" 200 None 2026-03-22 23:04:28,989 - lava_callback - DEBUG - Processing node 69bdd2780e93e23147242bf4: baseline-arm-clabbe 2026-03-22 23:04:28,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd2780e93e23147242bf4 2026-03-22 23:04:28,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:29,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd2780e93e23147242bf4 HTTP/1.1" 200 None 2026-03-22 23:04:29,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:29,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:04:29,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:29,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd2780e93e23147242bf4 HTTP/1.1" 200 None 2026-03-22 23:04:29,046 - lava_callback - ERROR - Error processing callback for node 69bdd2780e93e23147242bf4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57982 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:04:48,816 - lava_callback - INFO - Received callback for node 69bdd28f0e93e23147242bfe INFO: 172.20.0.1:57996 - "POST /node/69bdd28f0e93e23147242bfe HTTP/1.0" 202 Accepted 2026-03-22 23:04:48,822 - lava_callback - INFO - Starting async job submit for node 69bdd28f0e93e23147242bfe 2026-03-22 23:04:48,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:48,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd28f0e93e23147242bfe HTTP/1.1" 200 None 2026-03-22 23:04:48,847 - lava_callback - DEBUG - Processing node 69bdd28f0e93e23147242bfe: baseline-arm-baylibre 2026-03-22 23:04:48,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd28f0e93e23147242bfe 2026-03-22 23:04:48,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:48,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd28f0e93e23147242bfe HTTP/1.1" 200 None 2026-03-22 23:04:48,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:48,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:04:48,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:04:48,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd28f0e93e23147242bfe HTTP/1.1" 200 None 2026-03-22 23:04:48,907 - lava_callback - ERROR - Error processing callback for node 69bdd28f0e93e23147242bfe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35500 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:05:28,674 - lava_callback - INFO - Received callback for node 69bdd2ab0e93e23147242c09 INFO: 172.20.0.1:35514 - "POST /node/69bdd2ab0e93e23147242c09 HTTP/1.0" 202 Accepted 2026-03-22 23:05:28,680 - lava_callback - INFO - Starting async job submit for node 69bdd2ab0e93e23147242c09 2026-03-22 23:05:28,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:05:28,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd2ab0e93e23147242c09 HTTP/1.1" 200 None 2026-03-22 23:05:28,700 - lava_callback - DEBUG - Processing node 69bdd2ab0e93e23147242c09: baseline-arm-baylibre 2026-03-22 23:05:28,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd2ab0e93e23147242c09 2026-03-22 23:05:28,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:05:28,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd2ab0e93e23147242c09 HTTP/1.1" 200 None 2026-03-22 23:05:28,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:05:28,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:05:28,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:05:28,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd2ab0e93e23147242c09 HTTP/1.1" 200 None 2026-03-22 23:05:28,795 - lava_callback - ERROR - Error processing callback for node 69bdd2ab0e93e23147242c09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40418 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:05:48,709 - lava_callback - INFO - Received callback for node 69bdd2c60e93e23147242c14 INFO: 172.20.0.1:40428 - "POST /node/69bdd2c60e93e23147242c14 HTTP/1.0" 202 Accepted 2026-03-22 23:05:48,714 - lava_callback - INFO - Starting async job submit for node 69bdd2c60e93e23147242c14 2026-03-22 23:05:48,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:05:48,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd2c60e93e23147242c14 HTTP/1.1" 200 None 2026-03-22 23:05:48,730 - lava_callback - DEBUG - Processing node 69bdd2c60e93e23147242c14: baseline-arm-baylibre 2026-03-22 23:05:48,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd2c60e93e23147242c14 2026-03-22 23:05:48,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:05:48,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd2c60e93e23147242c14 HTTP/1.1" 200 None 2026-03-22 23:05:48,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:05:48,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:05:48,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:05:48,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd2c60e93e23147242c14 HTTP/1.1" 200 None 2026-03-22 23:05:48,820 - lava_callback - ERROR - Error processing callback for node 69bdd2c60e93e23147242c14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57884 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:13:09,270 - lava_callback - INFO - Received callback for node 69bdd4720e93e23147242c58 INFO: 172.20.0.1:57886 - "POST /node/69bdd4720e93e23147242c58 HTTP/1.0" 202 Accepted 2026-03-22 23:13:09,276 - lava_callback - INFO - Starting async job submit for node 69bdd4720e93e23147242c58 2026-03-22 23:13:09,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:13:09,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd4720e93e23147242c58 HTTP/1.1" 200 None 2026-03-22 23:13:09,291 - lava_callback - DEBUG - Processing node 69bdd4720e93e23147242c58: baseline-arm-baylibre 2026-03-22 23:13:09,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd4720e93e23147242c58 2026-03-22 23:13:09,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:13:09,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd4720e93e23147242c58 HTTP/1.1" 200 None 2026-03-22 23:13:09,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:13:09,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:13:09,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:13:09,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd4720e93e23147242c58 HTTP/1.1" 200 None 2026-03-22 23:13:09,360 - lava_callback - ERROR - Error processing callback for node 69bdd4720e93e23147242c58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34924 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:13:49,110 - lava_callback - INFO - Received callback for node 69bdd49c0e93e23147242c85 INFO: 172.20.0.1:34936 - "POST /node/69bdd49c0e93e23147242c85 HTTP/1.0" 202 Accepted 2026-03-22 23:13:49,116 - lava_callback - INFO - Starting async job submit for node 69bdd49c0e93e23147242c85 2026-03-22 23:13:49,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:13:49,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd49c0e93e23147242c85 HTTP/1.1" 200 None 2026-03-22 23:13:49,131 - lava_callback - DEBUG - Processing node 69bdd49c0e93e23147242c85: baseline-arm-baylibre 2026-03-22 23:13:49,131 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd49c0e93e23147242c85 2026-03-22 23:13:49,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:13:49,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd49c0e93e23147242c85 HTTP/1.1" 200 None 2026-03-22 23:13:49,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:13:49,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:13:49,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:13:49,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd49c0e93e23147242c85 HTTP/1.1" 200 None 2026-03-22 23:13:49,194 - lava_callback - ERROR - Error processing callback for node 69bdd49c0e93e23147242c85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59112 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:14:29,177 - lava_callback - INFO - Received callback for node 69bdd4d20e93e23147242ca5 INFO: 172.20.0.1:59114 - "POST /node/69bdd4d20e93e23147242ca5 HTTP/1.0" 202 Accepted 2026-03-22 23:14:29,182 - lava_callback - INFO - Starting async job submit for node 69bdd4d20e93e23147242ca5 2026-03-22 23:14:29,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:14:29,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd4d20e93e23147242ca5 HTTP/1.1" 200 None 2026-03-22 23:14:29,198 - lava_callback - DEBUG - Processing node 69bdd4d20e93e23147242ca5: baseline-arm64-baylibre 2026-03-22 23:14:29,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd4d20e93e23147242ca5 2026-03-22 23:14:29,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:14:29,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd4d20e93e23147242ca5 HTTP/1.1" 200 None 2026-03-22 23:14:29,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:14:29,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 23:14:29,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:14:29,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd4d20e93e23147242ca5 HTTP/1.1" 200 None 2026-03-22 23:14:29,254 - lava_callback - ERROR - Error processing callback for node 69bdd4d20e93e23147242ca5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42660 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:24:09,844 - lava_callback - INFO - Received callback for node 69bdd70a0e93e23147242d1a INFO: 172.20.0.1:54924 - "POST /node/69bdd70a0e93e23147242d1a HTTP/1.0" 202 Accepted 2026-03-22 23:24:09,850 - lava_callback - INFO - Starting async job submit for node 69bdd70a0e93e23147242d1a 2026-03-22 23:24:09,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:24:09,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd70a0e93e23147242d1a HTTP/1.1" 200 None 2026-03-22 23:24:09,866 - lava_callback - DEBUG - Processing node 69bdd70a0e93e23147242d1a: baseline-arm64-clabbe 2026-03-22 23:24:09,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd70a0e93e23147242d1a 2026-03-22 23:24:09,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:24:09,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd70a0e93e23147242d1a HTTP/1.1" 200 None 2026-03-22 23:24:09,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:24:09,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 23:24:09,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:24:09,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd70a0e93e23147242d1a HTTP/1.1" 200 None 2026-03-22 23:24:09,918 - lava_callback - ERROR - Error processing callback for node 69bdd70a0e93e23147242d1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:25:29,887 - lava_callback - INFO - Received callback for node 69bdd75c0e93e23147242d27 INFO: 172.20.0.1:38140 - "POST /node/69bdd75c0e93e23147242d27 HTTP/1.0" 202 Accepted 2026-03-22 23:25:29,893 - lava_callback - INFO - Starting async job submit for node 69bdd75c0e93e23147242d27 2026-03-22 23:25:29,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:25:29,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd75c0e93e23147242d27 HTTP/1.1" 200 None 2026-03-22 23:25:29,909 - lava_callback - DEBUG - Processing node 69bdd75c0e93e23147242d27: baseline-arm64-clabbe 2026-03-22 23:25:29,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd75c0e93e23147242d27 2026-03-22 23:25:29,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:25:29,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd75c0e93e23147242d27 HTTP/1.1" 200 None 2026-03-22 23:25:29,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:25:29,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 23:25:29,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:25:29,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd75c0e93e23147242d27 HTTP/1.1" 200 None 2026-03-22 23:25:29,960 - lava_callback - ERROR - Error processing callback for node 69bdd75c0e93e23147242d27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39564 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:26:49,954 - lava_callback - INFO - Received callback for node 69bdd7ad0e93e23147242d32 INFO: 172.20.0.1:42706 - "POST /node/69bdd7ad0e93e23147242d32 HTTP/1.0" 202 Accepted 2026-03-22 23:26:49,959 - lava_callback - INFO - Starting async job submit for node 69bdd7ad0e93e23147242d32 2026-03-22 23:26:49,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:26:49,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd7ad0e93e23147242d32 HTTP/1.1" 200 None 2026-03-22 23:26:49,982 - lava_callback - DEBUG - Processing node 69bdd7ad0e93e23147242d32: baseline-arm64-clabbe 2026-03-22 23:26:49,982 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd7ad0e93e23147242d32 2026-03-22 23:26:49,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:26:49,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd7ad0e93e23147242d32 HTTP/1.1" 200 None 2026-03-22 23:26:49,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:26:50,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 23:26:50,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:26:50,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd7ad0e93e23147242d32 HTTP/1.1" 200 None 2026-03-22 23:26:50,046 - lava_callback - ERROR - Error processing callback for node 69bdd7ad0e93e23147242d32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59208 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:28:10,093 - lava_callback - INFO - Received callback for node 69bdd7fc0e93e23147242d3a INFO: 172.20.0.1:59526 - "POST /node/69bdd7fc0e93e23147242d3a HTTP/1.0" 202 Accepted 2026-03-22 23:28:10,098 - lava_callback - INFO - Starting async job submit for node 69bdd7fc0e93e23147242d3a 2026-03-22 23:28:10,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:28:10,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd7fc0e93e23147242d3a HTTP/1.1" 200 None 2026-03-22 23:28:10,114 - lava_callback - DEBUG - Processing node 69bdd7fc0e93e23147242d3a: baseline-arm64-clabbe 2026-03-22 23:28:10,115 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd7fc0e93e23147242d3a 2026-03-22 23:28:10,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:28:10,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd7fc0e93e23147242d3a HTTP/1.1" 200 None 2026-03-22 23:28:10,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:28:10,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 23:28:10,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:28:10,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd7fc0e93e23147242d3a HTTP/1.1" 200 None 2026-03-22 23:28:10,179 - lava_callback - ERROR - Error processing callback for node 69bdd7fc0e93e23147242d3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39208 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:29:10,083 - lava_callback - INFO - Received callback for node 69bdd8410e93e23147242d45 INFO: 172.20.0.1:39372 - "POST /node/69bdd8410e93e23147242d45 HTTP/1.0" 202 Accepted 2026-03-22 23:29:10,088 - lava_callback - INFO - Starting async job submit for node 69bdd8410e93e23147242d45 2026-03-22 23:29:10,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:29:10,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd8410e93e23147242d45 HTTP/1.1" 200 None 2026-03-22 23:29:10,104 - lava_callback - DEBUG - Processing node 69bdd8410e93e23147242d45: baseline-arm-baylibre 2026-03-22 23:29:10,104 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd8410e93e23147242d45 2026-03-22 23:29:10,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:29:10,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd8410e93e23147242d45 HTTP/1.1" 200 None 2026-03-22 23:29:10,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:29:10,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:29:10,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:29:10,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd8410e93e23147242d45 HTTP/1.1" 200 None 2026-03-22 23:29:10,165 - lava_callback - ERROR - Error processing callback for node 69bdd8410e93e23147242d45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54282 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:30:10,263 - lava_callback - INFO - Received callback for node 69bdd87c0e93e23147242d56 INFO: 172.20.0.1:59804 - "POST /node/69bdd87c0e93e23147242d56 HTTP/1.0" 202 Accepted 2026-03-22 23:30:10,269 - lava_callback - INFO - Starting async job submit for node 69bdd87c0e93e23147242d56 2026-03-22 23:30:10,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:30:10,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd87c0e93e23147242d56 HTTP/1.1" 200 None 2026-03-22 23:30:10,310 - lava_callback - DEBUG - Processing node 69bdd87c0e93e23147242d56: baseline-arm64-clabbe 2026-03-22 23:30:10,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd87c0e93e23147242d56 2026-03-22 23:30:10,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:30:10,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd87c0e93e23147242d56 HTTP/1.1" 200 None 2026-03-22 23:30:10,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:30:10,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 23:30:10,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:30:10,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd87c0e93e23147242d56 HTTP/1.1" 200 None 2026-03-22 23:30:10,395 - lava_callback - ERROR - Error processing callback for node 69bdd87c0e93e23147242d56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54380 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:31:30,148 - lava_callback - INFO - Received callback for node 69bdd8cd0e93e23147242d69 INFO: 172.20.0.1:33004 - "POST /node/69bdd8cd0e93e23147242d69 HTTP/1.0" 202 Accepted 2026-03-22 23:31:30,154 - lava_callback - INFO - Starting async job submit for node 69bdd8cd0e93e23147242d69 2026-03-22 23:31:30,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:31:30,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd8cd0e93e23147242d69 HTTP/1.1" 200 None 2026-03-22 23:31:30,169 - lava_callback - DEBUG - Processing node 69bdd8cd0e93e23147242d69: baseline-arm64-clabbe 2026-03-22 23:31:30,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd8cd0e93e23147242d69 2026-03-22 23:31:30,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:31:30,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd8cd0e93e23147242d69 HTTP/1.1" 200 None 2026-03-22 23:31:30,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:31:30,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd73420e93e2314723e2c4 HTTP/1.1" 200 None 2026-03-22 23:31:30,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:31:30,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd8cd0e93e23147242d69 HTTP/1.1" 200 None 2026-03-22 23:31:30,226 - lava_callback - ERROR - Error processing callback for node 69bdd8cd0e93e23147242d69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56918 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:32:30,497 - lava_callback - INFO - Received callback for node 69bdd9080e93e23147242d73 INFO: 172.20.0.1:50064 - "POST /node/69bdd9080e93e23147242d73 HTTP/1.0" 202 Accepted 2026-03-22 23:32:30,502 - lava_callback - INFO - Starting async job submit for node 69bdd9080e93e23147242d73 2026-03-22 23:32:30,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:32:30,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd9080e93e23147242d73 HTTP/1.1" 200 None 2026-03-22 23:32:30,518 - lava_callback - DEBUG - Processing node 69bdd9080e93e23147242d73: baseline-arm-baylibre 2026-03-22 23:32:30,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd9080e93e23147242d73 2026-03-22 23:32:30,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:32:30,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd9080e93e23147242d73 HTTP/1.1" 200 None 2026-03-22 23:32:30,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:32:30,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:32:30,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:32:30,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd9080e93e23147242d73 HTTP/1.1" 200 None 2026-03-22 23:32:30,574 - lava_callback - ERROR - Error processing callback for node 69bdd9080e93e23147242d73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55952 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:33:10,159 - lava_callback - INFO - Received callback for node 69bdd9320e93e23147242d8a INFO: 172.20.0.1:39524 - "POST /node/69bdd9320e93e23147242d8a HTTP/1.0" 202 Accepted 2026-03-22 23:33:10,164 - lava_callback - INFO - Starting async job submit for node 69bdd9320e93e23147242d8a 2026-03-22 23:33:10,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:33:10,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd9320e93e23147242d8a HTTP/1.1" 200 None 2026-03-22 23:33:10,180 - lava_callback - DEBUG - Processing node 69bdd9320e93e23147242d8a: baseline-arm-baylibre 2026-03-22 23:33:10,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd9320e93e23147242d8a 2026-03-22 23:33:10,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:33:10,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd9320e93e23147242d8a HTTP/1.1" 200 None 2026-03-22 23:33:10,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:33:10,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:33:10,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:33:10,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd9320e93e23147242d8a HTTP/1.1" 200 None 2026-03-22 23:33:10,253 - lava_callback - ERROR - Error processing callback for node 69bdd9320e93e23147242d8a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57070 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:33:50,386 - lava_callback - INFO - Received callback for node 69bdd95a0e93e23147242d97 INFO: 172.20.0.1:40134 - "POST /node/69bdd95a0e93e23147242d97 HTTP/1.0" 202 Accepted 2026-03-22 23:33:50,392 - lava_callback - INFO - Starting async job submit for node 69bdd95a0e93e23147242d97 2026-03-22 23:33:50,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:33:50,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd95a0e93e23147242d97 HTTP/1.1" 200 None 2026-03-22 23:33:50,411 - lava_callback - DEBUG - Processing node 69bdd95a0e93e23147242d97: baseline-arm-baylibre 2026-03-22 23:33:50,411 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd95a0e93e23147242d97 2026-03-22 23:33:50,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:33:50,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd95a0e93e23147242d97 HTTP/1.1" 200 None 2026-03-22 23:33:50,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:33:50,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:33:50,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:33:50,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd95a0e93e23147242d97 HTTP/1.1" 200 None 2026-03-22 23:33:50,472 - lava_callback - ERROR - Error processing callback for node 69bdd95a0e93e23147242d97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51830 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:34:30,265 - lava_callback - INFO - Received callback for node 69bdd9830e93e23147242da8 INFO: 172.20.0.1:47148 - "POST /node/69bdd9830e93e23147242da8 HTTP/1.0" 202 Accepted 2026-03-22 23:34:30,270 - lava_callback - INFO - Starting async job submit for node 69bdd9830e93e23147242da8 2026-03-22 23:34:30,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:34:30,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd9830e93e23147242da8 HTTP/1.1" 200 None 2026-03-22 23:34:30,299 - lava_callback - DEBUG - Processing node 69bdd9830e93e23147242da8: baseline-arm-baylibre 2026-03-22 23:34:30,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd9830e93e23147242da8 2026-03-22 23:34:30,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:34:30,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd9830e93e23147242da8 HTTP/1.1" 200 None 2026-03-22 23:34:30,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:34:30,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:34:30,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:34:30,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd9830e93e23147242da8 HTTP/1.1" 200 None 2026-03-22 23:34:30,363 - lava_callback - ERROR - Error processing callback for node 69bdd9830e93e23147242da8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44420 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:35:30,309 - lava_callback - INFO - Received callback for node 69bdd9af0e93e23147242db9 INFO: 172.20.0.1:50784 - "POST /node/69bdd9af0e93e23147242db9 HTTP/1.0" 202 Accepted 2026-03-22 23:35:30,315 - lava_callback - INFO - Starting async job submit for node 69bdd9af0e93e23147242db9 2026-03-22 23:35:30,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:35:30,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd9af0e93e23147242db9 HTTP/1.1" 200 None 2026-03-22 23:35:30,334 - lava_callback - DEBUG - Processing node 69bdd9af0e93e23147242db9: baseline-arm-clabbe 2026-03-22 23:35:30,334 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd9af0e93e23147242db9 2026-03-22 23:35:30,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:35:30,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd9af0e93e23147242db9 HTTP/1.1" 200 None 2026-03-22 23:35:30,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:35:30,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:35:30,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:35:30,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd9af0e93e23147242db9 HTTP/1.1" 200 None 2026-03-22 23:35:30,393 - lava_callback - ERROR - Error processing callback for node 69bdd9af0e93e23147242db9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47348 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:36:10,321 - lava_callback - INFO - Received callback for node 69bdd9db0e93e23147242dcb INFO: 172.20.0.1:40624 - "POST /node/69bdd9db0e93e23147242dcb HTTP/1.0" 202 Accepted 2026-03-22 23:36:10,326 - lava_callback - INFO - Starting async job submit for node 69bdd9db0e93e23147242dcb 2026-03-22 23:36:10,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:36:10,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd9db0e93e23147242dcb HTTP/1.1" 200 None 2026-03-22 23:36:10,343 - lava_callback - DEBUG - Processing node 69bdd9db0e93e23147242dcb: baseline-arm-clabbe 2026-03-22 23:36:10,343 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdd9db0e93e23147242dcb 2026-03-22 23:36:10,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:36:10,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdd9db0e93e23147242dcb HTTP/1.1" 200 None 2026-03-22 23:36:10,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:36:10,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:36:10,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:36:10,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdd9db0e93e23147242dcb HTTP/1.1" 200 None 2026-03-22 23:36:10,409 - lava_callback - ERROR - Error processing callback for node 69bdd9db0e93e23147242dcb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58614 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:36:50,407 - lava_callback - INFO - Received callback for node 69bdda040e93e23147242ddc INFO: 172.20.0.1:45594 - "POST /node/69bdda040e93e23147242ddc HTTP/1.0" 202 Accepted 2026-03-22 23:36:50,412 - lava_callback - INFO - Starting async job submit for node 69bdda040e93e23147242ddc 2026-03-22 23:36:50,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:36:50,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdda040e93e23147242ddc HTTP/1.1" 200 None 2026-03-22 23:36:50,428 - lava_callback - DEBUG - Processing node 69bdda040e93e23147242ddc: baseline-arm-clabbe 2026-03-22 23:36:50,428 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdda040e93e23147242ddc 2026-03-22 23:36:50,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:36:50,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdda040e93e23147242ddc HTTP/1.1" 200 None 2026-03-22 23:36:50,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:36:50,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:36:50,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:36:50,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdda040e93e23147242ddc HTTP/1.1" 200 None 2026-03-22 23:36:50,485 - lava_callback - ERROR - Error processing callback for node 69bdda040e93e23147242ddc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33566 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:37:30,391 - lava_callback - INFO - Received callback for node 69bdda2e0e93e23147242ded INFO: 172.20.0.1:44646 - "POST /node/69bdda2e0e93e23147242ded HTTP/1.0" 202 Accepted 2026-03-22 23:37:30,397 - lava_callback - INFO - Starting async job submit for node 69bdda2e0e93e23147242ded 2026-03-22 23:37:30,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:37:30,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdda2e0e93e23147242ded HTTP/1.1" 200 None 2026-03-22 23:37:30,415 - lava_callback - DEBUG - Processing node 69bdda2e0e93e23147242ded: baseline-arm-clabbe 2026-03-22 23:37:30,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdda2e0e93e23147242ded 2026-03-22 23:37:30,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:37:30,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdda2e0e93e23147242ded HTTP/1.1" 200 None 2026-03-22 23:37:30,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:37:30,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:37:30,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:37:30,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdda2e0e93e23147242ded HTTP/1.1" 200 None 2026-03-22 23:37:30,501 - lava_callback - ERROR - Error processing callback for node 69bdda2e0e93e23147242ded: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55704 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:39:30,478 - lava_callback - INFO - Received callback for node 69bddaaa0e93e23147242e2a INFO: 172.20.0.1:38162 - "POST /node/69bddaaa0e93e23147242e2a HTTP/1.0" 202 Accepted 2026-03-22 23:39:30,483 - lava_callback - INFO - Starting async job submit for node 69bddaaa0e93e23147242e2a 2026-03-22 23:39:30,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:39:30,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddaaa0e93e23147242e2a HTTP/1.1" 200 None 2026-03-22 23:39:30,523 - lava_callback - DEBUG - Processing node 69bddaaa0e93e23147242e2a: baseline-arm-clabbe 2026-03-22 23:39:30,524 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddaaa0e93e23147242e2a 2026-03-22 23:39:30,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:39:30,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddaaa0e93e23147242e2a HTTP/1.1" 200 None 2026-03-22 23:39:30,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:39:30,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:39:30,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:39:30,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddaaa0e93e23147242e2a HTTP/1.1" 200 None 2026-03-22 23:39:30,592 - lava_callback - ERROR - Error processing callback for node 69bddaaa0e93e23147242e2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39912 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:41:30,574 - lava_callback - INFO - Received callback for node 69bddb240e93e23147242e5b INFO: 172.20.0.1:54620 - "POST /node/69bddb240e93e23147242e5b HTTP/1.0" 202 Accepted 2026-03-22 23:41:30,579 - lava_callback - INFO - Starting async job submit for node 69bddb240e93e23147242e5b 2026-03-22 23:41:30,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:41:30,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddb240e93e23147242e5b HTTP/1.1" 200 None 2026-03-22 23:41:30,595 - lava_callback - DEBUG - Processing node 69bddb240e93e23147242e5b: baseline-arm-clabbe 2026-03-22 23:41:30,595 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddb240e93e23147242e5b 2026-03-22 23:41:30,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:41:30,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddb240e93e23147242e5b HTTP/1.1" 200 None 2026-03-22 23:41:30,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:41:30,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-22 23:41:30,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:41:30,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddb240e93e23147242e5b HTTP/1.1" 200 None 2026-03-22 23:41:30,671 - lava_callback - ERROR - Error processing callback for node 69bddb240e93e23147242e5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35960 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:42:10,553 - lava_callback - INFO - Received callback for node 69bddb4e0e93e23147242e6e INFO: 172.20.0.1:53816 - "POST /node/69bddb4e0e93e23147242e6e HTTP/1.0" 202 Accepted 2026-03-22 23:42:10,558 - lava_callback - INFO - Starting async job submit for node 69bddb4e0e93e23147242e6e 2026-03-22 23:42:10,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:42:10,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddb4e0e93e23147242e6e HTTP/1.1" 200 None 2026-03-22 23:42:10,574 - lava_callback - DEBUG - Processing node 69bddb4e0e93e23147242e6e: baseline-arm-clabbe 2026-03-22 23:42:10,574 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddb4e0e93e23147242e6e 2026-03-22 23:42:10,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:42:10,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddb4e0e93e23147242e6e HTTP/1.1" 200 None 2026-03-22 23:42:10,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:42:10,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-22 23:42:10,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:42:10,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddb4e0e93e23147242e6e HTTP/1.1" 200 None 2026-03-22 23:42:10,649 - lava_callback - ERROR - Error processing callback for node 69bddb4e0e93e23147242e6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48630 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:42:50,647 - lava_callback - INFO - Received callback for node 69bddb770e93e23147242e7f INFO: 172.20.0.1:45976 - "POST /node/69bddb770e93e23147242e7f HTTP/1.0" 202 Accepted 2026-03-22 23:42:50,655 - lava_callback - INFO - Starting async job submit for node 69bddb770e93e23147242e7f 2026-03-22 23:42:50,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:42:50,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddb770e93e23147242e7f HTTP/1.1" 200 None 2026-03-22 23:42:50,679 - lava_callback - DEBUG - Processing node 69bddb770e93e23147242e7f: baseline-arm-clabbe 2026-03-22 23:42:50,679 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddb770e93e23147242e7f 2026-03-22 23:42:50,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:42:50,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddb770e93e23147242e7f HTTP/1.1" 200 None 2026-03-22 23:42:50,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:42:50,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-22 23:42:50,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:42:50,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddb770e93e23147242e7f HTTP/1.1" 200 None 2026-03-22 23:42:50,738 - lava_callback - ERROR - Error processing callback for node 69bddb770e93e23147242e7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53790 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:43:50,669 - lava_callback - INFO - Received callback for node 69bddba10e93e23147242e99 INFO: 172.20.0.1:37516 - "POST /node/69bddba10e93e23147242e99 HTTP/1.0" 202 Accepted 2026-03-22 23:43:50,674 - lava_callback - INFO - Starting async job submit for node 69bddba10e93e23147242e99 2026-03-22 23:43:50,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:43:50,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddba10e93e23147242e99 HTTP/1.1" 200 None 2026-03-22 23:43:50,690 - lava_callback - DEBUG - Processing node 69bddba10e93e23147242e99: baseline-arm-clabbe 2026-03-22 23:43:50,690 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddba10e93e23147242e99 2026-03-22 23:43:50,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:43:50,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddba10e93e23147242e99 HTTP/1.1" 200 None 2026-03-22 23:43:50,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:43:50,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-22 23:43:50,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:43:50,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddba10e93e23147242e99 HTTP/1.1" 200 None 2026-03-22 23:43:50,754 - lava_callback - ERROR - Error processing callback for node 69bddba10e93e23147242e99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41362 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:44:30,664 - lava_callback - INFO - Received callback for node 69bddbca0e93e23147242ea1 INFO: 172.20.0.1:37944 - "POST /node/69bddbca0e93e23147242ea1 HTTP/1.0" 202 Accepted 2026-03-22 23:44:30,670 - lava_callback - INFO - Starting async job submit for node 69bddbca0e93e23147242ea1 2026-03-22 23:44:30,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:44:30,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddbca0e93e23147242ea1 HTTP/1.1" 200 None 2026-03-22 23:44:30,687 - lava_callback - DEBUG - Processing node 69bddbca0e93e23147242ea1: baseline-arm-clabbe 2026-03-22 23:44:30,687 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddbca0e93e23147242ea1 2026-03-22 23:44:30,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:44:30,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddbca0e93e23147242ea1 HTTP/1.1" 200 None 2026-03-22 23:44:30,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:44:30,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd732d0e93e2314723e2b5 HTTP/1.1" 200 None 2026-03-22 23:44:30,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:44:30,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddbca0e93e23147242ea1 HTTP/1.1" 200 None 2026-03-22 23:44:30,740 - lava_callback - ERROR - Error processing callback for node 69bddbca0e93e23147242ea1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54396 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:45:10,749 - lava_callback - INFO - Received callback for node 69bddbf30e93e23147242eb2 INFO: 172.20.0.1:39350 - "POST /node/69bddbf30e93e23147242eb2 HTTP/1.0" 202 Accepted 2026-03-22 23:45:10,755 - lava_callback - INFO - Starting async job submit for node 69bddbf30e93e23147242eb2 2026-03-22 23:45:10,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:45:10,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddbf30e93e23147242eb2 HTTP/1.1" 200 None 2026-03-22 23:45:10,771 - lava_callback - DEBUG - Processing node 69bddbf30e93e23147242eb2: baseline-arm-clabbe 2026-03-22 23:45:10,772 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddbf30e93e23147242eb2 2026-03-22 23:45:10,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:45:10,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddbf30e93e23147242eb2 HTTP/1.1" 200 None 2026-03-22 23:45:10,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:45:10,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:45:10,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:45:10,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddbf30e93e23147242eb2 HTTP/1.1" 200 None 2026-03-22 23:45:10,829 - lava_callback - ERROR - Error processing callback for node 69bddbf30e93e23147242eb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58968 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:45:50,734 - lava_callback - INFO - Received callback for node 69bddc1c0e93e23147242ec7 INFO: 172.20.0.1:34320 - "POST /node/69bddc1c0e93e23147242ec7 HTTP/1.0" 202 Accepted 2026-03-22 23:45:50,740 - lava_callback - INFO - Starting async job submit for node 69bddc1c0e93e23147242ec7 2026-03-22 23:45:50,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:45:50,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddc1c0e93e23147242ec7 HTTP/1.1" 200 None 2026-03-22 23:45:50,758 - lava_callback - DEBUG - Processing node 69bddc1c0e93e23147242ec7: baseline-arm-clabbe 2026-03-22 23:45:50,759 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddc1c0e93e23147242ec7 2026-03-22 23:45:50,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:45:50,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddc1c0e93e23147242ec7 HTTP/1.1" 200 None 2026-03-22 23:45:50,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:45:50,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:45:50,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:45:50,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddc1c0e93e23147242ec7 HTTP/1.1" 200 None 2026-03-22 23:45:50,810 - lava_callback - ERROR - Error processing callback for node 69bddc1c0e93e23147242ec7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45538 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:46:30,767 - lava_callback - INFO - Received callback for node 69bddc450e93e23147242ed2 INFO: 172.20.0.1:51136 - "POST /node/69bddc450e93e23147242ed2 HTTP/1.0" 202 Accepted 2026-03-22 23:46:30,773 - lava_callback - INFO - Starting async job submit for node 69bddc450e93e23147242ed2 2026-03-22 23:46:30,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:46:30,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddc450e93e23147242ed2 HTTP/1.1" 200 None 2026-03-22 23:46:30,789 - lava_callback - DEBUG - Processing node 69bddc450e93e23147242ed2: baseline-arm-clabbe 2026-03-22 23:46:30,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddc450e93e23147242ed2 2026-03-22 23:46:30,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:46:30,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddc450e93e23147242ed2 HTTP/1.1" 200 None 2026-03-22 23:46:30,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:46:30,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:46:30,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:46:30,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddc450e93e23147242ed2 HTTP/1.1" 200 None 2026-03-22 23:46:30,877 - lava_callback - ERROR - Error processing callback for node 69bddc450e93e23147242ed2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55684 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:47:10,866 - lava_callback - INFO - Received callback for node 69bddc6f0e93e23147242ee1 INFO: 172.20.0.1:46900 - "POST /node/69bddc6f0e93e23147242ee1 HTTP/1.0" 202 Accepted 2026-03-22 23:47:10,872 - lava_callback - INFO - Starting async job submit for node 69bddc6f0e93e23147242ee1 2026-03-22 23:47:10,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:47:10,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddc6f0e93e23147242ee1 HTTP/1.1" 200 None 2026-03-22 23:47:10,887 - lava_callback - DEBUG - Processing node 69bddc6f0e93e23147242ee1: baseline-arm-clabbe 2026-03-22 23:47:10,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddc6f0e93e23147242ee1 2026-03-22 23:47:10,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:47:10,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddc6f0e93e23147242ee1 HTTP/1.1" 200 None 2026-03-22 23:47:10,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:47:10,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:47:10,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:47:10,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddc6f0e93e23147242ee1 HTTP/1.1" 200 None 2026-03-22 23:47:10,950 - lava_callback - ERROR - Error processing callback for node 69bddc6f0e93e23147242ee1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60314 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:47:50,854 - lava_callback - INFO - Received callback for node 69bddc980e93e23147242ef3 INFO: 172.20.0.1:60562 - "POST /node/69bddc980e93e23147242ef3 HTTP/1.0" 202 Accepted 2026-03-22 23:47:50,860 - lava_callback - INFO - Starting async job submit for node 69bddc980e93e23147242ef3 2026-03-22 23:47:50,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:47:50,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddc980e93e23147242ef3 HTTP/1.1" 200 None 2026-03-22 23:47:50,875 - lava_callback - DEBUG - Processing node 69bddc980e93e23147242ef3: baseline-arm-clabbe 2026-03-22 23:47:50,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddc980e93e23147242ef3 2026-03-22 23:47:50,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:47:50,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddc980e93e23147242ef3 HTTP/1.1" 200 None 2026-03-22 23:47:50,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:47:50,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d330b8ad4124cec3553 HTTP/1.1" 200 None 2026-03-22 23:47:50,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:47:50,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddc980e93e23147242ef3 HTTP/1.1" 200 None 2026-03-22 23:47:50,931 - lava_callback - ERROR - Error processing callback for node 69bddc980e93e23147242ef3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42048 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:48:31,120 - lava_callback - INFO - Received callback for node 69bddcc60e93e23147242f03 INFO: 172.20.0.1:46052 - "POST /node/69bddcc60e93e23147242f03 HTTP/1.0" 202 Accepted 2026-03-22 23:48:31,127 - lava_callback - INFO - Starting async job submit for node 69bddcc60e93e23147242f03 2026-03-22 23:48:31,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:48:31,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddcc60e93e23147242f03 HTTP/1.1" 200 None 2026-03-22 23:48:31,144 - lava_callback - DEBUG - Processing node 69bddcc60e93e23147242f03: baseline-arm64-baylibre 2026-03-22 23:48:31,145 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddcc60e93e23147242f03 2026-03-22 23:48:31,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:48:31,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddcc60e93e23147242f03 HTTP/1.1" 200 None 2026-03-22 23:48:31,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:48:31,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 23:48:31,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:48:31,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddcc60e93e23147242f03 HTTP/1.1" 200 None 2026-03-22 23:48:31,211 - lava_callback - ERROR - Error processing callback for node 69bddcc60e93e23147242f03: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44122 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:49:51,236 - lava_callback - INFO - Received callback for node 69bddd150e93e23147242f10 INFO: 172.20.0.1:45944 - "POST /node/69bddd150e93e23147242f10 HTTP/1.0" 202 Accepted 2026-03-22 23:49:51,241 - lava_callback - INFO - Starting async job submit for node 69bddd150e93e23147242f10 2026-03-22 23:49:51,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:49:51,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddd150e93e23147242f10 HTTP/1.1" 200 None 2026-03-22 23:49:51,257 - lava_callback - DEBUG - Processing node 69bddd150e93e23147242f10: baseline-arm64-baylibre 2026-03-22 23:49:51,257 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddd150e93e23147242f10 2026-03-22 23:49:51,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:49:51,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddd150e93e23147242f10 HTTP/1.1" 200 None 2026-03-22 23:49:51,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:49:51,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 23:49:51,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:49:51,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddd150e93e23147242f10 HTTP/1.1" 200 None 2026-03-22 23:49:51,308 - lava_callback - ERROR - Error processing callback for node 69bddd150e93e23147242f10: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43312 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:51:11,216 - lava_callback - INFO - Received callback for node 69bddd650e93e23147242f1d INFO: 172.20.0.1:52450 - "POST /node/69bddd650e93e23147242f1d HTTP/1.0" 202 Accepted 2026-03-22 23:51:11,222 - lava_callback - INFO - Starting async job submit for node 69bddd650e93e23147242f1d 2026-03-22 23:51:11,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:51:11,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddd650e93e23147242f1d HTTP/1.1" 200 None 2026-03-22 23:51:11,238 - lava_callback - DEBUG - Processing node 69bddd650e93e23147242f1d: baseline-arm64-baylibre 2026-03-22 23:51:11,239 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddd650e93e23147242f1d 2026-03-22 23:51:11,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:51:11,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddd650e93e23147242f1d HTTP/1.1" 200 None 2026-03-22 23:51:11,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:51:11,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 23:51:11,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:51:11,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddd650e93e23147242f1d HTTP/1.1" 200 None 2026-03-22 23:51:11,306 - lava_callback - ERROR - Error processing callback for node 69bddd650e93e23147242f1d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47998 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:52:31,531 - lava_callback - INFO - Received callback for node 69bdddb50e93e23147242f27 INFO: 172.20.0.1:37252 - "POST /node/69bdddb50e93e23147242f27 HTTP/1.0" 202 Accepted 2026-03-22 23:52:31,536 - lava_callback - INFO - Starting async job submit for node 69bdddb50e93e23147242f27 2026-03-22 23:52:31,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:52:31,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdddb50e93e23147242f27 HTTP/1.1" 200 None 2026-03-22 23:52:31,552 - lava_callback - DEBUG - Processing node 69bdddb50e93e23147242f27: baseline-arm64-baylibre 2026-03-22 23:52:31,552 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdddb50e93e23147242f27 2026-03-22 23:52:31,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:52:31,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdddb50e93e23147242f27 HTTP/1.1" 200 None 2026-03-22 23:52:31,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:52:31,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 23:52:31,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:52:31,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdddb50e93e23147242f27 HTTP/1.1" 200 None 2026-03-22 23:52:31,613 - lava_callback - ERROR - Error processing callback for node 69bdddb50e93e23147242f27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50676 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:53:51,312 - lava_callback - INFO - Received callback for node 69bdde060e93e23147242f34 INFO: 172.20.0.1:50076 - "POST /node/69bdde060e93e23147242f34 HTTP/1.0" 202 Accepted 2026-03-22 23:53:51,317 - lava_callback - INFO - Starting async job submit for node 69bdde060e93e23147242f34 2026-03-22 23:53:51,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:53:51,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdde060e93e23147242f34 HTTP/1.1" 200 None 2026-03-22 23:53:51,333 - lava_callback - DEBUG - Processing node 69bdde060e93e23147242f34: baseline-arm64-baylibre 2026-03-22 23:53:51,333 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdde060e93e23147242f34 2026-03-22 23:53:51,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:53:51,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdde060e93e23147242f34 HTTP/1.1" 200 None 2026-03-22 23:53:51,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:53:51,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-22 23:53:51,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:53:51,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdde060e93e23147242f34 HTTP/1.1" 200 None 2026-03-22 23:53:51,392 - lava_callback - ERROR - Error processing callback for node 69bdde060e93e23147242f34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39162 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:55:11,475 - lava_callback - INFO - Received callback for node 69bdde520e93e23147242f3d INFO: 172.20.0.1:56360 - "POST /node/69bdde520e93e23147242f3d HTTP/1.0" 202 Accepted 2026-03-22 23:55:11,480 - lava_callback - INFO - Starting async job submit for node 69bdde520e93e23147242f3d 2026-03-22 23:55:11,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:55:11,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdde520e93e23147242f3d HTTP/1.1" 200 None 2026-03-22 23:55:11,496 - lava_callback - DEBUG - Processing node 69bdde520e93e23147242f3d: baseline-arm64-baylibre 2026-03-22 23:55:11,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bdde520e93e23147242f3d 2026-03-22 23:55:11,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:55:11,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bdde520e93e23147242f3d HTTP/1.1" 200 None 2026-03-22 23:55:11,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:55:11,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 23:55:11,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:55:11,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bdde520e93e23147242f3d HTTP/1.1" 200 None 2026-03-22 23:55:11,555 - lava_callback - ERROR - Error processing callback for node 69bdde520e93e23147242f3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47966 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:48932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51604 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:57:51,677 - lava_callback - INFO - Received callback for node 69bddef10e93e23147242f53 INFO: 172.20.0.1:39682 - "POST /node/69bddef10e93e23147242f53 HTTP/1.0" 202 Accepted 2026-03-22 23:57:51,683 - lava_callback - INFO - Starting async job submit for node 69bddef10e93e23147242f53 2026-03-22 23:57:51,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:57:51,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddef10e93e23147242f53 HTTP/1.1" 200 None 2026-03-22 23:57:51,699 - lava_callback - DEBUG - Processing node 69bddef10e93e23147242f53: baseline-arm64-baylibre 2026-03-22 23:57:51,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddef10e93e23147242f53 2026-03-22 23:57:51,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:57:51,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddef10e93e23147242f53 HTTP/1.1" 200 None 2026-03-22 23:57:51,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:57:51,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 23:57:51,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:57:51,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddef10e93e23147242f53 HTTP/1.1" 200 None 2026-03-22 23:57:51,750 - lava_callback - ERROR - Error processing callback for node 69bddef10e93e23147242f53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET / HTTP/1.0" 200 OK 2026-03-22 23:59:11,726 - lava_callback - INFO - Received callback for node 69bddf410e93e23147242f5b INFO: 172.20.0.1:39268 - "POST /node/69bddf410e93e23147242f5b HTTP/1.0" 202 Accepted 2026-03-22 23:59:11,732 - lava_callback - INFO - Starting async job submit for node 69bddf410e93e23147242f5b 2026-03-22 23:59:11,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:59:11,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddf410e93e23147242f5b HTTP/1.1" 200 None 2026-03-22 23:59:11,748 - lava_callback - DEBUG - Processing node 69bddf410e93e23147242f5b: baseline-arm64-baylibre 2026-03-22 23:59:11,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddf410e93e23147242f5b 2026-03-22 23:59:11,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:59:11,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddf410e93e23147242f5b HTTP/1.1" 200 None 2026-03-22 23:59:11,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:59:11,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-22 23:59:11,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-22 23:59:11,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddf410e93e23147242f5b HTTP/1.1" 200 None 2026-03-22 23:59:11,815 - lava_callback - ERROR - Error processing callback for node 69bddf410e93e23147242f5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42072 - "GET / HTTP/1.0" 200 OK 2026-03-23 00:00:31,765 - lava_callback - INFO - Received callback for node 69bddf970e93e23147242f68 INFO: 172.20.0.1:44670 - "POST /node/69bddf970e93e23147242f68 HTTP/1.0" 202 Accepted 2026-03-23 00:00:31,773 - lava_callback - INFO - Starting async job submit for node 69bddf970e93e23147242f68 2026-03-23 00:00:31,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:00:31,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddf970e93e23147242f68 HTTP/1.1" 200 None 2026-03-23 00:00:31,803 - lava_callback - DEBUG - Processing node 69bddf970e93e23147242f68: baseline-arm64-baylibre 2026-03-23 00:00:31,803 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bddf970e93e23147242f68 2026-03-23 00:00:31,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:00:31,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bddf970e93e23147242f68 HTTP/1.1" 200 None 2026-03-23 00:00:31,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:00:31,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-23 00:00:31,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:00:31,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bddf970e93e23147242f68 HTTP/1.1" 200 None 2026-03-23 00:00:31,893 - lava_callback - ERROR - Error processing callback for node 69bddf970e93e23147242f68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43758 - "GET / HTTP/1.0" 200 OK 2026-03-23 00:03:11,873 - lava_callback - INFO - Received callback for node 69bde0380e93e23147242f81 INFO: 172.20.0.1:53378 - "POST /node/69bde0380e93e23147242f81 HTTP/1.0" 202 Accepted 2026-03-23 00:03:11,879 - lava_callback - INFO - Starting async job submit for node 69bde0380e93e23147242f81 2026-03-23 00:03:11,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:03:11,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde0380e93e23147242f81 HTTP/1.1" 200 None 2026-03-23 00:03:11,896 - lava_callback - DEBUG - Processing node 69bde0380e93e23147242f81: baseline-arm64-baylibre 2026-03-23 00:03:11,897 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bde0380e93e23147242f81 2026-03-23 00:03:11,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:03:11,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde0380e93e23147242f81 HTTP/1.1" 200 None 2026-03-23 00:03:11,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:03:11,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b82d510b8ad4124cec3566 HTTP/1.1" 200 None 2026-03-23 00:03:11,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:03:11,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde0380e93e23147242f81 HTTP/1.1" 200 None 2026-03-23 00:03:11,963 - lava_callback - ERROR - Error processing callback for node 69bde0380e93e23147242f81: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58624 - "GET / HTTP/1.0" 200 OK 2026-03-23 00:04:32,024 - lava_callback - INFO - Received callback for node 69bde08a0e93e23147242f89 INFO: 172.20.0.1:35332 - "POST /node/69bde08a0e93e23147242f89 HTTP/1.0" 202 Accepted 2026-03-23 00:04:32,030 - lava_callback - INFO - Starting async job submit for node 69bde08a0e93e23147242f89 2026-03-23 00:04:32,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:04:32,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde08a0e93e23147242f89 HTTP/1.1" 200 None 2026-03-23 00:04:32,046 - lava_callback - DEBUG - Processing node 69bde08a0e93e23147242f89: baseline-arm64-baylibre 2026-03-23 00:04:32,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bde08a0e93e23147242f89 2026-03-23 00:04:32,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:04:32,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde08a0e93e23147242f89 HTTP/1.1" 200 None 2026-03-23 00:04:32,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:04:32,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-23 00:04:32,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:04:32,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde08a0e93e23147242f89 HTTP/1.1" 200 None 2026-03-23 00:04:32,106 - lava_callback - ERROR - Error processing callback for node 69bde08a0e93e23147242f89: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36638 - "GET / HTTP/1.0" 200 OK 2026-03-23 00:05:52,054 - lava_callback - INFO - Received callback for node 69bde0dc0e93e23147242f98 INFO: 172.20.0.1:40170 - "POST /node/69bde0dc0e93e23147242f98 HTTP/1.0" 202 Accepted 2026-03-23 00:05:52,059 - lava_callback - INFO - Starting async job submit for node 69bde0dc0e93e23147242f98 2026-03-23 00:05:52,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:05:52,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde0dc0e93e23147242f98 HTTP/1.1" 200 None 2026-03-23 00:05:52,075 - lava_callback - DEBUG - Processing node 69bde0dc0e93e23147242f98: baseline-arm64-baylibre 2026-03-23 00:05:52,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bde0dc0e93e23147242f98 2026-03-23 00:05:52,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:05:52,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde0dc0e93e23147242f98 HTTP/1.1" 200 None 2026-03-23 00:05:52,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:05:52,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-23 00:05:52,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:05:52,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde0dc0e93e23147242f98 HTTP/1.1" 200 None 2026-03-23 00:05:52,138 - lava_callback - ERROR - Error processing callback for node 69bde0dc0e93e23147242f98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57800 - "GET / HTTP/1.0" 200 OK 2026-03-23 00:07:12,084 - lava_callback - INFO - Received callback for node 69bde12a0e93e23147242fa6 INFO: 172.20.0.1:35924 - "POST /node/69bde12a0e93e23147242fa6 HTTP/1.0" 202 Accepted 2026-03-23 00:07:12,090 - lava_callback - INFO - Starting async job submit for node 69bde12a0e93e23147242fa6 2026-03-23 00:07:12,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:07:12,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde12a0e93e23147242fa6 HTTP/1.1" 200 None 2026-03-23 00:07:12,105 - lava_callback - DEBUG - Processing node 69bde12a0e93e23147242fa6: baseline-arm64-baylibre 2026-03-23 00:07:12,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bde12a0e93e23147242fa6 2026-03-23 00:07:12,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:07:12,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde12a0e93e23147242fa6 HTTP/1.1" 200 None 2026-03-23 00:07:12,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:07:12,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-23 00:07:12,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:07:12,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde12a0e93e23147242fa6 HTTP/1.1" 200 None 2026-03-23 00:07:12,165 - lava_callback - ERROR - Error processing callback for node 69bde12a0e93e23147242fa6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45484 - "GET / HTTP/1.0" 200 OK 2026-03-23 00:08:32,206 - lava_callback - INFO - Received callback for node 69bde1790e93e23147242fac INFO: 172.20.0.1:57490 - "POST /node/69bde1790e93e23147242fac HTTP/1.0" 202 Accepted 2026-03-23 00:08:32,212 - lava_callback - INFO - Starting async job submit for node 69bde1790e93e23147242fac 2026-03-23 00:08:32,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:08:32,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde1790e93e23147242fac HTTP/1.1" 200 None 2026-03-23 00:08:32,240 - lava_callback - DEBUG - Processing node 69bde1790e93e23147242fac: baseline-arm64-baylibre 2026-03-23 00:08:32,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bde1790e93e23147242fac 2026-03-23 00:08:32,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:08:32,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde1790e93e23147242fac HTTP/1.1" 200 None 2026-03-23 00:08:32,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:08:32,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-23 00:08:32,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:08:32,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde1790e93e23147242fac HTTP/1.1" 200 None 2026-03-23 00:08:32,295 - lava_callback - ERROR - Error processing callback for node 69bde1790e93e23147242fac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53352 - "GET / HTTP/1.0" 200 OK 2026-03-23 00:09:52,623 - lava_callback - INFO - Received callback for node 69bde1ca0e93e23147242fb9 INFO: 172.20.0.1:54424 - "POST /node/69bde1ca0e93e23147242fb9 HTTP/1.0" 202 Accepted 2026-03-23 00:09:52,628 - lava_callback - INFO - Starting async job submit for node 69bde1ca0e93e23147242fb9 2026-03-23 00:09:52,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:09:52,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde1ca0e93e23147242fb9 HTTP/1.1" 200 None 2026-03-23 00:09:52,647 - lava_callback - DEBUG - Processing node 69bde1ca0e93e23147242fb9: baseline-arm64-baylibre 2026-03-23 00:09:52,648 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bde1ca0e93e23147242fb9 2026-03-23 00:09:52,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:09:52,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde1ca0e93e23147242fb9 HTTP/1.1" 200 None 2026-03-23 00:09:52,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:09:52,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-23 00:09:52,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:09:52,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde1ca0e93e23147242fb9 HTTP/1.1" 200 None 2026-03-23 00:09:52,704 - lava_callback - ERROR - Error processing callback for node 69bde1ca0e93e23147242fb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35202 - "GET / HTTP/1.0" 200 OK 2026-03-23 00:11:31,916 - lava_callback - INFO - Received callback for node 69bde2220e93e23147242fc9 INFO: 172.20.0.1:57378 - "POST /node/69bde2220e93e23147242fc9 HTTP/1.0" 202 Accepted 2026-03-23 00:11:31,931 - lava_callback - INFO - Starting async job submit for node 69bde2220e93e23147242fc9 2026-03-23 00:11:31,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:11:31,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde2220e93e23147242fc9 HTTP/1.1" 200 None 2026-03-23 00:11:31,966 - lava_callback - DEBUG - Processing node 69bde2220e93e23147242fc9: baseline-arm64-clabbe 2026-03-23 00:11:31,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bde2220e93e23147242fc9 2026-03-23 00:11:31,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:11:31,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bde2220e93e23147242fc9 HTTP/1.1" 200 None 2026-03-23 00:11:31,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:11:32,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bad0a721a77312e2220ef1 HTTP/1.1" 200 None 2026-03-23 00:11:32,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-23 00:11:32,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bde2220e93e23147242fc9 HTTP/1.1" 200 None 2026-03-23 00:11:32,034 - lava_callback - ERROR - Error processing callback for node 69bde2220e93e23147242fc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54578 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]