INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33510 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:23:11,885 - lava_callback - INFO - Received callback for node 69afd4ecc9209b146d8e1daa 2026-03-12 08:23:11,891 - lava_callback - INFO - Starting async job submit for node 69afd4ecc9209b146d8e1daa INFO: 172.20.0.1:43900 - "POST /node/69afd4ecc9209b146d8e1daa HTTP/1.0" 202 Accepted 2026-03-12 08:23:11,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:23:11,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd4ecc9209b146d8e1daa HTTP/1.1" 200 None 2026-03-12 08:23:11,930 - lava_callback - DEBUG - Processing node 69afd4ecc9209b146d8e1daa: baseline-arm64-baylibre 2026-03-12 08:23:11,931 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd4ecc9209b146d8e1daa 2026-03-12 08:23:11,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:23:11,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd4ecc9209b146d8e1daa HTTP/1.1" 200 None 2026-03-12 08:23:11,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:23:11,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 08:23:11,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:23:12,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd4ecc9209b146d8e1daa HTTP/1.1" 200 None 2026-03-12 08:23:12,020 - lava_callback - ERROR - Error processing callback for node 69afd4ecc9209b146d8e1daa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52158 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:25:11,986 - lava_callback - INFO - Received callback for node 69afd560c9209b146d8e1db5 INFO: 172.20.0.1:49742 - "POST /node/69afd560c9209b146d8e1db5 HTTP/1.0" 202 Accepted 2026-03-12 08:25:11,991 - lava_callback - INFO - Starting async job submit for node 69afd560c9209b146d8e1db5 2026-03-12 08:25:11,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:25:12,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd560c9209b146d8e1db5 HTTP/1.1" 200 None 2026-03-12 08:25:12,008 - lava_callback - DEBUG - Processing node 69afd560c9209b146d8e1db5: baseline-arm64-baylibre 2026-03-12 08:25:12,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd560c9209b146d8e1db5 2026-03-12 08:25:12,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:25:12,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd560c9209b146d8e1db5 HTTP/1.1" 200 None 2026-03-12 08:25:12,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:25:12,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 08:25:12,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:25:12,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd560c9209b146d8e1db5 HTTP/1.1" 200 None 2026-03-12 08:25:12,081 - lava_callback - ERROR - Error processing callback for node 69afd560c9209b146d8e1db5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39448 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:26:32,305 - lava_callback - INFO - Received callback for node 69afd5afc9209b146d8e1f8b INFO: 172.20.0.1:41934 - "POST /node/69afd5afc9209b146d8e1f8b HTTP/1.0" 202 Accepted 2026-03-12 08:26:32,311 - lava_callback - INFO - Starting async job submit for node 69afd5afc9209b146d8e1f8b 2026-03-12 08:26:32,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:26:32,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd5afc9209b146d8e1f8b HTTP/1.1" 200 None 2026-03-12 08:26:32,327 - lava_callback - DEBUG - Processing node 69afd5afc9209b146d8e1f8b: baseline-arm64-baylibre 2026-03-12 08:26:32,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd5afc9209b146d8e1f8b 2026-03-12 08:26:32,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:26:32,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd5afc9209b146d8e1f8b HTTP/1.1" 200 None 2026-03-12 08:26:32,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:26:32,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 08:26:32,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:26:32,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd5afc9209b146d8e1f8b HTTP/1.1" 200 None 2026-03-12 08:26:32,415 - lava_callback - ERROR - Error processing callback for node 69afd5afc9209b146d8e1f8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55082 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:27:52,089 - lava_callback - INFO - Received callback for node 69afd5fbc9209b146d8e1f94 INFO: 172.20.0.1:40366 - "POST /node/69afd5fbc9209b146d8e1f94 HTTP/1.0" 202 Accepted 2026-03-12 08:27:52,094 - lava_callback - INFO - Starting async job submit for node 69afd5fbc9209b146d8e1f94 2026-03-12 08:27:52,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:27:52,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd5fbc9209b146d8e1f94 HTTP/1.1" 200 None 2026-03-12 08:27:52,116 - lava_callback - DEBUG - Processing node 69afd5fbc9209b146d8e1f94: baseline-arm64-baylibre 2026-03-12 08:27:52,116 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd5fbc9209b146d8e1f94 2026-03-12 08:27:52,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:27:52,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd5fbc9209b146d8e1f94 HTTP/1.1" 200 None 2026-03-12 08:27:52,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:27:52,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 08:27:52,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:27:52,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd5fbc9209b146d8e1f94 HTTP/1.1" 200 None 2026-03-12 08:27:52,184 - lava_callback - ERROR - Error processing callback for node 69afd5fbc9209b146d8e1f94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54936 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:29:21,835 - lava_callback - INFO - Received callback for node 69afd659c9209b146d8e1fa2 INFO: 172.20.0.1:46226 - "POST /node/69afd659c9209b146d8e1fa2 HTTP/1.0" 202 Accepted 2026-03-12 08:29:21,840 - lava_callback - INFO - Starting async job submit for node 69afd659c9209b146d8e1fa2 2026-03-12 08:29:21,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:29:21,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd659c9209b146d8e1fa2 HTTP/1.1" 200 None 2026-03-12 08:29:21,867 - lava_callback - DEBUG - Processing node 69afd659c9209b146d8e1fa2: baseline-arm64-clabbe 2026-03-12 08:29:21,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd659c9209b146d8e1fa2 2026-03-12 08:29:21,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:29:21,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd659c9209b146d8e1fa2 HTTP/1.1" 200 None 2026-03-12 08:29:21,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:29:21,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 08:29:21,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:29:21,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd659c9209b146d8e1fa2 HTTP/1.1" 200 None 2026-03-12 08:29:21,927 - lava_callback - ERROR - Error processing callback for node 69afd659c9209b146d8e1fa2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59456 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:30:41,835 - lava_callback - INFO - Received callback for node 69afd6a8c9209b146d8e1fb0 INFO: 172.20.0.1:38156 - "POST /node/69afd6a8c9209b146d8e1fb0 HTTP/1.0" 202 Accepted 2026-03-12 08:30:41,840 - lava_callback - INFO - Starting async job submit for node 69afd6a8c9209b146d8e1fb0 2026-03-12 08:30:41,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:30:41,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd6a8c9209b146d8e1fb0 HTTP/1.1" 200 None 2026-03-12 08:30:41,856 - lava_callback - DEBUG - Processing node 69afd6a8c9209b146d8e1fb0: baseline-arm64-clabbe 2026-03-12 08:30:41,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd6a8c9209b146d8e1fb0 2026-03-12 08:30:41,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:30:41,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd6a8c9209b146d8e1fb0 HTTP/1.1" 200 None 2026-03-12 08:30:41,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:30:41,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 08:30:41,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:30:41,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd6a8c9209b146d8e1fb0 HTTP/1.1" 200 None 2026-03-12 08:30:41,917 - lava_callback - ERROR - Error processing callback for node 69afd6a8c9209b146d8e1fb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50358 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:31:06,464 - lava_callback - INFO - Received callback for node 69b279494bce73bcf0233d9d INFO: 172.20.0.1:47546 - "POST /node/69b279494bce73bcf0233d9d HTTP/1.0" 202 Accepted 2026-03-12 08:31:06,470 - lava_callback - INFO - Starting async job submit for node 69b279494bce73bcf0233d9d 2026-03-12 08:31:06,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:31:06,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b279494bce73bcf0233d9d HTTP/1.1" 200 None 2026-03-12 08:31:06,524 - lava_callback - DEBUG - Processing node 69b279494bce73bcf0233d9d: baseline-arm INFO: 172.20.0.1:47560 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:31:06,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:31:06,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:31:06,903 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b279494bce73bcf0233d9d/log.txt.gz 2026-03-12 08:31:07,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:31:07,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:31:07,201 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b279494bce73bcf0233d9d/lava-logs.yaml 2026-03-12 08:31:07,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:31:07,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:31:07,378 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b279494bce73bcf0233d9d/lava_callback.json.gz 2026-03-12 08:31:07,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:31:07,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b279494bce73bcf0233d9d HTTP/1.1" 200 None 2026-03-12 08:31:07,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:31:07,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2766c4bce73bcf0233d1a HTTP/1.1" 200 None 2026-03-12 08:31:07,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:31:07,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b279494bce73bcf0233d9d HTTP/1.1" 200 None 2026-03-12 08:31:07,459 - lava_callback - INFO - Completed processing callback for node 69b279494bce73bcf0233d9d 2026-03-12 08:31:12,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:31:12,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36268 - "GET /login HTTP/1.0" 404 Not Found INFO: 172.20.0.1:36280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41734 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:40:32,988 - lava_callback - INFO - Received callback for node 69afd8f4ae557f2600f0f07e INFO: 172.20.0.1:44444 - "POST /node/69afd8f4ae557f2600f0f07e HTTP/1.0" 202 Accepted 2026-03-12 08:40:32,993 - lava_callback - INFO - Starting async job submit for node 69afd8f4ae557f2600f0f07e 2026-03-12 08:40:32,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:40:33,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd8f4ae557f2600f0f07e HTTP/1.1" 200 None 2026-03-12 08:40:33,011 - lava_callback - DEBUG - Processing node 69afd8f4ae557f2600f0f07e: baseline-arm-baylibre 2026-03-12 08:40:33,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd8f4ae557f2600f0f07e 2026-03-12 08:40:33,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:40:33,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd8f4ae557f2600f0f07e HTTP/1.1" 200 None 2026-03-12 08:40:33,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:40:33,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 08:40:33,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:40:33,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd8f4ae557f2600f0f07e HTTP/1.1" 200 None 2026-03-12 08:40:33,075 - lava_callback - ERROR - Error processing callback for node 69afd8f4ae557f2600f0f07e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39038 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:41:12,820 - lava_callback - INFO - Received callback for node 69afd91cae557f2600f0f089 INFO: 172.20.0.1:60384 - "POST /node/69afd91cae557f2600f0f089 HTTP/1.0" 202 Accepted 2026-03-12 08:41:12,826 - lava_callback - INFO - Starting async job submit for node 69afd91cae557f2600f0f089 2026-03-12 08:41:12,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:41:12,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd91cae557f2600f0f089 HTTP/1.1" 200 None 2026-03-12 08:41:12,845 - lava_callback - DEBUG - Processing node 69afd91cae557f2600f0f089: baseline-arm-baylibre 2026-03-12 08:41:12,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd91cae557f2600f0f089 2026-03-12 08:41:12,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:41:12,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd91cae557f2600f0f089 HTTP/1.1" 200 None 2026-03-12 08:41:12,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:41:12,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 08:41:12,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:41:12,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd91cae557f2600f0f089 HTTP/1.1" 200 None 2026-03-12 08:41:12,899 - lava_callback - ERROR - Error processing callback for node 69afd91cae557f2600f0f089: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49608 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:41:52,886 - lava_callback - INFO - Received callback for node 69afd944ae557f2600f0f094 INFO: 172.20.0.1:52296 - "POST /node/69afd944ae557f2600f0f094 HTTP/1.0" 202 Accepted 2026-03-12 08:41:52,891 - lava_callback - INFO - Starting async job submit for node 69afd944ae557f2600f0f094 2026-03-12 08:41:52,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:41:52,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd944ae557f2600f0f094 HTTP/1.1" 200 None 2026-03-12 08:41:52,908 - lava_callback - DEBUG - Processing node 69afd944ae557f2600f0f094: baseline-arm-baylibre 2026-03-12 08:41:52,908 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd944ae557f2600f0f094 2026-03-12 08:41:52,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:41:52,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd944ae557f2600f0f094 HTTP/1.1" 200 None 2026-03-12 08:41:52,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:41:52,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 08:41:52,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:41:52,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd944ae557f2600f0f094 HTTP/1.1" 200 None 2026-03-12 08:41:52,969 - lava_callback - ERROR - Error processing callback for node 69afd944ae557f2600f0f094: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46122 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:42:32,852 - lava_callback - INFO - Received callback for node 69afd969ae557f2600f0f0a3 INFO: 172.20.0.1:49918 - "POST /node/69afd969ae557f2600f0f0a3 HTTP/1.0" 202 Accepted 2026-03-12 08:42:32,857 - lava_callback - INFO - Starting async job submit for node 69afd969ae557f2600f0f0a3 2026-03-12 08:42:32,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:42:32,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd969ae557f2600f0f0a3 HTTP/1.1" 200 None 2026-03-12 08:42:32,873 - lava_callback - DEBUG - Processing node 69afd969ae557f2600f0f0a3: baseline-arm-baylibre 2026-03-12 08:42:32,873 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd969ae557f2600f0f0a3 2026-03-12 08:42:32,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:42:32,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd969ae557f2600f0f0a3 HTTP/1.1" 200 None 2026-03-12 08:42:32,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:42:32,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 08:42:32,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:42:32,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd969ae557f2600f0f0a3 HTTP/1.1" 200 None 2026-03-12 08:42:32,930 - lava_callback - ERROR - Error processing callback for node 69afd969ae557f2600f0f0a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55466 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:43:13,016 - lava_callback - INFO - Received callback for node 69afd98fae557f2600f0f0ad INFO: 172.20.0.1:44460 - "POST /node/69afd98fae557f2600f0f0ad HTTP/1.0" 202 Accepted 2026-03-12 08:43:13,024 - lava_callback - INFO - Starting async job submit for node 69afd98fae557f2600f0f0ad 2026-03-12 08:43:13,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:43:13,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd98fae557f2600f0f0ad HTTP/1.1" 200 None 2026-03-12 08:43:13,049 - lava_callback - DEBUG - Processing node 69afd98fae557f2600f0f0ad: baseline-arm-baylibre 2026-03-12 08:43:13,049 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd98fae557f2600f0f0ad 2026-03-12 08:43:13,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:43:13,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd98fae557f2600f0f0ad HTTP/1.1" 200 None 2026-03-12 08:43:13,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:43:13,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 08:43:13,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:43:13,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd98fae557f2600f0f0ad HTTP/1.1" 200 None 2026-03-12 08:43:13,149 - lava_callback - ERROR - Error processing callback for node 69afd98fae557f2600f0f0ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39114 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:43:42,383 - lava_callback - INFO - Received callback for node 69afd9b5ae557f2600f0f45a INFO: 172.20.0.1:35102 - "POST /node/69afd9b5ae557f2600f0f45a HTTP/1.0" 202 Accepted 2026-03-12 08:43:42,394 - lava_callback - INFO - Starting async job submit for node 69afd9b5ae557f2600f0f45a 2026-03-12 08:43:42,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:43:42,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd9b5ae557f2600f0f45a HTTP/1.1" 200 None 2026-03-12 08:43:42,423 - lava_callback - DEBUG - Processing node 69afd9b5ae557f2600f0f45a: baseline-arm-clabbe 2026-03-12 08:43:42,423 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd9b5ae557f2600f0f45a 2026-03-12 08:43:42,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:43:42,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd9b5ae557f2600f0f45a HTTP/1.1" 200 None 2026-03-12 08:43:42,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:43:42,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 08:43:42,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:43:42,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd9b5ae557f2600f0f45a HTTP/1.1" 200 None 2026-03-12 08:43:42,519 - lava_callback - ERROR - Error processing callback for node 69afd9b5ae557f2600f0f45a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57478 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:44:22,352 - lava_callback - INFO - Received callback for node 69afd9dbae557f2600f0f465 INFO: 172.20.0.1:47064 - "POST /node/69afd9dbae557f2600f0f465 HTTP/1.0" 202 Accepted 2026-03-12 08:44:22,362 - lava_callback - INFO - Starting async job submit for node 69afd9dbae557f2600f0f465 2026-03-12 08:44:22,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:44:22,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd9dbae557f2600f0f465 HTTP/1.1" 200 None 2026-03-12 08:44:22,388 - lava_callback - DEBUG - Processing node 69afd9dbae557f2600f0f465: baseline-arm-clabbe 2026-03-12 08:44:22,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afd9dbae557f2600f0f465 2026-03-12 08:44:22,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:44:22,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd9dbae557f2600f0f465 HTTP/1.1" 200 None 2026-03-12 08:44:22,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:44:22,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 08:44:22,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:44:22,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afd9dbae557f2600f0f465 HTTP/1.1" 200 None 2026-03-12 08:44:22,443 - lava_callback - ERROR - Error processing callback for node 69afd9dbae557f2600f0f465: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55466 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:45:02,393 - lava_callback - INFO - Received callback for node 69afda01ae557f2600f0f478 INFO: 172.20.0.1:44812 - "POST /node/69afda01ae557f2600f0f478 HTTP/1.0" 202 Accepted 2026-03-12 08:45:02,402 - lava_callback - INFO - Starting async job submit for node 69afda01ae557f2600f0f478 2026-03-12 08:45:02,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:02,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afda01ae557f2600f0f478 HTTP/1.1" 200 None 2026-03-12 08:45:02,417 - lava_callback - DEBUG - Processing node 69afda01ae557f2600f0f478: baseline-arm-clabbe 2026-03-12 08:45:02,418 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afda01ae557f2600f0f478 2026-03-12 08:45:02,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:02,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afda01ae557f2600f0f478 HTTP/1.1" 200 None 2026-03-12 08:45:02,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:02,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 08:45:02,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:02,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afda01ae557f2600f0f478 HTTP/1.1" 200 None 2026-03-12 08:45:02,476 - lava_callback - ERROR - Error processing callback for node 69afda01ae557f2600f0f478: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:45:02,535 - lava_callback - INFO - Received callback for node 69b27d044bce73bcf02341e3 INFO: 172.20.0.1:44818 - "POST /node/69b27d044bce73bcf02341e3 HTTP/1.0" 202 Accepted 2026-03-12 08:45:02,547 - lava_callback - INFO - Starting async job submit for node 69b27d044bce73bcf02341e3 2026-03-12 08:45:02,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:02,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d044bce73bcf02341e3 HTTP/1.1" 200 None 2026-03-12 08:45:02,579 - lava_callback - DEBUG - Processing node 69b27d044bce73bcf02341e3: baseline-arm 2026-03-12 08:45:02,587 - lava_callback - WARNING - Failed to upload log 2026-03-12 08:45:02,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:45:02,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:45:02,757 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d044bce73bcf02341e3/lava-logs.yaml 2026-03-12 08:45:02,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:45:02,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:45:02,886 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d044bce73bcf02341e3/lava_callback.json.gz 2026-03-12 08:45:02,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:02,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d044bce73bcf02341e3 HTTP/1.1" 200 None 2026-03-12 08:45:02,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:02,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:45:02,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:03,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d044bce73bcf02341e3 HTTP/1.1" 200 None 2026-03-12 08:45:03,010 - lava_callback - INFO - Completed processing callback for node 69b27d044bce73bcf02341e3 INFO: 172.20.0.1:44824 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:45:12,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:12,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54388 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:45:25,172 - lava_callback - INFO - Received callback for node 69b27d054bce73bcf02341e6 INFO: 172.20.0.1:39428 - "POST /node/69b27d054bce73bcf02341e6 HTTP/1.0" 202 Accepted 2026-03-12 08:45:25,225 - lava_callback - INFO - Starting async job submit for node 69b27d054bce73bcf02341e6 2026-03-12 08:45:25,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:25,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d054bce73bcf02341e6 HTTP/1.1" 200 None 2026-03-12 08:45:25,260 - lava_callback - DEBUG - Processing node 69b27d054bce73bcf02341e6: baseline-arm 2026-03-12 08:45:25,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:45:25,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:45:25,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d054bce73bcf02341e6/log.txt.gz 2026-03-12 08:45:25,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:45:25,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:45:25,863 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d054bce73bcf02341e6/lava-logs.yaml 2026-03-12 08:45:25,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:45:26,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:45:26,030 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d054bce73bcf02341e6/lava_callback.json.gz 2026-03-12 08:45:26,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:26,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d054bce73bcf02341e6 HTTP/1.1" 200 None 2026-03-12 08:45:26,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:26,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:45:26,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:26,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d054bce73bcf02341e6 HTTP/1.1" 200 None 2026-03-12 08:45:26,120 - lava_callback - INFO - Completed processing callback for node 69b27d054bce73bcf02341e6 INFO: 172.20.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58606 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:45:42,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:42,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-12 08:45:45,391 - lava_callback - INFO - Received callback for node 69b27d084bce73bcf02341ef INFO: 172.20.0.1:50100 - "POST /node/69b27d084bce73bcf02341ef HTTP/1.0" 202 Accepted 2026-03-12 08:45:45,438 - lava_callback - INFO - Starting async job submit for node 69b27d084bce73bcf02341ef 2026-03-12 08:45:45,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:45,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d084bce73bcf02341ef HTTP/1.1" 200 None 2026-03-12 08:45:45,472 - lava_callback - DEBUG - Processing node 69b27d084bce73bcf02341ef: baseline-arm 2026-03-12 08:45:45,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:45:45,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:45:45,736 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d084bce73bcf02341ef/log.txt.gz 2026-03-12 08:45:45,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:45:45,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:45:45,986 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d084bce73bcf02341ef/lava-logs.yaml 2026-03-12 08:45:45,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:45:46,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:45:46,159 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d084bce73bcf02341ef/lava_callback.json.gz 2026-03-12 08:45:46,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:46,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d084bce73bcf02341ef HTTP/1.1" 200 None 2026-03-12 08:45:46,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:46,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:45:46,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:45:46,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d084bce73bcf02341ef HTTP/1.1" 200 None 2026-03-12 08:45:46,239 - lava_callback - INFO - Completed processing callback for node 69b27d084bce73bcf02341ef INFO: 172.20.0.1:50110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53598 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:46:12,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:12,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55262 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:46:23,335 - lava_callback - INFO - Received callback for node 69b27d274bce73bcf0234268 INFO: 172.20.0.1:58946 - "POST /node/69b27d274bce73bcf0234268 HTTP/1.0" 202 Accepted 2026-03-12 08:46:23,381 - lava_callback - INFO - Starting async job submit for node 69b27d274bce73bcf0234268 2026-03-12 08:46:23,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:23,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d274bce73bcf0234268 HTTP/1.1" 200 None 2026-03-12 08:46:23,416 - lava_callback - DEBUG - Processing node 69b27d274bce73bcf0234268: baseline-arm-pengutronix 2026-03-12 08:46:23,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:23,630 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:23,632 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d274bce73bcf0234268/log.txt.gz 2026-03-12 08:46:23,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:23,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:23,887 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d274bce73bcf0234268/lava-logs.yaml 2026-03-12 08:46:23,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:24,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:24,016 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d274bce73bcf0234268/lava_callback.json.gz 2026-03-12 08:46:24,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:24,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d274bce73bcf0234268 HTTP/1.1" 200 None 2026-03-12 08:46:24,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:24,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:46:24,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:24,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d274bce73bcf0234268 HTTP/1.1" 200 None 2026-03-12 08:46:24,096 - lava_callback - INFO - Completed processing callback for node 69b27d274bce73bcf0234268 INFO: 172.20.0.1:58962 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:46:26,616 - lava_callback - INFO - Received callback for node 69b27d234bce73bcf023425b INFO: 172.20.0.1:58972 - "POST /node/69b27d234bce73bcf023425b HTTP/1.0" 202 Accepted 2026-03-12 08:46:26,622 - lava_callback - INFO - Starting async job submit for node 69b27d234bce73bcf023425b 2026-03-12 08:46:26,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:26,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d234bce73bcf023425b HTTP/1.1" 200 None 2026-03-12 08:46:26,660 - lava_callback - DEBUG - Processing node 69b27d234bce73bcf023425b: baseline-arm-pengutronix 2026-03-12 08:46:26,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:26,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:26,942 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d234bce73bcf023425b/log.txt.gz 2026-03-12 08:46:27,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:27,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:27,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d234bce73bcf023425b/lava-logs.yaml 2026-03-12 08:46:27,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:27,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:27,365 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d234bce73bcf023425b/lava_callback.json.gz 2026-03-12 08:46:27,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:27,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d234bce73bcf023425b HTTP/1.1" 200 None 2026-03-12 08:46:27,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:27,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:46:27,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:27,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d234bce73bcf023425b HTTP/1.1" 200 None 2026-03-12 08:46:27,457 - lava_callback - INFO - Completed processing callback for node 69b27d234bce73bcf023425b 2026-03-12 08:46:28,630 - lava_callback - INFO - Received callback for node 69b27d244bce73bcf023425e INFO: 172.20.0.1:58988 - "POST /node/69b27d244bce73bcf023425e HTTP/1.0" 202 Accepted 2026-03-12 08:46:28,636 - lava_callback - INFO - Starting async job submit for node 69b27d244bce73bcf023425e 2026-03-12 08:46:28,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:28,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d244bce73bcf023425e HTTP/1.1" 200 None 2026-03-12 08:46:28,675 - lava_callback - DEBUG - Processing node 69b27d244bce73bcf023425e: baseline-arm-pengutronix 2026-03-12 08:46:28,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:28,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:28,950 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d244bce73bcf023425e/log.txt.gz 2026-03-12 08:46:28,991 - lava_callback - INFO - Received callback for node 69b27d044bce73bcf02341e0 2026-03-12 08:46:29,008 - lava_callback - INFO - Starting async job submit for node 69b27d044bce73bcf02341e0 INFO: 172.20.0.1:58998 - "POST /node/69b27d044bce73bcf02341e0 HTTP/1.0" 202 Accepted 2026-03-12 08:46:29,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:29,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:29,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d044bce73bcf02341e0 HTTP/1.1" 200 None 2026-03-12 08:46:29,092 - lava_callback - DEBUG - Processing node 69b27d044bce73bcf02341e0: baseline-arm 2026-03-12 08:46:29,290 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:29,292 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d244bce73bcf023425e/lava-logs.yaml 2026-03-12 08:46:29,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:29,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:29,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:29,409 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d044bce73bcf02341e0/log.txt.gz 2026-03-12 08:46:29,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:29,533 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d244bce73bcf023425e/lava_callback.json.gz 2026-03-12 08:46:29,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:29,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:29,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d244bce73bcf023425e HTTP/1.1" 200 None 2026-03-12 08:46:29,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:29,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:46:29,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:29,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d244bce73bcf023425e HTTP/1.1" 200 None 2026-03-12 08:46:29,661 - lava_callback - INFO - Completed processing callback for node 69b27d244bce73bcf023425e 2026-03-12 08:46:29,805 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:29,808 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d044bce73bcf02341e0/lava-logs.yaml 2026-03-12 08:46:29,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:30,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:30,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d044bce73bcf02341e0/lava_callback.json.gz 2026-03-12 08:46:30,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:30,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d044bce73bcf02341e0 HTTP/1.1" 200 None 2026-03-12 08:46:30,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:30,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:46:30,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:30,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d044bce73bcf02341e0 HTTP/1.1" 200 None 2026-03-12 08:46:30,115 - lava_callback - INFO - Completed processing callback for node 69b27d044bce73bcf02341e0 2026-03-12 08:46:31,190 - lava_callback - INFO - Received callback for node 69b27d074bce73bcf02341ec INFO: 172.20.0.1:60514 - "POST /node/69b27d074bce73bcf02341ec HTTP/1.0" 202 Accepted 2026-03-12 08:46:31,196 - lava_callback - INFO - Starting async job submit for node 69b27d074bce73bcf02341ec 2026-03-12 08:46:31,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:31,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d074bce73bcf02341ec HTTP/1.1" 200 None 2026-03-12 08:46:31,233 - lava_callback - DEBUG - Processing node 69b27d074bce73bcf02341ec: baseline-arm 2026-03-12 08:46:31,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:31,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:31,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d074bce73bcf02341ec/log.txt.gz 2026-03-12 08:46:31,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:31,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:31,980 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d074bce73bcf02341ec/lava-logs.yaml 2026-03-12 08:46:31,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:32,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:32,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d074bce73bcf02341ec/lava_callback.json.gz 2026-03-12 08:46:32,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:32,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d074bce73bcf02341ec HTTP/1.1" 200 None 2026-03-12 08:46:32,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:32,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:46:32,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:32,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d074bce73bcf02341ec HTTP/1.1" 200 None 2026-03-12 08:46:32,224 - lava_callback - INFO - Completed processing callback for node 69b27d074bce73bcf02341ec INFO: 172.20.0.1:60528 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:46:42,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:42,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:58322 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:46:46,771 - lava_callback - INFO - Received callback for node 69b27d174bce73bcf0234236 INFO: 172.20.0.1:58332 - "POST /node/69b27d174bce73bcf0234236 HTTP/1.0" 202 Accepted 2026-03-12 08:46:46,780 - lava_callback - INFO - Starting async job submit for node 69b27d174bce73bcf0234236 2026-03-12 08:46:46,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:46,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d174bce73bcf0234236 HTTP/1.1" 200 None 2026-03-12 08:46:46,855 - lava_callback - DEBUG - Processing node 69b27d174bce73bcf0234236: baseline-arm-clabbe 2026-03-12 08:46:47,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:47,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:47,255 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d174bce73bcf0234236/log.txt.gz 2026-03-12 08:46:47,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:47,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:47,804 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d174bce73bcf0234236/lava-logs.yaml 2026-03-12 08:46:47,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:47,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:47,956 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d174bce73bcf0234236/lava_callback.json.gz 2026-03-12 08:46:47,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:47,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d174bce73bcf0234236 HTTP/1.1" 200 None 2026-03-12 08:46:47,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:48,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:46:48,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:48,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d174bce73bcf0234236 HTTP/1.1" 200 None 2026-03-12 08:46:48,053 - lava_callback - INFO - Completed processing callback for node 69b27d174bce73bcf0234236 2026-03-12 08:46:52,882 - lava_callback - INFO - Received callback for node 69b27d134bce73bcf0234227 INFO: 172.20.0.1:33394 - "POST /node/69b27d134bce73bcf0234227 HTTP/1.0" 202 Accepted 2026-03-12 08:46:52,891 - lava_callback - INFO - Starting async job submit for node 69b27d134bce73bcf0234227 2026-03-12 08:46:52,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:52,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d134bce73bcf0234227 HTTP/1.1" 200 None 2026-03-12 08:46:52,961 - lava_callback - DEBUG - Processing node 69b27d134bce73bcf0234227: baseline-arm-broonie 2026-03-12 08:46:53,247 - lava_callback - INFO - Received callback for node 69afda66ae557f2600f0f48d 2026-03-12 08:46:53,267 - lava_callback - INFO - Starting async job submit for node 69afda66ae557f2600f0f48d 2026-03-12 08:46:53,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:33410 - "POST /node/69afda66ae557f2600f0f48d HTTP/1.0" 202 Accepted 2026-03-12 08:46:53,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afda66ae557f2600f0f48d HTTP/1.1" 200 None 2026-03-12 08:46:53,346 - lava_callback - DEBUG - Processing node 69afda66ae557f2600f0f48d: baseline-arm64-baylibre 2026-03-12 08:46:53,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afda66ae557f2600f0f48d 2026-03-12 08:46:53,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:53,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:53,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afda66ae557f2600f0f48d HTTP/1.1" 200 None 2026-03-12 08:46:53,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:53,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 08:46:53,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:53,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afda66ae557f2600f0f48d HTTP/1.1" 200 None 2026-03-12 08:46:53,413 - lava_callback - ERROR - Error processing callback for node 69afda66ae557f2600f0f48d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:46:53,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:53,543 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d134bce73bcf0234227/log.txt.gz INFO: 172.20.0.1:33412 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:46:53,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:54,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:54,118 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d134bce73bcf0234227/lava-logs.yaml 2026-03-12 08:46:54,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:54,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:54,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d134bce73bcf0234227/lava_callback.json.gz 2026-03-12 08:46:54,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:54,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d134bce73bcf0234227 HTTP/1.1" 200 None 2026-03-12 08:46:54,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:54,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:46:54,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:54,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d134bce73bcf0234227 HTTP/1.1" 200 None 2026-03-12 08:46:54,387 - lava_callback - INFO - Completed processing callback for node 69b27d134bce73bcf0234227 2026-03-12 08:46:56,448 - lava_callback - INFO - Received callback for node 69b27d204bce73bcf0234250 INFO: 172.20.0.1:33420 - "POST /node/69b27d204bce73bcf0234250 HTTP/1.0" 202 Accepted 2026-03-12 08:46:56,455 - lava_callback - INFO - Starting async job submit for node 69b27d204bce73bcf0234250 2026-03-12 08:46:56,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:33436 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:46:56,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d204bce73bcf0234250 HTTP/1.1" 200 None 2026-03-12 08:46:56,520 - lava_callback - DEBUG - Processing node 69b27d204bce73bcf0234250: baseline-arm-kontron 2026-03-12 08:46:56,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:56,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:56,987 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b27d204bce73bcf0234250/log.txt.gz 2026-03-12 08:46:57,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:57,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:57,270 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b27d204bce73bcf0234250/lava-logs.yaml 2026-03-12 08:46:57,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:46:57,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:46:57,648 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69b27d204bce73bcf0234250/lava_callback.json.gz 2026-03-12 08:46:57,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:57,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d204bce73bcf0234250 HTTP/1.1" 200 None 2026-03-12 08:46:57,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:57,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:46:57,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:46:57,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d204bce73bcf0234250 HTTP/1.1" 200 None 2026-03-12 08:46:57,773 - lava_callback - INFO - Completed processing callback for node 69b27d204bce73bcf0234250 INFO: 172.20.0.1:45876 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:47:07,029 - lava_callback - INFO - Received callback for node 69b27d114bce73bcf0234204 INFO: 172.20.0.1:45886 - "POST /node/69b27d114bce73bcf0234204 HTTP/1.0" 202 Accepted 2026-03-12 08:47:07,035 - lava_callback - INFO - Starting async job submit for node 69b27d114bce73bcf0234204 2026-03-12 08:47:07,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:07,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d114bce73bcf0234204 HTTP/1.1" 200 None 2026-03-12 08:47:07,078 - lava_callback - DEBUG - Processing node 69b27d114bce73bcf0234204: baseline-arm-broonie 2026-03-12 08:47:07,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:07,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:07,687 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d114bce73bcf0234204/log.txt.gz 2026-03-12 08:47:07,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:08,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:08,020 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d114bce73bcf0234204/lava-logs.yaml 2026-03-12 08:47:08,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:08,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:08,508 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d114bce73bcf0234204/lava_callback.json.gz 2026-03-12 08:47:08,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:08,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d114bce73bcf0234204 HTTP/1.1" 200 None 2026-03-12 08:47:08,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:08,565 - lava_callback - INFO - Received callback for node 69b27d124bce73bcf0234224 INFO: 172.20.0.1:45892 - "POST /node/69b27d124bce73bcf0234224 HTTP/1.0" 202 Accepted 2026-03-12 08:47:08,575 - lava_callback - INFO - Starting async job submit for node 69b27d124bce73bcf0234224 2026-03-12 08:47:08,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:08,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:47:08,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:08,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d124bce73bcf0234224 HTTP/1.1" 200 None 2026-03-12 08:47:08,624 - lava_callback - DEBUG - Processing node 69b27d124bce73bcf0234224: baseline-arm-broonie 2026-03-12 08:47:08,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d114bce73bcf0234204 HTTP/1.1" 200 None 2026-03-12 08:47:08,733 - lava_callback - INFO - Completed processing callback for node 69b27d114bce73bcf0234204 2026-03-12 08:47:08,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:09,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:09,379 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d124bce73bcf0234224/log.txt.gz 2026-03-12 08:47:09,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:09,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:09,730 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d124bce73bcf0234224/lava-logs.yaml 2026-03-12 08:47:09,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:10,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:10,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d124bce73bcf0234224/lava_callback.json.gz 2026-03-12 08:47:10,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:10,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d124bce73bcf0234224 HTTP/1.1" 200 None 2026-03-12 08:47:10,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:10,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:47:10,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:10,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d124bce73bcf0234224 HTTP/1.1" 200 None 2026-03-12 08:47:10,331 - lava_callback - INFO - Completed processing callback for node 69b27d124bce73bcf0234224 2026-03-12 08:47:12,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:12,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-12 08:47:12,861 - lava_callback - INFO - Received callback for node 69b27d0f4bce73bcf0234201 INFO: 172.20.0.1:37842 - "POST /node/69b27d0f4bce73bcf0234201 HTTP/1.0" 202 Accepted 2026-03-12 08:47:12,875 - lava_callback - INFO - Starting async job submit for node 69b27d0f4bce73bcf0234201 2026-03-12 08:47:12,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:12,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d0f4bce73bcf0234201 HTTP/1.1" 200 None 2026-03-12 08:47:12,949 - lava_callback - DEBUG - Processing node 69b27d0f4bce73bcf0234201: baseline-arm-broonie 2026-03-12 08:47:13,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:13,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:13,413 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d0f4bce73bcf0234201/log.txt.gz 2026-03-12 08:47:13,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:14,006 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:14,009 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d0f4bce73bcf0234201/lava-logs.yaml 2026-03-12 08:47:14,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:14,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:14,159 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69b27d0f4bce73bcf0234201/lava_callback.json.gz 2026-03-12 08:47:14,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:14,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d0f4bce73bcf0234201 HTTP/1.1" 200 None 2026-03-12 08:47:14,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:14,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:47:14,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:14,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d0f4bce73bcf0234201 HTTP/1.1" 200 None 2026-03-12 08:47:14,239 - lava_callback - INFO - Completed processing callback for node 69b27d0f4bce73bcf0234201 INFO: 172.20.0.1:37858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47004 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:47:42,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:42,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46644 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:47:53,193 - lava_callback - INFO - Received callback for node 69afdaa5ae557f2600f0f528 INFO: 172.20.0.1:44560 - "POST /node/69afdaa5ae557f2600f0f528 HTTP/1.0" 202 Accepted 2026-03-12 08:47:53,199 - lava_callback - INFO - Starting async job submit for node 69afdaa5ae557f2600f0f528 2026-03-12 08:47:53,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:53,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaa5ae557f2600f0f528 HTTP/1.1" 200 None 2026-03-12 08:47:53,217 - lava_callback - DEBUG - Processing node 69afdaa5ae557f2600f0f528: baseline-arm64-baylibre 2026-03-12 08:47:53,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdaa5ae557f2600f0f528 2026-03-12 08:47:53,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:53,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaa5ae557f2600f0f528 HTTP/1.1" 200 None 2026-03-12 08:47:53,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:53,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:47:53,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:53,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdaa5ae557f2600f0f528 HTTP/1.1" 200 None 2026-03-12 08:47:53,280 - lava_callback - ERROR - Error processing callback for node 69afdaa5ae557f2600f0f528: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44566 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:47:53,940 - lava_callback - INFO - Received callback for node 69afdaa7ae557f2600f0f52b INFO: 172.20.0.1:44580 - "POST /node/69afdaa7ae557f2600f0f52b HTTP/1.0" 202 Accepted 2026-03-12 08:47:53,946 - lava_callback - INFO - Starting async job submit for node 69afdaa7ae557f2600f0f52b 2026-03-12 08:47:53,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:53,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaa7ae557f2600f0f52b HTTP/1.1" 200 None 2026-03-12 08:47:53,963 - lava_callback - DEBUG - Processing node 69afdaa7ae557f2600f0f52b: baseline-arm64-baylibre 2026-03-12 08:47:53,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdaa7ae557f2600f0f52b 2026-03-12 08:47:53,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:53,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaa7ae557f2600f0f52b HTTP/1.1" 200 None 2026-03-12 08:47:53,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:53,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:47:53,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:54,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdaa7ae557f2600f0f52b HTTP/1.1" 200 None 2026-03-12 08:47:54,022 - lava_callback - ERROR - Error processing callback for node 69afdaa7ae557f2600f0f52b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:47:54,624 - lava_callback - INFO - Received callback for node 69afdaa9ae557f2600f0f52f INFO: 172.20.0.1:44584 - "POST /node/69afdaa9ae557f2600f0f52f HTTP/1.0" 202 Accepted 2026-03-12 08:47:54,634 - lava_callback - INFO - Starting async job submit for node 69afdaa9ae557f2600f0f52f 2026-03-12 08:47:54,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:54,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaa9ae557f2600f0f52f HTTP/1.1" 200 None 2026-03-12 08:47:54,662 - lava_callback - DEBUG - Processing node 69afdaa9ae557f2600f0f52f: baseline-arm64-baylibre 2026-03-12 08:47:54,662 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdaa9ae557f2600f0f52f 2026-03-12 08:47:54,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:54,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaa9ae557f2600f0f52f HTTP/1.1" 200 None 2026-03-12 08:47:54,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:54,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:47:54,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:54,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdaa9ae557f2600f0f52f HTTP/1.1" 200 None 2026-03-12 08:47:54,722 - lava_callback - ERROR - Error processing callback for node 69afdaa9ae557f2600f0f52f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:47:55,299 - lava_callback - INFO - Received callback for node 69afdaacae557f2600f0f539 INFO: 172.20.0.1:44598 - "POST /node/69afdaacae557f2600f0f539 HTTP/1.0" 202 Accepted 2026-03-12 08:47:55,309 - lava_callback - INFO - Starting async job submit for node 69afdaacae557f2600f0f539 2026-03-12 08:47:55,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:55,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaacae557f2600f0f539 HTTP/1.1" 200 None 2026-03-12 08:47:55,339 - lava_callback - DEBUG - Processing node 69afdaacae557f2600f0f539: baseline-arm64-baylibre 2026-03-12 08:47:55,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdaacae557f2600f0f539 2026-03-12 08:47:55,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:55,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaacae557f2600f0f539 HTTP/1.1" 200 None 2026-03-12 08:47:55,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:55,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:47:55,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:55,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdaacae557f2600f0f539 HTTP/1.1" 200 None 2026-03-12 08:47:55,399 - lava_callback - ERROR - Error processing callback for node 69afdaacae557f2600f0f539: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:47:55,974 - lava_callback - INFO - Received callback for node 69afdaaeae557f2600f0f53c INFO: 172.20.0.1:44604 - "POST /node/69afdaaeae557f2600f0f53c HTTP/1.0" 202 Accepted 2026-03-12 08:47:55,980 - lava_callback - INFO - Starting async job submit for node 69afdaaeae557f2600f0f53c 2026-03-12 08:47:55,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:56,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaaeae557f2600f0f53c HTTP/1.1" 200 None 2026-03-12 08:47:56,005 - lava_callback - DEBUG - Processing node 69afdaaeae557f2600f0f53c: baseline-arm64-baylibre 2026-03-12 08:47:56,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdaaeae557f2600f0f53c 2026-03-12 08:47:56,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:56,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaaeae557f2600f0f53c HTTP/1.1" 200 None 2026-03-12 08:47:56,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:56,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:47:56,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:56,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdaaeae557f2600f0f53c HTTP/1.1" 200 None 2026-03-12 08:47:56,063 - lava_callback - ERROR - Error processing callback for node 69afdaaeae557f2600f0f53c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44606 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:47:56,662 - lava_callback - INFO - Received callback for node 69afdab0ae557f2600f0f53f INFO: 172.20.0.1:44620 - "POST /node/69afdab0ae557f2600f0f53f HTTP/1.0" 202 Accepted 2026-03-12 08:47:56,670 - lava_callback - INFO - Starting async job submit for node 69afdab0ae557f2600f0f53f 2026-03-12 08:47:56,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:56,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdab0ae557f2600f0f53f HTTP/1.1" 200 None 2026-03-12 08:47:56,692 - lava_callback - DEBUG - Processing node 69afdab0ae557f2600f0f53f: baseline-arm64-baylibre 2026-03-12 08:47:56,693 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdab0ae557f2600f0f53f 2026-03-12 08:47:56,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:56,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdab0ae557f2600f0f53f HTTP/1.1" 200 None 2026-03-12 08:47:56,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:56,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:47:56,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:56,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdab0ae557f2600f0f53f HTTP/1.1" 200 None 2026-03-12 08:47:56,756 - lava_callback - ERROR - Error processing callback for node 69afdab0ae557f2600f0f53f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:47:57,007 - lava_callback - INFO - Received callback for node 69b27d3e4bce73bcf02342a8 INFO: 172.20.0.1:44634 - "POST /node/69b27d3e4bce73bcf02342a8 HTTP/1.0" 202 Accepted 2026-03-12 08:47:57,031 - lava_callback - INFO - Starting async job submit for node 69b27d3e4bce73bcf02342a8 2026-03-12 08:47:57,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:57,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d3e4bce73bcf02342a8 HTTP/1.1" 200 None 2026-03-12 08:47:57,098 - lava_callback - DEBUG - Processing node 69b27d3e4bce73bcf02342a8: ltp-dio 2026-03-12 08:47:57,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:58,048 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:58,051 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69b27d3e4bce73bcf02342a8/log.txt.gz 2026-03-12 08:47:58,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:58,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:58,968 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69b27d3e4bce73bcf02342a8/lava-logs.yaml 2026-03-12 08:47:58,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:47:59,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:47:59,188 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69b27d3e4bce73bcf02342a8/lava_callback.json.gz 2026-03-12 08:47:59,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:59,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d3e4bce73bcf02342a8 HTTP/1.1" 200 None 2026-03-12 08:47:59,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:59,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:47:59,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:47:59,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d3e4bce73bcf02342a8 HTTP/1.1" 200 None 2026-03-12 08:47:59,350 - lava_callback - INFO - Completed processing callback for node 69b27d3e4bce73bcf02342a8 INFO: 172.20.0.1:42986 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:48:12,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:12,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53270 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:48:22,611 - lava_callback - INFO - Received callback for node 69afdac6ae557f2600f0f594 INFO: 172.20.0.1:35642 - "POST /node/69afdac6ae557f2600f0f594 HTTP/1.0" 202 Accepted 2026-03-12 08:48:22,621 - lava_callback - INFO - Starting async job submit for node 69afdac6ae557f2600f0f594 2026-03-12 08:48:22,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:22,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdac6ae557f2600f0f594 HTTP/1.1" 200 None 2026-03-12 08:48:22,651 - lava_callback - DEBUG - Processing node 69afdac6ae557f2600f0f594: baseline-arm64-clabbe 2026-03-12 08:48:22,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdac6ae557f2600f0f594 2026-03-12 08:48:22,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:22,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdac6ae557f2600f0f594 HTTP/1.1" 200 None 2026-03-12 08:48:22,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:22,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:48:22,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:22,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdac6ae557f2600f0f594 HTTP/1.1" 200 None 2026-03-12 08:48:22,751 - lava_callback - ERROR - Error processing callback for node 69afdac6ae557f2600f0f594: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:48:23,283 - lava_callback - INFO - Received callback for node 69afdac9ae557f2600f0f597 INFO: 172.20.0.1:35652 - "POST /node/69afdac9ae557f2600f0f597 HTTP/1.0" 202 Accepted 2026-03-12 08:48:23,292 - lava_callback - INFO - Starting async job submit for node 69afdac9ae557f2600f0f597 2026-03-12 08:48:23,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:23,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdac9ae557f2600f0f597 HTTP/1.1" 200 None 2026-03-12 08:48:23,318 - lava_callback - DEBUG - Processing node 69afdac9ae557f2600f0f597: baseline-arm64-clabbe 2026-03-12 08:48:23,319 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdac9ae557f2600f0f597 2026-03-12 08:48:23,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:23,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdac9ae557f2600f0f597 HTTP/1.1" 200 None 2026-03-12 08:48:23,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:23,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:48:23,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:23,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdac9ae557f2600f0f597 HTTP/1.1" 200 None 2026-03-12 08:48:23,385 - lava_callback - ERROR - Error processing callback for node 69afdac9ae557f2600f0f597: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:48:23,909 - lava_callback - INFO - Received callback for node 69afdacaae557f2600f0f59a INFO: 172.20.0.1:35658 - "POST /node/69afdacaae557f2600f0f59a HTTP/1.0" 202 Accepted 2026-03-12 08:48:23,915 - lava_callback - INFO - Starting async job submit for node 69afdacaae557f2600f0f59a 2026-03-12 08:48:23,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:23,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdacaae557f2600f0f59a HTTP/1.1" 200 None 2026-03-12 08:48:23,937 - lava_callback - DEBUG - Processing node 69afdacaae557f2600f0f59a: baseline-arm64-clabbe 2026-03-12 08:48:23,937 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdacaae557f2600f0f59a 2026-03-12 08:48:23,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:23,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdacaae557f2600f0f59a HTTP/1.1" 200 None 2026-03-12 08:48:23,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:23,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:48:23,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:24,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdacaae557f2600f0f59a HTTP/1.1" 200 None 2026-03-12 08:48:24,017 - lava_callback - ERROR - Error processing callback for node 69afdacaae557f2600f0f59a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:48:24,530 - lava_callback - INFO - Received callback for node 69afdacbae557f2600f0f59d INFO: 172.20.0.1:35672 - "POST /node/69afdacbae557f2600f0f59d HTTP/1.0" 202 Accepted 2026-03-12 08:48:24,538 - lava_callback - INFO - Starting async job submit for node 69afdacbae557f2600f0f59d 2026-03-12 08:48:24,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:24,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdacbae557f2600f0f59d HTTP/1.1" 200 None 2026-03-12 08:48:24,565 - lava_callback - DEBUG - Processing node 69afdacbae557f2600f0f59d: baseline-arm64-clabbe 2026-03-12 08:48:24,566 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdacbae557f2600f0f59d 2026-03-12 08:48:24,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:24,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdacbae557f2600f0f59d HTTP/1.1" 200 None 2026-03-12 08:48:24,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:24,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:48:24,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:24,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdacbae557f2600f0f59d HTTP/1.1" 200 None 2026-03-12 08:48:24,701 - lava_callback - ERROR - Error processing callback for node 69afdacbae557f2600f0f59d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:48:25,166 - lava_callback - INFO - Received callback for node 69afdad0ae557f2600f0f5a6 INFO: 172.20.0.1:35686 - "POST /node/69afdad0ae557f2600f0f5a6 HTTP/1.0" 202 Accepted 2026-03-12 08:48:25,174 - lava_callback - INFO - Starting async job submit for node 69afdad0ae557f2600f0f5a6 2026-03-12 08:48:25,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:25,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdad0ae557f2600f0f5a6 HTTP/1.1" 200 None 2026-03-12 08:48:25,200 - lava_callback - DEBUG - Processing node 69afdad0ae557f2600f0f5a6: baseline-arm64-clabbe 2026-03-12 08:48:25,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdad0ae557f2600f0f5a6 2026-03-12 08:48:25,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:25,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdad0ae557f2600f0f5a6 HTTP/1.1" 200 None 2026-03-12 08:48:25,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:25,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:48:25,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:25,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdad0ae557f2600f0f5a6 HTTP/1.1" 200 None 2026-03-12 08:48:25,287 - lava_callback - ERROR - Error processing callback for node 69afdad0ae557f2600f0f5a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:48:25,822 - lava_callback - INFO - Received callback for node 69afdaceae557f2600f0f5a3 INFO: 172.20.0.1:35698 - "POST /node/69afdaceae557f2600f0f5a3 HTTP/1.0" 202 Accepted 2026-03-12 08:48:25,840 - lava_callback - INFO - Starting async job submit for node 69afdaceae557f2600f0f5a3 2026-03-12 08:48:25,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:25,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaceae557f2600f0f5a3 HTTP/1.1" 200 None 2026-03-12 08:48:25,913 - lava_callback - DEBUG - Processing node 69afdaceae557f2600f0f5a3: baseline-arm64-clabbe 2026-03-12 08:48:25,913 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdaceae557f2600f0f5a3 2026-03-12 08:48:25,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:25,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdaceae557f2600f0f5a3 HTTP/1.1" 200 None 2026-03-12 08:48:25,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:26,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 08:48:26,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:26,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdaceae557f2600f0f5a3 HTTP/1.1" 200 None 2026-03-12 08:48:26,155 - lava_callback - ERROR - Error processing callback for node 69afdaceae557f2600f0f5a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35706 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:48:33,800 - lava_callback - INFO - Received callback for node 69b27d194bce73bcf023423b INFO: 172.20.0.1:42460 - "POST /node/69b27d194bce73bcf023423b HTTP/1.0" 202 Accepted 2026-03-12 08:48:33,807 - lava_callback - INFO - Starting async job submit for node 69b27d194bce73bcf023423b 2026-03-12 08:48:33,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:33,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d194bce73bcf023423b HTTP/1.1" 200 None 2026-03-12 08:48:33,863 - lava_callback - DEBUG - Processing node 69b27d194bce73bcf023423b: baseline-arm-clabbe 2026-03-12 08:48:34,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:48:35,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:48:35,121 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d194bce73bcf023423b/log.txt.gz 2026-03-12 08:48:35,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:48:36,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:48:36,241 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d194bce73bcf023423b/lava-logs.yaml 2026-03-12 08:48:36,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:42466 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:48:37,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:48:37,090 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d194bce73bcf023423b/lava_callback.json.gz 2026-03-12 08:48:37,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:37,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d194bce73bcf023423b HTTP/1.1" 200 None 2026-03-12 08:48:37,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:37,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:48:37,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:37,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d194bce73bcf023423b HTTP/1.1" 200 None 2026-03-12 08:48:37,216 - lava_callback - INFO - Completed processing callback for node 69b27d194bce73bcf023423b 2026-03-12 08:48:42,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:42,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35288 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:48:47,342 - lava_callback - INFO - Received callback for node 69b2794a4bce73bcf0233da0 INFO: 172.20.0.1:35296 - "POST /node/69b2794a4bce73bcf0233da0 HTTP/1.0" 202 Accepted 2026-03-12 08:48:47,350 - lava_callback - INFO - Starting async job submit for node 69b2794a4bce73bcf0233da0 2026-03-12 08:48:47,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:47,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2794a4bce73bcf0233da0 HTTP/1.1" 200 None 2026-03-12 08:48:47,454 - lava_callback - DEBUG - Processing node 69b2794a4bce73bcf0233da0: ltp-smoke 2026-03-12 08:48:48,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:48:48,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:48:48,974 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b2794a4bce73bcf0233da0/log.txt.gz 2026-03-12 08:48:49,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:48:49,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:48:49,991 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b2794a4bce73bcf0233da0/lava-logs.yaml 2026-03-12 08:48:50,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:48:50,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:48:50,660 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b2794a4bce73bcf0233da0/lava_callback.json.gz 2026-03-12 08:48:50,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:50,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2794a4bce73bcf0233da0 HTTP/1.1" 200 None 2026-03-12 08:48:50,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:50,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2766c4bce73bcf0233d1a HTTP/1.1" 200 None 2026-03-12 08:48:50,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:48:50,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2794a4bce73bcf0233da0 HTTP/1.1" 200 None 2026-03-12 08:48:51,046 - lava_callback - INFO - Completed processing callback for node 69b2794a4bce73bcf0233da0 INFO: 172.20.0.1:49804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57770 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:49:12,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:12,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-12 08:49:12,776 - lava_callback - INFO - Received callback for node 69b27d4a4bce73bcf02342c3 INFO: 172.20.0.1:45266 - "POST /node/69b27d4a4bce73bcf02342c3 HTTP/1.0" 202 Accepted 2026-03-12 08:49:12,786 - lava_callback - INFO - Starting async job submit for node 69b27d4a4bce73bcf02342c3 2026-03-12 08:49:12,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:12,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d4a4bce73bcf02342c3 HTTP/1.1" 200 None 2026-03-12 08:49:12,864 - lava_callback - DEBUG - Processing node 69b27d4a4bce73bcf02342c3: ltp-syscalls 2026-03-12 08:49:13,047 - lava_callback - WARNING - Failed to upload log 2026-03-12 08:49:13,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:49:13,633 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:49:13,636 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b27d4a4bce73bcf02342c3/lava-logs.yaml 2026-03-12 08:49:13,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:49:14,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:49:14,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69b27d4a4bce73bcf02342c3/lava_callback.json.gz 2026-03-12 08:49:14,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:14,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d4a4bce73bcf02342c3 HTTP/1.1" 200 None 2026-03-12 08:49:14,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:14,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:49:14,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:14,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d4a4bce73bcf02342c3 HTTP/1.1" 200 None 2026-03-12 08:49:14,377 - lava_callback - INFO - Completed processing callback for node 69b27d4a4bce73bcf02342c3 INFO: 172.20.0.1:45280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43632 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:49:42,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:42,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45248 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:49:53,358 - lava_callback - INFO - Received callback for node 69b27d3a4bce73bcf02342a2 INFO: 172.20.0.1:58804 - "POST /node/69b27d3a4bce73bcf02342a2 HTTP/1.0" 202 Accepted 2026-03-12 08:49:53,364 - lava_callback - INFO - Received callback for node 69afdb20ae557f2600f0f653 INFO: 172.20.0.1:58806 - "POST /node/69afdb20ae557f2600f0f653 HTTP/1.0" 202 Accepted 2026-03-12 08:49:53,370 - lava_callback - INFO - Starting async job submit for node 69b27d3a4bce73bcf02342a2 2026-03-12 08:49:53,370 - lava_callback - INFO - Starting async job submit for node 69afdb20ae557f2600f0f653 2026-03-12 08:49:53,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:53,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:53,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb20ae557f2600f0f653 HTTP/1.1" 200 None 2026-03-12 08:49:53,414 - lava_callback - DEBUG - Processing node 69afdb20ae557f2600f0f653: baseline-arm-baylibre 2026-03-12 08:49:53,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb20ae557f2600f0f653 2026-03-12 08:49:53,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d3a4bce73bcf02342a2 HTTP/1.1" 200 None 2026-03-12 08:49:53,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:53,416 - lava_callback - DEBUG - Processing node 69b27d3a4bce73bcf02342a2: ltp-controllers 2026-03-12 08:49:53,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb20ae557f2600f0f653 HTTP/1.1" 200 None 2026-03-12 08:49:53,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:53,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:49:53,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:53,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb20ae557f2600f0f653 HTTP/1.1" 200 None 2026-03-12 08:49:53,760 - lava_callback - ERROR - Error processing callback for node 69afdb20ae557f2600f0f653: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58812 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:49:53,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:49:54,023 - lava_callback - INFO - Received callback for node 69afdb23ae557f2600f0f656 INFO: 172.20.0.1:58822 - "POST /node/69afdb23ae557f2600f0f656 HTTP/1.0" 202 Accepted 2026-03-12 08:49:54,032 - lava_callback - INFO - Starting async job submit for node 69afdb23ae557f2600f0f656 2026-03-12 08:49:54,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:54,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb23ae557f2600f0f656 HTTP/1.1" 200 None 2026-03-12 08:49:54,047 - lava_callback - DEBUG - Processing node 69afdb23ae557f2600f0f656: baseline-arm-baylibre 2026-03-12 08:49:54,048 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb23ae557f2600f0f656 2026-03-12 08:49:54,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:54,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb23ae557f2600f0f656 HTTP/1.1" 200 None 2026-03-12 08:49:54,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:54,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:49:54,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:54,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb23ae557f2600f0f656 HTTP/1.1" 200 None 2026-03-12 08:49:54,127 - lava_callback - ERROR - Error processing callback for node 69afdb23ae557f2600f0f656: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:49:54,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:49:54,188 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69b27d3a4bce73bcf02342a2/log.txt.gz 2026-03-12 08:49:54,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:49:54,685 - lava_callback - INFO - Received callback for node 69afdb26ae557f2600f0f659 INFO: 172.20.0.1:58832 - "POST /node/69afdb26ae557f2600f0f659 HTTP/1.0" 202 Accepted 2026-03-12 08:49:54,692 - lava_callback - INFO - Starting async job submit for node 69afdb26ae557f2600f0f659 2026-03-12 08:49:54,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:54,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:49:54,706 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69b27d3a4bce73bcf02342a2/lava-logs.yaml 2026-03-12 08:49:54,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb26ae557f2600f0f659 HTTP/1.1" 200 None 2026-03-12 08:49:54,713 - lava_callback - DEBUG - Processing node 69afdb26ae557f2600f0f659: baseline-arm-baylibre 2026-03-12 08:49:54,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb26ae557f2600f0f659 2026-03-12 08:49:54,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:54,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:49:54,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb26ae557f2600f0f659 HTTP/1.1" 200 None 2026-03-12 08:49:54,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:54,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:49:54,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:54,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb26ae557f2600f0f659 HTTP/1.1" 200 None 2026-03-12 08:49:54,772 - lava_callback - ERROR - Error processing callback for node 69afdb26ae557f2600f0f659: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:49:54,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:49:54,948 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69b27d3a4bce73bcf02342a2/lava_callback.json.gz 2026-03-12 08:49:54,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:55,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d3a4bce73bcf02342a2 HTTP/1.1" 200 None 2026-03-12 08:49:55,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:55,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:49:55,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:55,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d3a4bce73bcf02342a2 HTTP/1.1" 200 None 2026-03-12 08:49:55,069 - lava_callback - INFO - Completed processing callback for node 69b27d3a4bce73bcf02342a2 2026-03-12 08:49:55,330 - lava_callback - INFO - Received callback for node 69afdb29ae557f2600f0f661 INFO: 172.20.0.1:58838 - "POST /node/69afdb29ae557f2600f0f661 HTTP/1.0" 202 Accepted 2026-03-12 08:49:55,337 - lava_callback - INFO - Starting async job submit for node 69afdb29ae557f2600f0f661 2026-03-12 08:49:55,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:55,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb29ae557f2600f0f661 HTTP/1.1" 200 None 2026-03-12 08:49:55,358 - lava_callback - DEBUG - Processing node 69afdb29ae557f2600f0f661: baseline-arm-baylibre 2026-03-12 08:49:55,358 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb29ae557f2600f0f661 2026-03-12 08:49:55,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:55,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb29ae557f2600f0f661 HTTP/1.1" 200 None 2026-03-12 08:49:55,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:55,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:49:55,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:49:55,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb29ae557f2600f0f661 HTTP/1.1" 200 None 2026-03-12 08:49:55,424 - lava_callback - ERROR - Error processing callback for node 69afdb29ae557f2600f0f661: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48312 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:50:12,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:12,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-12 08:50:13,048 - lava_callback - INFO - Received callback for node 69b27d1c4bce73bcf0234247 INFO: 172.20.0.1:35746 - "POST /node/69b27d1c4bce73bcf0234247 HTTP/1.0" 202 Accepted 2026-03-12 08:50:13,054 - lava_callback - INFO - Starting async job submit for node 69b27d1c4bce73bcf0234247 2026-03-12 08:50:13,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:13,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d1c4bce73bcf0234247 HTTP/1.1" 200 None 2026-03-12 08:50:13,090 - lava_callback - DEBUG - Processing node 69b27d1c4bce73bcf0234247: baseline-arm-clabbe 2026-03-12 08:50:13,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:50:13,299 - lava_callback - INFO - Received callback for node 69afdb31ae557f2600f0f6af INFO: 172.20.0.1:35748 - "POST /node/69afdb31ae557f2600f0f6af HTTP/1.0" 202 Accepted 2026-03-12 08:50:13,305 - lava_callback - INFO - Starting async job submit for node 69afdb31ae557f2600f0f6af 2026-03-12 08:50:13,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:13,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb31ae557f2600f0f6af HTTP/1.1" 200 None 2026-03-12 08:50:13,320 - lava_callback - DEBUG - Processing node 69afdb31ae557f2600f0f6af: baseline-arm-baylibre 2026-03-12 08:50:13,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb31ae557f2600f0f6af 2026-03-12 08:50:13,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:13,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb31ae557f2600f0f6af HTTP/1.1" 200 None 2026-03-12 08:50:13,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:13,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:50:13,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:13,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb31ae557f2600f0f6af HTTP/1.1" 200 None 2026-03-12 08:50:13,377 - lava_callback - ERROR - Error processing callback for node 69afdb31ae557f2600f0f6af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:50:13,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:50:13,467 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d1c4bce73bcf0234247/log.txt.gz 2026-03-12 08:50:13,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:50:13,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:50:13,814 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d1c4bce73bcf0234247/lava-logs.yaml 2026-03-12 08:50:13,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:50:13,971 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:50:13,973 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d1c4bce73bcf0234247/lava_callback.json.gz 2026-03-12 08:50:13,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:13,995 - lava_callback - INFO - Received callback for node 69afdb34ae557f2600f0f6ba INFO: 172.20.0.1:35752 - "POST /node/69afdb34ae557f2600f0f6ba HTTP/1.0" 202 Accepted 2026-03-12 08:50:14,001 - lava_callback - INFO - Starting async job submit for node 69afdb34ae557f2600f0f6ba 2026-03-12 08:50:14,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:14,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d1c4bce73bcf0234247 HTTP/1.1" 200 None 2026-03-12 08:50:14,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb34ae557f2600f0f6ba HTTP/1.1" 200 None 2026-03-12 08:50:14,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:14,022 - lava_callback - DEBUG - Processing node 69afdb34ae557f2600f0f6ba: baseline-arm-baylibre 2026-03-12 08:50:14,022 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb34ae557f2600f0f6ba 2026-03-12 08:50:14,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:14,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:50:14,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb34ae557f2600f0f6ba HTTP/1.1" 200 None 2026-03-12 08:50:14,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:14,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:14,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:50:14,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:14,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d1c4bce73bcf0234247 HTTP/1.1" 200 None 2026-03-12 08:50:14,085 - lava_callback - INFO - Completed processing callback for node 69b27d1c4bce73bcf0234247 2026-03-12 08:50:14,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb34ae557f2600f0f6ba HTTP/1.1" 200 None 2026-03-12 08:50:14,144 - lava_callback - ERROR - Error processing callback for node 69afdb34ae557f2600f0f6ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35770 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:50:21,312 - lava_callback - INFO - Received callback for node 69b27e474bce73bcf02343e8 INFO: 172.20.0.1:57976 - "POST /node/69b27e474bce73bcf02343e8 HTTP/1.0" 202 Accepted 2026-03-12 08:50:21,318 - lava_callback - INFO - Starting async job submit for node 69b27e474bce73bcf02343e8 2026-03-12 08:50:21,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:21,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27e474bce73bcf02343e8 HTTP/1.1" 200 None 2026-03-12 08:50:21,336 - lava_callback - DEBUG - Processing node 69b27e474bce73bcf02343e8: baseline-arm 2026-03-12 08:50:21,342 - lava_callback - WARNING - Failed to upload log 2026-03-12 08:50:21,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:50:21,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:50:21,467 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b27e474bce73bcf02343e8/lava-logs.yaml 2026-03-12 08:50:21,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:50:21,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:50:21,586 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b27e474bce73bcf02343e8/lava_callback.json.gz 2026-03-12 08:50:21,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:21,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27e474bce73bcf02343e8 HTTP/1.1" 200 None 2026-03-12 08:50:21,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:21,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:50:21,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:21,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27e474bce73bcf02343e8 HTTP/1.1" 200 None 2026-03-12 08:50:21,648 - lava_callback - INFO - Completed processing callback for node 69b27e474bce73bcf02343e8 INFO: 172.20.0.1:57986 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:50:33,379 - lava_callback - INFO - Received callback for node 69b27d3d4bce73bcf02342a5 INFO: 172.20.0.1:40110 - "POST /node/69b27d3d4bce73bcf02342a5 HTTP/1.0" 202 Accepted 2026-03-12 08:50:33,388 - lava_callback - INFO - Starting async job submit for node 69b27d3d4bce73bcf02342a5 2026-03-12 08:50:33,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:33,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d3d4bce73bcf02342a5 HTTP/1.1" 200 None 2026-03-12 08:50:33,431 - lava_callback - DEBUG - Processing node 69b27d3d4bce73bcf02342a5: ltp-cve 2026-03-12 08:50:33,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:50:33,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:50:33,970 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69b27d3d4bce73bcf02342a5/log.txt.gz 2026-03-12 08:50:34,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:50:34,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:50:34,448 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69b27d3d4bce73bcf02342a5/lava-logs.yaml 2026-03-12 08:50:34,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:50:34,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:50:34,616 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69b27d3d4bce73bcf02342a5/lava_callback.json.gz 2026-03-12 08:50:34,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:34,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d3d4bce73bcf02342a5 HTTP/1.1" 200 None 2026-03-12 08:50:34,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:34,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:50:34,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:34,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d3d4bce73bcf02342a5 HTTP/1.1" 200 None 2026-03-12 08:50:34,720 - lava_callback - INFO - Completed processing callback for node 69b27d3d4bce73bcf02342a5 INFO: 172.20.0.1:40122 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:50:42,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:42,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-12 08:50:42,681 - lava_callback - INFO - Received callback for node 69afdb4bae557f2600f0f704 INFO: 172.20.0.1:46456 - "POST /node/69afdb4bae557f2600f0f704 HTTP/1.0" 202 Accepted 2026-03-12 08:50:42,686 - lava_callback - INFO - Starting async job submit for node 69afdb4bae557f2600f0f704 2026-03-12 08:50:42,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:42,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb4bae557f2600f0f704 HTTP/1.1" 200 None 2026-03-12 08:50:42,701 - lava_callback - DEBUG - Processing node 69afdb4bae557f2600f0f704: baseline-arm-clabbe 2026-03-12 08:50:42,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb4bae557f2600f0f704 2026-03-12 08:50:42,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:42,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb4bae557f2600f0f704 HTTP/1.1" 200 None 2026-03-12 08:50:42,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:42,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:50:42,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:42,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb4bae557f2600f0f704 HTTP/1.1" 200 None 2026-03-12 08:50:42,785 - lava_callback - ERROR - Error processing callback for node 69afdb4bae557f2600f0f704: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:50:43,302 - lava_callback - INFO - Received callback for node 69afdb57ae557f2600f0f725 INFO: 172.20.0.1:46466 - "POST /node/69afdb57ae557f2600f0f725 HTTP/1.0" 202 Accepted 2026-03-12 08:50:43,308 - lava_callback - INFO - Starting async job submit for node 69afdb57ae557f2600f0f725 2026-03-12 08:50:43,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:43,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb57ae557f2600f0f725 HTTP/1.1" 200 None 2026-03-12 08:50:43,324 - lava_callback - DEBUG - Processing node 69afdb57ae557f2600f0f725: baseline-arm-clabbe 2026-03-12 08:50:43,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb57ae557f2600f0f725 2026-03-12 08:50:43,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:43,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb57ae557f2600f0f725 HTTP/1.1" 200 None 2026-03-12 08:50:43,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:43,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:50:43,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:43,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb57ae557f2600f0f725 HTTP/1.1" 200 None 2026-03-12 08:50:43,383 - lava_callback - ERROR - Error processing callback for node 69afdb57ae557f2600f0f725: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:50:43,928 - lava_callback - INFO - Received callback for node 69afdb5aae557f2600f0f72e INFO: 172.20.0.1:46474 - "POST /node/69afdb5aae557f2600f0f72e HTTP/1.0" 202 Accepted 2026-03-12 08:50:43,933 - lava_callback - INFO - Starting async job submit for node 69afdb5aae557f2600f0f72e 2026-03-12 08:50:43,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:43,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb5aae557f2600f0f72e HTTP/1.1" 200 None 2026-03-12 08:50:43,952 - lava_callback - DEBUG - Processing node 69afdb5aae557f2600f0f72e: baseline-arm-clabbe 2026-03-12 08:50:43,953 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb5aae557f2600f0f72e 2026-03-12 08:50:43,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:43,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb5aae557f2600f0f72e HTTP/1.1" 200 None 2026-03-12 08:50:43,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:43,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:50:43,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:44,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb5aae557f2600f0f72e HTTP/1.1" 200 None 2026-03-12 08:50:44,005 - lava_callback - ERROR - Error processing callback for node 69afdb5aae557f2600f0f72e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:50:44,573 - lava_callback - INFO - Received callback for node 69afdb5cae557f2600f0f731 INFO: 172.20.0.1:46478 - "POST /node/69afdb5cae557f2600f0f731 HTTP/1.0" 202 Accepted 2026-03-12 08:50:44,579 - lava_callback - INFO - Starting async job submit for node 69afdb5cae557f2600f0f731 2026-03-12 08:50:44,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:44,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb5cae557f2600f0f731 HTTP/1.1" 200 None 2026-03-12 08:50:44,594 - lava_callback - DEBUG - Processing node 69afdb5cae557f2600f0f731: baseline-arm-clabbe 2026-03-12 08:50:44,594 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb5cae557f2600f0f731 2026-03-12 08:50:44,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:44,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb5cae557f2600f0f731 HTTP/1.1" 200 None 2026-03-12 08:50:44,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:44,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:50:44,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:44,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb5cae557f2600f0f731 HTTP/1.1" 200 None 2026-03-12 08:50:44,665 - lava_callback - ERROR - Error processing callback for node 69afdb5cae557f2600f0f731: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46480 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:50:53,339 - lava_callback - INFO - Received callback for node 69afdb5aae557f2600f0f72b INFO: 172.20.0.1:56788 - "POST /node/69afdb5aae557f2600f0f72b HTTP/1.0" 202 Accepted 2026-03-12 08:50:53,344 - lava_callback - INFO - Starting async job submit for node 69afdb5aae557f2600f0f72b 2026-03-12 08:50:53,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:53,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb5aae557f2600f0f72b HTTP/1.1" 200 None 2026-03-12 08:50:53,360 - lava_callback - DEBUG - Processing node 69afdb5aae557f2600f0f72b: baseline-arm64-baylibre 2026-03-12 08:50:53,360 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb5aae557f2600f0f72b 2026-03-12 08:50:53,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:53,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb5aae557f2600f0f72b HTTP/1.1" 200 None 2026-03-12 08:50:53,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:53,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 08:50:53,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:50:53,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb5aae557f2600f0f72b HTTP/1.1" 200 None 2026-03-12 08:50:53,418 - lava_callback - ERROR - Error processing callback for node 69afdb5aae557f2600f0f72b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56794 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:51:02,634 - lava_callback - INFO - Received callback for node 69afdb64ae557f2600f0f73f INFO: 172.20.0.1:54736 - "POST /node/69afdb64ae557f2600f0f73f HTTP/1.0" 202 Accepted 2026-03-12 08:51:02,639 - lava_callback - INFO - Starting async job submit for node 69afdb64ae557f2600f0f73f 2026-03-12 08:51:02,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:02,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb64ae557f2600f0f73f HTTP/1.1" 200 None 2026-03-12 08:51:02,658 - lava_callback - DEBUG - Processing node 69afdb64ae557f2600f0f73f: baseline-arm-clabbe 2026-03-12 08:51:02,658 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb64ae557f2600f0f73f 2026-03-12 08:51:02,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:02,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb64ae557f2600f0f73f HTTP/1.1" 200 None 2026-03-12 08:51:02,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:02,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 08:51:02,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:02,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb64ae557f2600f0f73f HTTP/1.1" 200 None 2026-03-12 08:51:02,712 - lava_callback - ERROR - Error processing callback for node 69afdb64ae557f2600f0f73f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47800 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:51:33,240 - lava_callback - INFO - Received callback for node 69b27d3f4bce73bcf02342ab INFO: 172.20.0.1:40012 - "POST /node/69b27d3f4bce73bcf02342ab HTTP/1.0" 202 Accepted 2026-03-12 08:51:33,246 - lava_callback - INFO - Starting async job submit for node 69b27d3f4bce73bcf02342ab 2026-03-12 08:51:33,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:33,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d3f4bce73bcf02342ab HTTP/1.1" 200 None 2026-03-12 08:51:33,286 - lava_callback - DEBUG - Processing node 69b27d3f4bce73bcf02342ab: ltp-fs_bind 2026-03-12 08:51:33,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:51:33,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:51:33,842 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b27d3f4bce73bcf02342ab/log.txt.gz 2026-03-12 08:51:34,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:51:34,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:51:34,269 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b27d3f4bce73bcf02342ab/lava-logs.yaml 2026-03-12 08:51:34,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:51:34,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:51:34,438 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69b27d3f4bce73bcf02342ab/lava_callback.json.gz 2026-03-12 08:51:34,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:34,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d3f4bce73bcf02342ab HTTP/1.1" 200 None 2026-03-12 08:51:34,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:34,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:51:34,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:34,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d3f4bce73bcf02342ab HTTP/1.1" 200 None 2026-03-12 08:51:34,535 - lava_callback - INFO - Completed processing callback for node 69b27d3f4bce73bcf02342ab INFO: 172.20.0.1:40014 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:51:42,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:42,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51096 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:51:52,070 - lava_callback - INFO - Received callback for node 69b27d1d4bce73bcf023424a INFO: 172.20.0.1:37864 - "POST /node/69b27d1d4bce73bcf023424a HTTP/1.0" 202 Accepted 2026-03-12 08:51:52,076 - lava_callback - INFO - Starting async job submit for node 69b27d1d4bce73bcf023424a 2026-03-12 08:51:52,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:52,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d1d4bce73bcf023424a HTTP/1.1" 200 None 2026-03-12 08:51:52,113 - lava_callback - DEBUG - Processing node 69b27d1d4bce73bcf023424a: baseline-arm-clabbe 2026-03-12 08:51:52,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:51:52,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:51:52,459 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d1d4bce73bcf023424a/log.txt.gz 2026-03-12 08:51:52,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:51:52,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:51:52,773 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d1d4bce73bcf023424a/lava-logs.yaml 2026-03-12 08:51:52,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:51:52,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:51:52,922 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69b27d1d4bce73bcf023424a/lava_callback.json.gz 2026-03-12 08:51:52,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:52,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d1d4bce73bcf023424a HTTP/1.1" 200 None 2026-03-12 08:51:52,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:52,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:51:52,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:53,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d1d4bce73bcf023424a HTTP/1.1" 200 None 2026-03-12 08:51:53,012 - lava_callback - INFO - Completed processing callback for node 69b27d1d4bce73bcf023424a 2026-03-12 08:51:53,537 - lava_callback - INFO - Received callback for node 69afdb9eae557f2600f0f7c4 INFO: 172.20.0.1:37876 - "POST /node/69afdb9eae557f2600f0f7c4 HTTP/1.0" 202 Accepted 2026-03-12 08:51:53,543 - lava_callback - INFO - Starting async job submit for node 69afdb9eae557f2600f0f7c4 2026-03-12 08:51:53,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:53,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb9eae557f2600f0f7c4 HTTP/1.1" 200 None 2026-03-12 08:51:53,561 - lava_callback - DEBUG - Processing node 69afdb9eae557f2600f0f7c4: baseline-arm-baylibre 2026-03-12 08:51:53,561 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdb9eae557f2600f0f7c4 2026-03-12 08:51:53,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:53,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdb9eae557f2600f0f7c4 HTTP/1.1" 200 None 2026-03-12 08:51:53,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:53,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 08:51:53,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:51:53,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdb9eae557f2600f0f7c4 HTTP/1.1" 200 None 2026-03-12 08:51:53,627 - lava_callback - ERROR - Error processing callback for node 69afdb9eae557f2600f0f7c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34786 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:52:12,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:52:12,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34774 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:53:07,791 - lava_callback - INFO - Received callback for node 69b27d374bce73bcf0234298 INFO: 172.20.0.1:34788 - "POST /node/69b27d374bce73bcf0234298 HTTP/1.0" 202 Accepted 2026-03-12 08:53:07,797 - lava_callback - INFO - Starting async job submit for node 69b27d374bce73bcf0234298 2026-03-12 08:53:07,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:53:07,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d374bce73bcf0234298 HTTP/1.1" 200 None 2026-03-12 08:53:07,846 - lava_callback - DEBUG - Processing node 69b27d374bce73bcf0234298: ltp-cap_bounds 2026-03-12 08:53:08,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:53:08,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:53:08,425 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b27d374bce73bcf0234298/log.txt.gz 2026-03-12 08:53:08,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:53:08,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:53:08,874 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b27d374bce73bcf0234298/lava-logs.yaml 2026-03-12 08:53:08,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:53:09,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:53:09,063 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69b27d374bce73bcf0234298/lava_callback.json.gz 2026-03-12 08:53:09,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:53:09,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d374bce73bcf0234298 HTTP/1.1" 200 None 2026-03-12 08:53:09,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:53:09,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:53:09,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:53:09,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d374bce73bcf0234298 HTTP/1.1" 200 None 2026-03-12 08:53:09,209 - lava_callback - INFO - Completed processing callback for node 69b27d374bce73bcf0234298 2026-03-12 08:53:12,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:53:12,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34450 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:53:53,334 - lava_callback - INFO - Received callback for node 69b27d394bce73bcf023429b INFO: 172.20.0.1:41480 - "POST /node/69b27d394bce73bcf023429b HTTP/1.0" 202 Accepted 2026-03-12 08:53:53,340 - lava_callback - INFO - Starting async job submit for node 69b27d394bce73bcf023429b 2026-03-12 08:53:53,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:53:53,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d394bce73bcf023429b HTTP/1.1" 200 None 2026-03-12 08:53:53,379 - lava_callback - DEBUG - Processing node 69b27d394bce73bcf023429b: ltp-containers 2026-03-12 08:53:53,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:41496 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:53:53,874 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:53:53,876 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69b27d394bce73bcf023429b/log.txt.gz 2026-03-12 08:53:54,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:53:54,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:53:54,286 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69b27d394bce73bcf023429b/lava-logs.yaml 2026-03-12 08:53:54,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:53:54,442 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:53:54,444 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69b27d394bce73bcf023429b/lava_callback.json.gz 2026-03-12 08:53:54,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:53:54,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d394bce73bcf023429b HTTP/1.1" 200 None 2026-03-12 08:53:54,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:53:54,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:53:54,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:53:54,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d394bce73bcf023429b HTTP/1.1" 200 None 2026-03-12 08:53:54,541 - lava_callback - INFO - Completed processing callback for node 69b27d394bce73bcf023429b INFO: 172.20.0.1:41512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50056 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:54:12,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:54:12,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-12 08:54:13,357 - lava_callback - INFO - Received callback for node 69b27d414bce73bcf02342b0 INFO: 172.20.0.1:56120 - "POST /node/69b27d414bce73bcf02342b0 HTTP/1.0" 202 Accepted 2026-03-12 08:54:13,364 - lava_callback - INFO - Starting async job submit for node 69b27d414bce73bcf02342b0 2026-03-12 08:54:13,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:54:13,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d414bce73bcf02342b0 HTTP/1.1" 200 None 2026-03-12 08:54:13,405 - lava_callback - DEBUG - Processing node 69b27d414bce73bcf02342b0: ltp-fsx 2026-03-12 08:54:13,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:54:13,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:54:13,889 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69b27d414bce73bcf02342b0/log.txt.gz 2026-03-12 08:54:14,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:54:14,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:54:14,331 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69b27d414bce73bcf02342b0/lava-logs.yaml 2026-03-12 08:54:14,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:54:14,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:54:14,494 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69b27d414bce73bcf02342b0/lava_callback.json.gz 2026-03-12 08:54:14,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:54:14,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d414bce73bcf02342b0 HTTP/1.1" 200 None 2026-03-12 08:54:14,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:54:14,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:54:14,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:54:14,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d414bce73bcf02342b0 HTTP/1.1" 200 None 2026-03-12 08:54:14,583 - lava_callback - INFO - Completed processing callback for node 69b27d414bce73bcf02342b0 INFO: 172.20.0.1:56128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37628 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:54:42,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:54:42,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50510 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:55:33,363 - lava_callback - INFO - Received callback for node 69b27d424bce73bcf02342b4 INFO: 172.20.0.1:32986 - "POST /node/69b27d424bce73bcf02342b4 HTTP/1.0" 202 Accepted 2026-03-12 08:55:33,373 - lava_callback - INFO - Starting async job submit for node 69b27d424bce73bcf02342b4 2026-03-12 08:55:33,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:55:33,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d424bce73bcf02342b4 HTTP/1.1" 200 None 2026-03-12 08:55:33,439 - lava_callback - DEBUG - Processing node 69b27d424bce73bcf02342b4: ltp-input 2026-03-12 08:55:34,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:55:34,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:55:34,234 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69b27d424bce73bcf02342b4/log.txt.gz 2026-03-12 08:55:34,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:55:34,847 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:55:34,849 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69b27d424bce73bcf02342b4/lava-logs.yaml 2026-03-12 08:55:34,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:55:35,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:55:35,039 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69b27d424bce73bcf02342b4/lava_callback.json.gz 2026-03-12 08:55:35,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:55:35,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d424bce73bcf02342b4 HTTP/1.1" 200 None 2026-03-12 08:55:35,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:55:35,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:55:35,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:55:35,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d424bce73bcf02342b4 HTTP/1.1" 200 None 2026-03-12 08:55:35,124 - lava_callback - INFO - Completed processing callback for node 69b27d424bce73bcf02342b4 INFO: 172.20.0.1:33002 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:55:42,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:55:42,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38666 - "GET /favicon.ico HTTP/1.0" 404 Not Found 2026-03-12 08:55:53,386 - lava_callback - INFO - Received callback for node 69b27d444bce73bcf02342b7 INFO: 172.20.0.1:38674 - "POST /node/69b27d444bce73bcf02342b7 HTTP/1.0" 202 Accepted 2026-03-12 08:55:53,392 - lava_callback - INFO - Starting async job submit for node 69b27d444bce73bcf02342b7 2026-03-12 08:55:53,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:55:53,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d444bce73bcf02342b7 HTTP/1.1" 200 None 2026-03-12 08:55:53,439 - lava_callback - DEBUG - Processing node 69b27d444bce73bcf02342b7: ltp-ipc 2026-03-12 08:55:53,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:38676 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:55:53,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:55:53,964 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69b27d444bce73bcf02342b7/log.txt.gz 2026-03-12 08:55:54,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:55:54,356 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:55:54,358 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69b27d444bce73bcf02342b7/lava-logs.yaml 2026-03-12 08:55:54,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:55:54,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:55:54,520 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69b27d444bce73bcf02342b7/lava_callback.json.gz 2026-03-12 08:55:54,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:55:54,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d444bce73bcf02342b7 HTTP/1.1" 200 None 2026-03-12 08:55:54,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:55:54,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:55:54,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:55:54,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d444bce73bcf02342b7 HTTP/1.1" 200 None 2026-03-12 08:55:54,606 - lava_callback - INFO - Completed processing callback for node 69b27d444bce73bcf02342b7 INFO: 172.20.0.1:38688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51992 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:56:12,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:56:12,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:56:33,671 - lava_callback - INFO - Received callback for node 69afdcb3ae557f2600f0f8ab INFO: 172.20.0.1:54064 - "POST /node/69afdcb3ae557f2600f0f8ab HTTP/1.0" 202 Accepted 2026-03-12 08:56:33,677 - lava_callback - INFO - Starting async job submit for node 69afdcb3ae557f2600f0f8ab 2026-03-12 08:56:33,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:56:33,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdcb3ae557f2600f0f8ab HTTP/1.1" 200 None 2026-03-12 08:56:33,693 - lava_callback - DEBUG - Processing node 69afdcb3ae557f2600f0f8ab: baseline-arm-baylibre 2026-03-12 08:56:33,694 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdcb3ae557f2600f0f8ab 2026-03-12 08:56:33,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:56:33,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdcb3ae557f2600f0f8ab HTTP/1.1" 200 None 2026-03-12 08:56:33,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:56:33,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 08:56:33,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:56:33,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdcb3ae557f2600f0f8ab HTTP/1.1" 200 None 2026-03-12 08:56:33,756 - lava_callback - ERROR - Error processing callback for node 69afdcb3ae557f2600f0f8ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43326 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:57:07,483 - lava_callback - INFO - Received callback for node 69b27d454bce73bcf02342ba INFO: 172.20.0.1:43336 - "POST /node/69b27d454bce73bcf02342ba HTTP/1.0" 202 Accepted 2026-03-12 08:57:07,489 - lava_callback - INFO - Starting async job submit for node 69b27d454bce73bcf02342ba 2026-03-12 08:57:07,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:07,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d454bce73bcf02342ba HTTP/1.1" 200 None 2026-03-12 08:57:07,528 - lava_callback - DEBUG - Processing node 69b27d454bce73bcf02342ba: ltp-pty 2026-03-12 08:57:07,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:57:08,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:57:08,010 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69b27d454bce73bcf02342ba/log.txt.gz 2026-03-12 08:57:08,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:57:08,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:57:08,537 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69b27d454bce73bcf02342ba/lava-logs.yaml 2026-03-12 08:57:08,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:57:08,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:57:08,706 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69b27d454bce73bcf02342ba/lava_callback.json.gz 2026-03-12 08:57:08,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:08,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d454bce73bcf02342ba HTTP/1.1" 200 None 2026-03-12 08:57:08,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:08,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:57:08,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:08,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d454bce73bcf02342ba HTTP/1.1" 200 None 2026-03-12 08:57:08,789 - lava_callback - INFO - Completed processing callback for node 69b27d454bce73bcf02342ba 2026-03-12 08:57:12,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:12,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-12 08:57:13,678 - lava_callback - INFO - Received callback for node 69afdcdcae557f2600f0f8bb INFO: 172.20.0.1:53492 - "POST /node/69afdcdcae557f2600f0f8bb HTTP/1.0" 202 Accepted 2026-03-12 08:57:13,683 - lava_callback - INFO - Starting async job submit for node 69afdcdcae557f2600f0f8bb 2026-03-12 08:57:13,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:13,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdcdcae557f2600f0f8bb HTTP/1.1" 200 None 2026-03-12 08:57:13,699 - lava_callback - DEBUG - Processing node 69afdcdcae557f2600f0f8bb: baseline-arm-baylibre 2026-03-12 08:57:13,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdcdcae557f2600f0f8bb 2026-03-12 08:57:13,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:13,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdcdcae557f2600f0f8bb HTTP/1.1" 200 None 2026-03-12 08:57:13,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:13,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 08:57:13,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:13,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdcdcae557f2600f0f8bb HTTP/1.1" 200 None 2026-03-12 08:57:13,750 - lava_callback - ERROR - Error processing callback for node 69afdcdcae557f2600f0f8bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54158 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:57:53,736 - lava_callback - INFO - Received callback for node 69afdd07ae557f2600f0f8d4 INFO: 172.20.0.1:33946 - "POST /node/69afdd07ae557f2600f0f8d4 HTTP/1.0" 202 Accepted 2026-03-12 08:57:53,745 - lava_callback - INFO - Starting async job submit for node 69afdd07ae557f2600f0f8d4 2026-03-12 08:57:53,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:53,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdd07ae557f2600f0f8d4 HTTP/1.1" 200 None 2026-03-12 08:57:53,773 - lava_callback - DEBUG - Processing node 69afdd07ae557f2600f0f8d4: baseline-arm-baylibre 2026-03-12 08:57:53,773 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdd07ae557f2600f0f8d4 2026-03-12 08:57:53,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:53,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdd07ae557f2600f0f8d4 HTTP/1.1" 200 None 2026-03-12 08:57:53,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:53,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 08:57:53,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:57:53,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdd07ae557f2600f0f8d4 HTTP/1.1" 200 None 2026-03-12 08:57:53,858 - lava_callback - ERROR - Error processing callback for node 69afdd07ae557f2600f0f8d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57034 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:58:33,714 - lava_callback - INFO - Received callback for node 69b27d484bce73bcf02342c0 INFO: 172.20.0.1:49146 - "POST /node/69b27d484bce73bcf02342c0 HTTP/1.0" 202 Accepted 2026-03-12 08:58:33,722 - lava_callback - INFO - Starting async job submit for node 69b27d484bce73bcf02342c0 2026-03-12 08:58:33,730 - lava_callback - INFO - Received callback for node 69afdd31ae557f2600f0f8ef INFO: 172.20.0.1:49158 - "POST /node/69afdd31ae557f2600f0f8ef HTTP/1.0" 202 Accepted 2026-03-12 08:58:33,745 - lava_callback - INFO - Starting async job submit for node 69afdd31ae557f2600f0f8ef 2026-03-12 08:58:33,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:58:33,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:58:33,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d484bce73bcf02342c0 HTTP/1.1" 200 None 2026-03-12 08:58:33,793 - lava_callback - DEBUG - Processing node 69b27d484bce73bcf02342c0: ltp-smoke 2026-03-12 08:58:33,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdd31ae557f2600f0f8ef HTTP/1.1" 200 None 2026-03-12 08:58:33,818 - lava_callback - DEBUG - Processing node 69afdd31ae557f2600f0f8ef: baseline-arm-baylibre 2026-03-12 08:58:33,819 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdd31ae557f2600f0f8ef 2026-03-12 08:58:33,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:58:33,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdd31ae557f2600f0f8ef HTTP/1.1" 200 None 2026-03-12 08:58:33,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:58:34,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 08:58:34,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:58:34,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdd31ae557f2600f0f8ef HTTP/1.1" 200 None 2026-03-12 08:58:34,153 - lava_callback - ERROR - Error processing callback for node 69afdd31ae557f2600f0f8ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 08:58:34,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:58:34,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:58:34,628 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b27d484bce73bcf02342c0/log.txt.gz 2026-03-12 08:58:34,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:58:35,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:58:35,090 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b27d484bce73bcf02342c0/lava-logs.yaml 2026-03-12 08:58:35,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:58:35,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:58:35,277 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b27d484bce73bcf02342c0/lava_callback.json.gz 2026-03-12 08:58:35,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:58:35,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d484bce73bcf02342c0 HTTP/1.1" 200 None 2026-03-12 08:58:35,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:58:35,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 08:58:35,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:58:35,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d484bce73bcf02342c0 HTTP/1.1" 200 None 2026-03-12 08:58:35,392 - lava_callback - INFO - Completed processing callback for node 69b27d484bce73bcf02342c0 INFO: 172.20.0.1:49160 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:58:42,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:58:42,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50568 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:59:13,788 - lava_callback - INFO - Received callback for node 69afdd5aae557f2600f0f90d INFO: 172.20.0.1:58832 - "POST /node/69afdd5aae557f2600f0f90d HTTP/1.0" 202 Accepted 2026-03-12 08:59:13,797 - lava_callback - INFO - Starting async job submit for node 69afdd5aae557f2600f0f90d 2026-03-12 08:59:13,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:13,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdd5aae557f2600f0f90d HTTP/1.1" 200 None 2026-03-12 08:59:13,821 - lava_callback - DEBUG - Processing node 69afdd5aae557f2600f0f90d: baseline-arm-baylibre 2026-03-12 08:59:13,822 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdd5aae557f2600f0f90d 2026-03-12 08:59:13,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:13,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdd5aae557f2600f0f90d HTTP/1.1" 200 None 2026-03-12 08:59:13,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:13,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 08:59:13,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:13,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdd5aae557f2600f0f90d HTTP/1.1" 200 None 2026-03-12 08:59:13,912 - lava_callback - ERROR - Error processing callback for node 69afdd5aae557f2600f0f90d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55892 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:59:43,498 - lava_callback - INFO - Received callback for node 69b2805c4bce73bcf02344c9 INFO: 172.20.0.1:51872 - "POST /node/69b2805c4bce73bcf02344c9 HTTP/1.0" 202 Accepted 2026-03-12 08:59:43,504 - lava_callback - INFO - Starting async job submit for node 69b2805c4bce73bcf02344c9 2026-03-12 08:59:43,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:43,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2805c4bce73bcf02344c9 HTTP/1.1" 200 None 2026-03-12 08:59:43,556 - lava_callback - DEBUG - Processing node 69b2805c4bce73bcf02344c9: baseline-x86 2026-03-12 08:59:43,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:59:43,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:59:43,964 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805c4bce73bcf02344c9/log.txt.gz 2026-03-12 08:59:44,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:59:44,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:59:44,317 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805c4bce73bcf02344c9/lava-logs.yaml 2026-03-12 08:59:44,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:59:44,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:59:44,476 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805c4bce73bcf02344c9/lava_callback.json.gz 2026-03-12 08:59:44,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:44,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2805c4bce73bcf02344c9 HTTP/1.1" 200 None 2026-03-12 08:59:44,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:44,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2768d4bce73bcf0233d3d HTTP/1.1" 200 None 2026-03-12 08:59:44,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:44,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2805c4bce73bcf02344c9 HTTP/1.1" 200 None 2026-03-12 08:59:44,568 - lava_callback - INFO - Completed processing callback for node 69b2805c4bce73bcf02344c9 INFO: 172.20.0.1:51886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41486 - "GET / HTTP/1.0" 200 OK 2026-03-12 08:59:57,331 - lava_callback - INFO - Received callback for node 69b2805e4bce73bcf02344d2 INFO: 172.20.0.1:41494 - "POST /node/69b2805e4bce73bcf02344d2 HTTP/1.0" 202 Accepted 2026-03-12 08:59:57,337 - lava_callback - INFO - Starting async job submit for node 69b2805e4bce73bcf02344d2 2026-03-12 08:59:57,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:57,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2805e4bce73bcf02344d2 HTTP/1.1" 200 None 2026-03-12 08:59:57,385 - lava_callback - DEBUG - Processing node 69b2805e4bce73bcf02344d2: baseline-x86 2026-03-12 08:59:57,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:59:57,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:59:57,713 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805e4bce73bcf02344d2/log.txt.gz 2026-03-12 08:59:57,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:59:58,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:59:58,084 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805e4bce73bcf02344d2/lava-logs.yaml 2026-03-12 08:59:58,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 08:59:58,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 08:59:58,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805e4bce73bcf02344d2/lava_callback.json.gz 2026-03-12 08:59:58,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:58,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2805e4bce73bcf02344d2 HTTP/1.1" 200 None 2026-03-12 08:59:58,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:58,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2768d4bce73bcf0233d3d HTTP/1.1" 200 None 2026-03-12 08:59:58,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 08:59:58,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2805e4bce73bcf02344d2 HTTP/1.1" 200 None 2026-03-12 08:59:58,795 - lava_callback - INFO - Completed processing callback for node 69b2805e4bce73bcf02344d2 2026-03-12 09:00:03,323 - lava_callback - INFO - Received callback for node 69afdd83ae557f2600f0f921 INFO: 172.20.0.1:50732 - "POST /node/69afdd83ae557f2600f0f921 HTTP/1.0" 202 Accepted 2026-03-12 09:00:03,333 - lava_callback - INFO - Starting async job submit for node 69afdd83ae557f2600f0f921 2026-03-12 09:00:03,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:03,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdd83ae557f2600f0f921 HTTP/1.1" 200 None 2026-03-12 09:00:03,352 - lava_callback - DEBUG - Processing node 69afdd83ae557f2600f0f921: baseline-arm-clabbe 2026-03-12 09:00:03,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdd83ae557f2600f0f921 2026-03-12 09:00:03,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:03,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdd83ae557f2600f0f921 HTTP/1.1" 200 None 2026-03-12 09:00:03,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:03,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 09:00:03,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:03,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdd83ae557f2600f0f921 HTTP/1.1" 200 None 2026-03-12 09:00:03,410 - lava_callback - ERROR - Error processing callback for node 69afdd83ae557f2600f0f921: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50738 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:00:12,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:12,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-12 09:00:13,979 - lava_callback - INFO - Received callback for node 69b27d4d4bce73bcf02342e4 INFO: 172.20.0.1:51418 - "POST /node/69b27d4d4bce73bcf02342e4 HTTP/1.0" 202 Accepted 2026-03-12 09:00:13,989 - lava_callback - INFO - Starting async job submit for node 69b27d4d4bce73bcf02342e4 2026-03-12 09:00:14,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:14,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d4d4bce73bcf02342e4 HTTP/1.1" 200 None 2026-03-12 09:00:14,068 - lava_callback - DEBUG - Processing node 69b27d4d4bce73bcf02342e4: ltp-timers 2026-03-12 09:00:14,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:14,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:14,914 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69b27d4d4bce73bcf02342e4/log.txt.gz INFO: 172.20.0.1:51434 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 09:00:15,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:15,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:15,380 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69b27d4d4bce73bcf02342e4/lava-logs.yaml 2026-03-12 09:00:15,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:15,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:15,759 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69b27d4d4bce73bcf02342e4/lava_callback.json.gz 2026-03-12 09:00:15,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:15,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d4d4bce73bcf02342e4 HTTP/1.1" 200 None 2026-03-12 09:00:15,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:15,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 09:00:15,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:15,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d4d4bce73bcf02342e4 HTTP/1.1" 200 None 2026-03-12 09:00:15,897 - lava_callback - INFO - Completed processing callback for node 69b27d4d4bce73bcf02342e4 INFO: 172.20.0.1:51446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59776 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:00:34,792 - lava_callback - INFO - Received callback for node 69b27d4b4bce73bcf02342c7 INFO: 172.20.0.1:46142 - "POST /node/69b27d4b4bce73bcf02342c7 HTTP/1.0" 202 Accepted 2026-03-12 09:00:34,801 - lava_callback - INFO - Starting async job submit for node 69b27d4b4bce73bcf02342c7 2026-03-12 09:00:34,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:34,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d4b4bce73bcf02342c7 HTTP/1.1" 200 None 2026-03-12 09:00:34,843 - lava_callback - DEBUG - Processing node 69b27d4b4bce73bcf02342c7: ltp-syscalls-ipc 2026-03-12 09:00:35,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:35,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:35,644 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b27d4b4bce73bcf02342c7/log.txt.gz 2026-03-12 09:00:35,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:36,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:36,350 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b27d4b4bce73bcf02342c7/lava-logs.yaml 2026-03-12 09:00:36,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:36,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:36,509 - lava_callback - INFO - Received callback for node 69b2805d4bce73bcf02344cf INFO: 172.20.0.1:46158 - "POST /node/69b2805d4bce73bcf02344cf HTTP/1.0" 202 Accepted 2026-03-12 09:00:36,517 - lava_callback - INFO - Starting async job submit for node 69b2805d4bce73bcf02344cf INFO: 172.20.0.1:46174 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:00:36,538 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69b27d4b4bce73bcf02342c7/lava_callback.json.gz 2026-03-12 09:00:36,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:36,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:36,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2805d4bce73bcf02344cf HTTP/1.1" 200 None 2026-03-12 09:00:36,584 - lava_callback - DEBUG - Processing node 69b2805d4bce73bcf02344cf: baseline-x86 2026-03-12 09:00:36,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d4b4bce73bcf02342c7 HTTP/1.1" 200 None 2026-03-12 09:00:36,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:36,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 09:00:36,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:36,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d4b4bce73bcf02342c7 HTTP/1.1" 200 None 2026-03-12 09:00:36,913 - lava_callback - INFO - Completed processing callback for node 69b27d4b4bce73bcf02342c7 2026-03-12 09:00:37,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:37,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:37,368 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805d4bce73bcf02344cf/log.txt.gz 2026-03-12 09:00:37,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:37,936 - lava_callback - INFO - Received callback for node 69b2805d4bce73bcf02344cc INFO: 172.20.0.1:46178 - "POST /node/69b2805d4bce73bcf02344cc HTTP/1.0" 202 Accepted 2026-03-12 09:00:37,947 - lava_callback - INFO - Starting async job submit for node 69b2805d4bce73bcf02344cc 2026-03-12 09:00:37,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:38,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2805d4bce73bcf02344cc HTTP/1.1" 200 None 2026-03-12 09:00:38,010 - lava_callback - DEBUG - Processing node 69b2805d4bce73bcf02344cc: baseline-x86 2026-03-12 09:00:38,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:38,108 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805d4bce73bcf02344cf/lava-logs.yaml 2026-03-12 09:00:38,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:38,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:38,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:38,432 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805d4bce73bcf02344cf/lava_callback.json.gz 2026-03-12 09:00:38,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:38,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2805d4bce73bcf02344cf HTTP/1.1" 200 None 2026-03-12 09:00:38,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:38,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2768d4bce73bcf0233d3d HTTP/1.1" 200 None 2026-03-12 09:00:38,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:38,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2805d4bce73bcf02344cf HTTP/1.1" 200 None 2026-03-12 09:00:38,575 - lava_callback - INFO - Completed processing callback for node 69b2805d4bce73bcf02344cf 2026-03-12 09:00:38,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:38,852 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805d4bce73bcf02344cc/log.txt.gz 2026-03-12 09:00:39,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:39,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:39,396 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805d4bce73bcf02344cc/lava-logs.yaml 2026-03-12 09:00:39,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:00:39,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:00:39,766 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69b2805d4bce73bcf02344cc/lava_callback.json.gz 2026-03-12 09:00:39,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:39,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2805d4bce73bcf02344cc HTTP/1.1" 200 None 2026-03-12 09:00:39,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:39,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2768d4bce73bcf0233d3d HTTP/1.1" 200 None 2026-03-12 09:00:39,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:39,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2805d4bce73bcf02344cc HTTP/1.1" 200 None 2026-03-12 09:00:39,883 - lava_callback - INFO - Completed processing callback for node 69b2805d4bce73bcf02344cc 2026-03-12 09:00:42,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:42,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-12 09:00:43,189 - lava_callback - INFO - Received callback for node 69afddabae557f2600f0f931 INFO: 172.20.0.1:58668 - "POST /node/69afddabae557f2600f0f931 HTTP/1.0" 202 Accepted 2026-03-12 09:00:43,198 - lava_callback - INFO - Starting async job submit for node 69afddabae557f2600f0f931 2026-03-12 09:00:43,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:43,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afddabae557f2600f0f931 HTTP/1.1" 200 None 2026-03-12 09:00:43,224 - lava_callback - DEBUG - Processing node 69afddabae557f2600f0f931: baseline-arm-clabbe 2026-03-12 09:00:43,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afddabae557f2600f0f931 2026-03-12 09:00:43,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:43,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afddabae557f2600f0f931 HTTP/1.1" 200 None 2026-03-12 09:00:43,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:43,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 09:00:43,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:43,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afddabae557f2600f0f931 HTTP/1.1" 200 None 2026-03-12 09:00:43,331 - lava_callback - ERROR - Error processing callback for node 69afddabae557f2600f0f931: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39844 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:00:59,423 - lava_callback - INFO - Received callback for node 69b27d064bce73bcf02341e9 INFO: 172.20.0.1:39848 - "POST /node/69b27d064bce73bcf02341e9 HTTP/1.0" 202 Accepted 2026-03-12 09:00:59,434 - lava_callback - INFO - Starting async job submit for node 69b27d064bce73bcf02341e9 2026-03-12 09:00:59,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:00:59,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d064bce73bcf02341e9 HTTP/1.1" 200 None 2026-03-12 09:00:59,517 - lava_callback - DEBUG - Processing node 69b27d064bce73bcf02341e9: baseline-arm 2026-03-12 09:01:00,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:01:00,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:01:00,511 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d064bce73bcf02341e9/log.txt.gz 2026-03-12 09:01:01,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:01:01,651 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:01:01,653 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d064bce73bcf02341e9/lava-logs.yaml 2026-03-12 09:01:01,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:01:01,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:01:01,836 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b27d064bce73bcf02341e9/lava_callback.json.gz DEBUG: Setup failed for 69b27d064bce73bcf02341e9. Transitting job node result: incomplete -> fail 2026-03-12 09:01:01,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:01,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d064bce73bcf02341e9 HTTP/1.1" 200 None 2026-03-12 09:01:01,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:01,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 09:01:01,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:02,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d064bce73bcf02341e9 HTTP/1.1" 200 None 2026-03-12 09:01:02,236 - lava_callback - INFO - Completed processing callback for node 69b27d064bce73bcf02341e9 INFO: 172.20.0.1:47358 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:01:07,708 - lava_callback - INFO - Received callback for node 69b27d4e4bce73bcf02342e9 INFO: 172.20.0.1:47372 - "POST /node/69b27d4e4bce73bcf02342e9 HTTP/1.0" 202 Accepted 2026-03-12 09:01:07,718 - lava_callback - INFO - Starting async job submit for node 69b27d4e4bce73bcf02342e9 2026-03-12 09:01:07,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:07,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d4e4bce73bcf02342e9 HTTP/1.1" 200 None 2026-03-12 09:01:07,795 - lava_callback - DEBUG - Processing node 69b27d4e4bce73bcf02342e9: ltp-watchqueue 2026-03-12 09:01:08,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:01:08,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:01:08,579 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b27d4e4bce73bcf02342e9/log.txt.gz 2026-03-12 09:01:08,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:01:09,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:01:09,300 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b27d4e4bce73bcf02342e9/lava-logs.yaml 2026-03-12 09:01:09,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:01:09,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:01:09,454 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69b27d4e4bce73bcf02342e9/lava_callback.json.gz 2026-03-12 09:01:09,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:09,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d4e4bce73bcf02342e9 HTTP/1.1" 200 None 2026-03-12 09:01:09,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:09,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 09:01:09,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:09,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d4e4bce73bcf02342e9 HTTP/1.1" 200 None 2026-03-12 09:01:09,546 - lava_callback - INFO - Completed processing callback for node 69b27d4e4bce73bcf02342e9 2026-03-12 09:01:12,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:12,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44206 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:01:23,281 - lava_callback - INFO - Received callback for node 69afddd3ae557f2600f0f944 INFO: 172.20.0.1:52172 - "POST /node/69afddd3ae557f2600f0f944 HTTP/1.0" 202 Accepted 2026-03-12 09:01:23,287 - lava_callback - INFO - Starting async job submit for node 69afddd3ae557f2600f0f944 2026-03-12 09:01:23,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:23,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afddd3ae557f2600f0f944 HTTP/1.1" 200 None 2026-03-12 09:01:23,304 - lava_callback - DEBUG - Processing node 69afddd3ae557f2600f0f944: baseline-arm-clabbe 2026-03-12 09:01:23,304 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afddd3ae557f2600f0f944 2026-03-12 09:01:23,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:23,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afddd3ae557f2600f0f944 HTTP/1.1" 200 None 2026-03-12 09:01:23,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:23,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 09:01:23,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:01:23,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afddd3ae557f2600f0f944 HTTP/1.1" 200 None 2026-03-12 09:01:23,382 - lava_callback - ERROR - Error processing callback for node 69afddd3ae557f2600f0f944: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34168 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:03:54,095 - lava_callback - INFO - Received callback for node 69afde65ae557f2600f0f973 INFO: 172.20.0.1:34174 - "POST /node/69afde65ae557f2600f0f973 HTTP/1.0" 202 Accepted 2026-03-12 09:03:54,100 - lava_callback - INFO - Starting async job submit for node 69afde65ae557f2600f0f973 2026-03-12 09:03:54,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:03:54,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afde65ae557f2600f0f973 HTTP/1.1" 200 None 2026-03-12 09:03:54,116 - lava_callback - DEBUG - Processing node 69afde65ae557f2600f0f973: baseline-arm64-baylibre 2026-03-12 09:03:54,116 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afde65ae557f2600f0f973 2026-03-12 09:03:54,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:03:54,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afde65ae557f2600f0f973 HTTP/1.1" 200 None 2026-03-12 09:03:54,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:03:54,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 09:03:54,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:03:54,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afde65ae557f2600f0f973 HTTP/1.1" 200 None 2026-03-12 09:03:54,171 - lava_callback - ERROR - Error processing callback for node 69afde65ae557f2600f0f973: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56524 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:04:51,695 - lava_callback - INFO - Received callback for node 69b27d254bce73bcf0234261 INFO: 172.20.0.1:55020 - "POST /node/69b27d254bce73bcf0234261 HTTP/1.0" 202 Accepted 2026-03-12 09:04:51,741 - lava_callback - INFO - Starting async job submit for node 69b27d254bce73bcf0234261 2026-03-12 09:04:51,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:04:51,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d254bce73bcf0234261 HTTP/1.1" 200 None 2026-03-12 09:04:51,782 - lava_callback - DEBUG - Processing node 69b27d254bce73bcf0234261: baseline-arm-pengutronix 2026-03-12 09:04:51,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:04:52,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:04:52,047 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d254bce73bcf0234261/log.txt.gz 2026-03-12 09:04:52,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:04:52,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:04:52,286 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d254bce73bcf0234261/lava-logs.yaml 2026-03-12 09:04:52,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:04:52,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:04:52,430 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b27d254bce73bcf0234261/lava_callback.json.gz DEBUG: Setup failed for 69b27d254bce73bcf0234261. Transitting job node result: incomplete -> fail 2026-03-12 09:04:52,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:04:52,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d254bce73bcf0234261 HTTP/1.1" 200 None 2026-03-12 09:04:52,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:04:52,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 09:04:52,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:04:52,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d254bce73bcf0234261 HTTP/1.1" 200 None 2026-03-12 09:04:52,613 - lava_callback - INFO - Completed processing callback for node 69b27d254bce73bcf0234261 INFO: 172.20.0.1:55036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55702 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:05:12,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:05:12,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43100 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:09:54,446 - lava_callback - INFO - Received callback for node 69afdfceae557f2600f0fa0b INFO: 172.20.0.1:43108 - "POST /node/69afdfceae557f2600f0fa0b HTTP/1.0" 202 Accepted 2026-03-12 09:09:54,452 - lava_callback - INFO - Starting async job submit for node 69afdfceae557f2600f0fa0b 2026-03-12 09:09:54,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:09:54,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdfceae557f2600f0fa0b HTTP/1.1" 200 None 2026-03-12 09:09:54,468 - lava_callback - DEBUG - Processing node 69afdfceae557f2600f0fa0b: baseline-arm64-baylibre 2026-03-12 09:09:54,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afdfceae557f2600f0fa0b 2026-03-12 09:09:54,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:09:54,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afdfceae557f2600f0fa0b HTTP/1.1" 200 None 2026-03-12 09:09:54,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:09:54,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 09:09:54,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:09:54,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afdfceae557f2600f0fa0b HTTP/1.1" 200 None 2026-03-12 09:09:54,520 - lava_callback - ERROR - Error processing callback for node 69afdfceae557f2600f0fa0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58450 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:12:15,047 - lava_callback - INFO - Received callback for node 69afe063ae557f2600f0fa5a INFO: 172.20.0.1:44432 - "POST /node/69afe063ae557f2600f0fa5a HTTP/1.0" 202 Accepted 2026-03-12 09:12:15,052 - lava_callback - INFO - Starting async job submit for node 69afe063ae557f2600f0fa5a 2026-03-12 09:12:15,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:15,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe063ae557f2600f0fa5a HTTP/1.1" 200 None 2026-03-12 09:12:15,069 - lava_callback - DEBUG - Processing node 69afe063ae557f2600f0fa5a: baseline-arm-baylibre 2026-03-12 09:12:15,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe063ae557f2600f0fa5a 2026-03-12 09:12:15,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:15,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe063ae557f2600f0fa5a HTTP/1.1" 200 None 2026-03-12 09:12:15,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:15,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:12:15,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:15,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe063ae557f2600f0fa5a HTTP/1.1" 200 None 2026-03-12 09:12:15,153 - lava_callback - ERROR - Error processing callback for node 69afe063ae557f2600f0fa5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44442 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 09:12:15,737 - lava_callback - INFO - Received callback for node 69afe067ae557f2600f0fa5d INFO: 172.20.0.1:44444 - "POST /node/69afe067ae557f2600f0fa5d HTTP/1.0" 202 Accepted 2026-03-12 09:12:15,742 - lava_callback - INFO - Starting async job submit for node 69afe067ae557f2600f0fa5d 2026-03-12 09:12:15,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:15,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe067ae557f2600f0fa5d HTTP/1.1" 200 None 2026-03-12 09:12:15,758 - lava_callback - DEBUG - Processing node 69afe067ae557f2600f0fa5d: baseline-arm-baylibre 2026-03-12 09:12:15,758 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe067ae557f2600f0fa5d 2026-03-12 09:12:15,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:15,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe067ae557f2600f0fa5d HTTP/1.1" 200 None 2026-03-12 09:12:15,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:15,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:12:15,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:15,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe067ae557f2600f0fa5d HTTP/1.1" 200 None 2026-03-12 09:12:15,820 - lava_callback - ERROR - Error processing callback for node 69afe067ae557f2600f0fa5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:12:34,510 - lava_callback - INFO - Received callback for node 69afe06bae557f2600f0fa64 INFO: 172.20.0.1:53500 - "POST /node/69afe06bae557f2600f0fa64 HTTP/1.0" 202 Accepted 2026-03-12 09:12:34,519 - lava_callback - INFO - Starting async job submit for node 69afe06bae557f2600f0fa64 2026-03-12 09:12:34,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:34,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe06bae557f2600f0fa64 HTTP/1.1" 200 None 2026-03-12 09:12:34,547 - lava_callback - DEBUG - Processing node 69afe06bae557f2600f0fa64: baseline-arm-baylibre 2026-03-12 09:12:34,547 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe06bae557f2600f0fa64 2026-03-12 09:12:34,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:34,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe06bae557f2600f0fa64 HTTP/1.1" 200 None 2026-03-12 09:12:34,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:34,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:12:34,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:34,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe06bae557f2600f0fa64 HTTP/1.1" 200 None 2026-03-12 09:12:34,640 - lava_callback - ERROR - Error processing callback for node 69afe06bae557f2600f0fa64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:12:35,205 - lava_callback - INFO - Received callback for node 69afe06fae557f2600f0fa69 INFO: 172.20.0.1:53506 - "POST /node/69afe06fae557f2600f0fa69 HTTP/1.0" 202 Accepted 2026-03-12 09:12:35,211 - lava_callback - INFO - Starting async job submit for node 69afe06fae557f2600f0fa69 2026-03-12 09:12:35,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:35,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe06fae557f2600f0fa69 HTTP/1.1" 200 None 2026-03-12 09:12:35,228 - lava_callback - DEBUG - Processing node 69afe06fae557f2600f0fa69: baseline-arm-baylibre 2026-03-12 09:12:35,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe06fae557f2600f0fa69 2026-03-12 09:12:35,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:35,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe06fae557f2600f0fa69 HTTP/1.1" 200 None 2026-03-12 09:12:35,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:35,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:12:35,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:35,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe06fae557f2600f0fa69 HTTP/1.1" 200 None 2026-03-12 09:12:35,295 - lava_callback - ERROR - Error processing callback for node 69afe06fae557f2600f0fa69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:12:35,893 - lava_callback - INFO - Received callback for node 69afe078ae557f2600f0fa87 INFO: 172.20.0.1:53516 - "POST /node/69afe078ae557f2600f0fa87 HTTP/1.0" 202 Accepted 2026-03-12 09:12:35,898 - lava_callback - INFO - Starting async job submit for node 69afe078ae557f2600f0fa87 2026-03-12 09:12:35,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:35,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe078ae557f2600f0fa87 HTTP/1.1" 200 None 2026-03-12 09:12:35,917 - lava_callback - DEBUG - Processing node 69afe078ae557f2600f0fa87: baseline-arm-baylibre 2026-03-12 09:12:35,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe078ae557f2600f0fa87 2026-03-12 09:12:35,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:35,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe078ae557f2600f0fa87 HTTP/1.1" 200 None 2026-03-12 09:12:35,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:35,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:12:35,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:35,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe078ae557f2600f0fa87 HTTP/1.1" 200 None 2026-03-12 09:12:35,981 - lava_callback - ERROR - Error processing callback for node 69afe078ae557f2600f0fa87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53522 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:12:36,540 - lava_callback - INFO - Received callback for node 69afe07bae557f2600f0fa8a INFO: 172.20.0.1:53528 - "POST /node/69afe07bae557f2600f0fa8a HTTP/1.0" 202 Accepted 2026-03-12 09:12:36,545 - lava_callback - INFO - Starting async job submit for node 69afe07bae557f2600f0fa8a 2026-03-12 09:12:36,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:36,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe07bae557f2600f0fa8a HTTP/1.1" 200 None 2026-03-12 09:12:36,564 - lava_callback - DEBUG - Processing node 69afe07bae557f2600f0fa8a: baseline-arm-baylibre 2026-03-12 09:12:36,564 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe07bae557f2600f0fa8a 2026-03-12 09:12:36,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:36,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe07bae557f2600f0fa8a HTTP/1.1" 200 None 2026-03-12 09:12:36,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:36,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:12:36,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:12:36,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe07bae557f2600f0fa8a HTTP/1.1" 200 None 2026-03-12 09:12:36,627 - lava_callback - ERROR - Error processing callback for node 69afe07bae557f2600f0fa8a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46390 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:13:03,724 - lava_callback - INFO - Received callback for node 69afe095ae557f2600f0fab8 INFO: 172.20.0.1:40346 - "POST /node/69afe095ae557f2600f0fab8 HTTP/1.0" 202 Accepted 2026-03-12 09:13:03,733 - lava_callback - INFO - Starting async job submit for node 69afe095ae557f2600f0fab8 2026-03-12 09:13:03,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:03,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe095ae557f2600f0fab8 HTTP/1.1" 200 None 2026-03-12 09:13:03,764 - lava_callback - DEBUG - Processing node 69afe095ae557f2600f0fab8: baseline-arm-clabbe 2026-03-12 09:13:03,765 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe095ae557f2600f0fab8 2026-03-12 09:13:03,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:03,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe095ae557f2600f0fab8 HTTP/1.1" 200 None 2026-03-12 09:13:03,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:03,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:13:03,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:03,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe095ae557f2600f0fab8 HTTP/1.1" 200 None 2026-03-12 09:13:03,857 - lava_callback - ERROR - Error processing callback for node 69afe095ae557f2600f0fab8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37114 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:13:23,677 - lava_callback - INFO - Received callback for node 69afe0a2ae557f2600f0fac9 INFO: 172.20.0.1:41004 - "POST /node/69afe0a2ae557f2600f0fac9 HTTP/1.0" 202 Accepted 2026-03-12 09:13:23,686 - lava_callback - INFO - Starting async job submit for node 69afe0a2ae557f2600f0fac9 2026-03-12 09:13:23,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:23,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe0a2ae557f2600f0fac9 HTTP/1.1" 200 None 2026-03-12 09:13:23,742 - lava_callback - DEBUG - Processing node 69afe0a2ae557f2600f0fac9: baseline-arm-clabbe 2026-03-12 09:13:23,744 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe0a2ae557f2600f0fac9 2026-03-12 09:13:23,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:23,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe0a2ae557f2600f0fac9 HTTP/1.1" 200 None 2026-03-12 09:13:23,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:23,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:13:23,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:23,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe0a2ae557f2600f0fac9 HTTP/1.1" 200 None 2026-03-12 09:13:23,854 - lava_callback - ERROR - Error processing callback for node 69afe0a2ae557f2600f0fac9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:13:24,367 - lava_callback - INFO - Received callback for node 69afe0a4ae557f2600f0facc INFO: 172.20.0.1:41010 - "POST /node/69afe0a4ae557f2600f0facc HTTP/1.0" 202 Accepted 2026-03-12 09:13:24,372 - lava_callback - INFO - Starting async job submit for node 69afe0a4ae557f2600f0facc 2026-03-12 09:13:24,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:24,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe0a4ae557f2600f0facc HTTP/1.1" 200 None 2026-03-12 09:13:24,389 - lava_callback - DEBUG - Processing node 69afe0a4ae557f2600f0facc: baseline-arm-clabbe 2026-03-12 09:13:24,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe0a4ae557f2600f0facc 2026-03-12 09:13:24,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:24,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe0a4ae557f2600f0facc HTTP/1.1" 200 None 2026-03-12 09:13:24,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:24,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:13:24,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:24,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe0a4ae557f2600f0facc HTTP/1.1" 200 None 2026-03-12 09:13:24,459 - lava_callback - ERROR - Error processing callback for node 69afe0a4ae557f2600f0facc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:13:25,039 - lava_callback - INFO - Received callback for node 69afe0a7ae557f2600f0facf INFO: 172.20.0.1:41016 - "POST /node/69afe0a7ae557f2600f0facf HTTP/1.0" 202 Accepted 2026-03-12 09:13:25,044 - lava_callback - INFO - Starting async job submit for node 69afe0a7ae557f2600f0facf 2026-03-12 09:13:25,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:25,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe0a7ae557f2600f0facf HTTP/1.1" 200 None 2026-03-12 09:13:25,061 - lava_callback - DEBUG - Processing node 69afe0a7ae557f2600f0facf: baseline-arm-clabbe 2026-03-12 09:13:25,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe0a7ae557f2600f0facf 2026-03-12 09:13:25,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:25,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe0a7ae557f2600f0facf HTTP/1.1" 200 None 2026-03-12 09:13:25,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:25,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:13:25,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:25,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe0a7ae557f2600f0facf HTTP/1.1" 200 None 2026-03-12 09:13:25,131 - lava_callback - ERROR - Error processing callback for node 69afe0a7ae557f2600f0facf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:13:25,689 - lava_callback - INFO - Received callback for node 69afe0afae557f2600f0fae9 INFO: 172.20.0.1:41020 - "POST /node/69afe0afae557f2600f0fae9 HTTP/1.0" 202 Accepted 2026-03-12 09:13:25,698 - lava_callback - INFO - Starting async job submit for node 69afe0afae557f2600f0fae9 2026-03-12 09:13:25,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:25,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe0afae557f2600f0fae9 HTTP/1.1" 200 None 2026-03-12 09:13:25,721 - lava_callback - DEBUG - Processing node 69afe0afae557f2600f0fae9: baseline-arm-clabbe 2026-03-12 09:13:25,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe0afae557f2600f0fae9 2026-03-12 09:13:25,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:25,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe0afae557f2600f0fae9 HTTP/1.1" 200 None 2026-03-12 09:13:25,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:25,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 09:13:25,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:25,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe0afae557f2600f0fae9 HTTP/1.1" 200 None 2026-03-12 09:13:25,799 - lava_callback - ERROR - Error processing callback for node 69afe0afae557f2600f0fae9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:13:54,684 - lava_callback - INFO - Received callback for node 69afe0beae557f2600f0fb08 INFO: 172.20.0.1:46676 - "POST /node/69afe0beae557f2600f0fb08 HTTP/1.0" 202 Accepted 2026-03-12 09:13:54,693 - lava_callback - INFO - Starting async job submit for node 69afe0beae557f2600f0fb08 2026-03-12 09:13:54,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:54,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe0beae557f2600f0fb08 HTTP/1.1" 200 None 2026-03-12 09:13:54,724 - lava_callback - DEBUG - Processing node 69afe0beae557f2600f0fb08: baseline-arm64-baylibre 2026-03-12 09:13:54,725 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe0beae557f2600f0fb08 2026-03-12 09:13:54,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:54,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe0beae557f2600f0fb08 HTTP/1.1" 200 None 2026-03-12 09:13:54,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:54,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 09:13:54,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:13:54,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe0beae557f2600f0fb08 HTTP/1.1" 200 None 2026-03-12 09:13:54,842 - lava_callback - ERROR - Error processing callback for node 69afe0beae557f2600f0fb08: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40488 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:14:39,845 - lava_callback - INFO - Received callback for node 69b283ec4bce73bcf0234608 INFO: 172.20.0.1:36718 - "POST /node/69b283ec4bce73bcf0234608 HTTP/1.0" 202 Accepted 2026-03-12 09:14:39,851 - lava_callback - INFO - Starting async job submit for node 69b283ec4bce73bcf0234608 2026-03-12 09:14:39,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:14:39,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ec4bce73bcf0234608 HTTP/1.1" 200 None 2026-03-12 09:14:39,869 - lava_callback - DEBUG - Processing node 69b283ec4bce73bcf0234608: baseline-arm64 2026-03-12 09:14:39,874 - lava_callback - WARNING - Failed to upload log 2026-03-12 09:14:39,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:14:39,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:14:39,990 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ec4bce73bcf0234608/lava-logs.yaml 2026-03-12 09:14:39,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:14:40,106 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:14:40,107 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ec4bce73bcf0234608/lava_callback.json.gz 2026-03-12 09:14:40,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:14:40,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ec4bce73bcf0234608 HTTP/1.1" 200 None 2026-03-12 09:14:40,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:14:40,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:14:40,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:14:40,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283ec4bce73bcf0234608 HTTP/1.1" 200 None 2026-03-12 09:14:40,179 - lava_callback - INFO - Completed processing callback for node 69b283ec4bce73bcf0234608 2026-03-12 09:14:42,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:14:42,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40026 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:15:05,045 - lava_callback - INFO - Received callback for node 69b284074bce73bcf023466d INFO: 172.20.0.1:42768 - "POST /node/69b284074bce73bcf023466d HTTP/1.0" 202 Accepted 2026-03-12 09:15:05,053 - lava_callback - INFO - Starting async job submit for node 69b284074bce73bcf023466d 2026-03-12 09:15:05,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:05,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284074bce73bcf023466d HTTP/1.1" 200 None 2026-03-12 09:15:05,070 - lava_callback - DEBUG - Processing node 69b284074bce73bcf023466d: baseline-nfs-arm64 2026-03-12 09:15:05,072 - lava_callback - WARNING - Failed to upload log 2026-03-12 09:15:05,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:05,188 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:05,189 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284074bce73bcf023466d/lava-logs.yaml 2026-03-12 09:15:05,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:05,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:05,340 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284074bce73bcf023466d/lava_callback.json.gz 2026-03-12 09:15:05,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:05,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284074bce73bcf023466d HTTP/1.1" 200 None 2026-03-12 09:15:05,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:05,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:15:05,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:05,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284074bce73bcf023466d HTTP/1.1" 200 None 2026-03-12 09:15:05,417 - lava_callback - INFO - Completed processing callback for node 69b284074bce73bcf023466d INFO: 172.20.0.1:42780 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:15:12,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:12,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50534 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:15:27,863 - lava_callback - INFO - Received callback for node 69b283f84bce73bcf0234629 INFO: 172.20.0.1:50540 - "POST /node/69b283f84bce73bcf0234629 HTTP/1.0" 202 Accepted 2026-03-12 09:15:27,870 - lava_callback - INFO - Starting async job submit for node 69b283f84bce73bcf0234629 2026-03-12 09:15:27,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:27,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f84bce73bcf0234629 HTTP/1.1" 200 None 2026-03-12 09:15:27,904 - lava_callback - DEBUG - Processing node 69b283f84bce73bcf0234629: baseline-arm64 2026-03-12 09:15:28,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:28,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:28,193 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f84bce73bcf0234629/log.txt.gz 2026-03-12 09:15:28,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:28,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:28,640 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f84bce73bcf0234629/lava-logs.yaml 2026-03-12 09:15:28,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:28,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:28,861 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f84bce73bcf0234629/lava_callback.json.gz 2026-03-12 09:15:28,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:28,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f84bce73bcf0234629 HTTP/1.1" 200 None 2026-03-12 09:15:28,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:28,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:15:28,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:29,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283f84bce73bcf0234629 HTTP/1.1" 200 None 2026-03-12 09:15:29,011 - lava_callback - INFO - Completed processing callback for node 69b283f84bce73bcf0234629 INFO: 172.20.0.1:50668 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:15:41,856 - lava_callback - INFO - Received callback for node 69b283ed4bce73bcf023460e INFO: 172.20.0.1:45946 - "POST /node/69b283ed4bce73bcf023460e HTTP/1.0" 202 Accepted 2026-03-12 09:15:41,866 - lava_callback - INFO - Starting async job submit for node 69b283ed4bce73bcf023460e 2026-03-12 09:15:41,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:41,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ed4bce73bcf023460e HTTP/1.1" 200 None 2026-03-12 09:15:41,946 - lava_callback - DEBUG - Processing node 69b283ed4bce73bcf023460e: baseline-arm64 2026-03-12 09:15:42,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:42,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:42,505 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ed4bce73bcf023460e/log.txt.gz 2026-03-12 09:15:42,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:42,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:42,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-12 09:15:42,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:42,912 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ed4bce73bcf023460e/lava-logs.yaml 2026-03-12 09:15:42,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:43,077 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:43,079 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ed4bce73bcf023460e/lava_callback.json.gz 2026-03-12 09:15:43,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:43,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ed4bce73bcf023460e HTTP/1.1" 200 None 2026-03-12 09:15:43,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:43,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:15:43,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:43,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283ed4bce73bcf023460e HTTP/1.1" 200 None 2026-03-12 09:15:43,172 - lava_callback - INFO - Completed processing callback for node 69b283ed4bce73bcf023460e INFO: 172.20.0.1:45952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47532 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:15:54,511 - lava_callback - INFO - Received callback for node 69b27d474bce73bcf02342bd INFO: 172.20.0.1:47546 - "POST /node/69b27d474bce73bcf02342bd HTTP/1.0" 202 Accepted 2026-03-12 09:15:54,559 - lava_callback - INFO - Starting async job submit for node 69b27d474bce73bcf02342bd 2026-03-12 09:15:54,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:54,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d474bce73bcf02342bd HTTP/1.1" 200 None 2026-03-12 09:15:54,604 - lava_callback - DEBUG - Processing node 69b27d474bce73bcf02342bd: ltp-sched 2026-03-12 09:15:54,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:54,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:54,829 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69b27d474bce73bcf02342bd/log.txt.gz 2026-03-12 09:15:54,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:55,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:55,074 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69b27d474bce73bcf02342bd/lava-logs.yaml 2026-03-12 09:15:55,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:15:55,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:15:55,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69b27d474bce73bcf02342bd/lava_callback.json.gz 2026-03-12 09:15:55,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:55,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b27d474bce73bcf02342bd HTTP/1.1" 200 None 2026-03-12 09:15:55,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:55,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 09:15:55,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:15:55,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b27d474bce73bcf02342bd HTTP/1.1" 200 None 2026-03-12 09:15:55,474 - lava_callback - INFO - Completed processing callback for node 69b27d474bce73bcf02342bd INFO: 172.20.0.1:47550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43092 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:16:12,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:12,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-12 09:16:13,900 - lava_callback - INFO - Received callback for node 69b283ed4bce73bcf023460b INFO: 172.20.0.1:43382 - "POST /node/69b283ed4bce73bcf023460b HTTP/1.0" 202 Accepted 2026-03-12 09:16:13,906 - lava_callback - INFO - Starting async job submit for node 69b283ed4bce73bcf023460b 2026-03-12 09:16:13,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:13,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ed4bce73bcf023460b HTTP/1.1" 200 None 2026-03-12 09:16:13,946 - lava_callback - DEBUG - Processing node 69b283ed4bce73bcf023460b: baseline-arm64 2026-03-12 09:16:14,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:14,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:14,315 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ed4bce73bcf023460b/log.txt.gz 2026-03-12 09:16:14,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:14,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:14,669 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ed4bce73bcf023460b/lava-logs.yaml 2026-03-12 09:16:14,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:14,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:14,810 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ed4bce73bcf023460b/lava_callback.json.gz 2026-03-12 09:16:14,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:14,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ed4bce73bcf023460b HTTP/1.1" 200 None 2026-03-12 09:16:14,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:14,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:14,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:14,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283ed4bce73bcf023460b HTTP/1.1" 200 None 2026-03-12 09:16:14,897 - lava_callback - INFO - Completed processing callback for node 69b283ed4bce73bcf023460b INFO: 172.20.0.1:43394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43410 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:16:20,142 - lava_callback - INFO - Received callback for node 69b283f54bce73bcf0234623 INFO: 172.20.0.1:37276 - "POST /node/69b283f54bce73bcf0234623 HTTP/1.0" 202 Accepted 2026-03-12 09:16:20,152 - lava_callback - INFO - Starting async job submit for node 69b283f54bce73bcf0234623 2026-03-12 09:16:20,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:20,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f54bce73bcf0234623 HTTP/1.1" 200 None 2026-03-12 09:16:20,214 - lava_callback - DEBUG - Processing node 69b283f54bce73bcf0234623: baseline-arm64 2026-03-12 09:16:20,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:21,006 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:21,008 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f54bce73bcf0234623/log.txt.gz 2026-03-12 09:16:21,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:21,469 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:21,472 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f54bce73bcf0234623/lava-logs.yaml 2026-03-12 09:16:21,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:21,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:21,671 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f54bce73bcf0234623/lava_callback.json.gz 2026-03-12 09:16:21,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:21,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f54bce73bcf0234623 HTTP/1.1" 200 None 2026-03-12 09:16:21,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:21,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:21,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:21,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283f54bce73bcf0234623 HTTP/1.1" 200 None 2026-03-12 09:16:21,765 - lava_callback - INFO - Completed processing callback for node 69b283f54bce73bcf0234623 2026-03-12 09:16:23,264 - lava_callback - INFO - Received callback for node 69b283f44bce73bcf0234620 INFO: 172.20.0.1:37290 - "POST /node/69b283f44bce73bcf0234620 HTTP/1.0" 202 Accepted 2026-03-12 09:16:23,270 - lava_callback - INFO - Starting async job submit for node 69b283f44bce73bcf0234620 2026-03-12 09:16:23,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:23,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f44bce73bcf0234620 HTTP/1.1" 200 None 2026-03-12 09:16:23,308 - lava_callback - DEBUG - Processing node 69b283f44bce73bcf0234620: baseline-arm64 2026-03-12 09:16:23,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:23,895 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:23,897 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f44bce73bcf0234620/log.txt.gz 2026-03-12 09:16:24,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:24,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:24,430 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f44bce73bcf0234620/lava-logs.yaml 2026-03-12 09:16:24,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:24,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:24,607 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f44bce73bcf0234620/lava_callback.json.gz 2026-03-12 09:16:24,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:24,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f44bce73bcf0234620 HTTP/1.1" 200 None 2026-03-12 09:16:24,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:24,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:24,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:24,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283f44bce73bcf0234620 HTTP/1.1" 200 None 2026-03-12 09:16:24,701 - lava_callback - INFO - Completed processing callback for node 69b283f44bce73bcf0234620 INFO: 172.20.0.1:37298 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:16:28,442 - lava_callback - INFO - Received callback for node 69b2842f4bce73bcf02346e6 INFO: 172.20.0.1:37300 - "POST /node/69b2842f4bce73bcf02346e6 HTTP/1.0" 202 Accepted 2026-03-12 09:16:28,448 - lava_callback - INFO - Starting async job submit for node 69b2842f4bce73bcf02346e6 2026-03-12 09:16:28,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:28,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2842f4bce73bcf02346e6 HTTP/1.1" 200 None 2026-03-12 09:16:28,484 - lava_callback - DEBUG - Processing node 69b2842f4bce73bcf02346e6: baseline-arm64-pengutronix 2026-03-12 09:16:28,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:28,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:28,786 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b2842f4bce73bcf02346e6/log.txt.gz 2026-03-12 09:16:28,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:29,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:29,084 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b2842f4bce73bcf02346e6/lava-logs.yaml 2026-03-12 09:16:29,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:29,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:29,246 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b2842f4bce73bcf02346e6/lava_callback.json.gz 2026-03-12 09:16:29,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:29,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2842f4bce73bcf02346e6 HTTP/1.1" 200 None 2026-03-12 09:16:29,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:29,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:29,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:29,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2842f4bce73bcf02346e6 HTTP/1.1" 200 None 2026-03-12 09:16:29,333 - lava_callback - INFO - Completed processing callback for node 69b2842f4bce73bcf02346e6 2026-03-12 09:16:31,486 - lava_callback - INFO - Received callback for node 69b283f14bce73bcf0234617 INFO: 172.20.0.1:46278 - "POST /node/69b283f14bce73bcf0234617 HTTP/1.0" 202 Accepted 2026-03-12 09:16:31,492 - lava_callback - INFO - Starting async job submit for node 69b283f14bce73bcf0234617 2026-03-12 09:16:31,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:31,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f14bce73bcf0234617 HTTP/1.1" 200 None 2026-03-12 09:16:31,551 - lava_callback - DEBUG - Processing node 69b283f14bce73bcf0234617: baseline-arm64 2026-03-12 09:16:31,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:31,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:31,836 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f14bce73bcf0234617/log.txt.gz 2026-03-12 09:16:31,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:32,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:32,069 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f14bce73bcf0234617/lava-logs.yaml 2026-03-12 09:16:32,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:32,189 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:32,191 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f14bce73bcf0234617/lava_callback.json.gz 2026-03-12 09:16:32,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:32,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f14bce73bcf0234617 HTTP/1.1" 200 None 2026-03-12 09:16:32,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:32,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:32,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:32,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283f14bce73bcf0234617 HTTP/1.1" 200 None 2026-03-12 09:16:32,363 - lava_callback - INFO - Completed processing callback for node 69b283f14bce73bcf0234617 2026-03-12 09:16:36,120 - lava_callback - INFO - Received callback for node 69b284334bce73bcf02346f3 INFO: 172.20.0.1:46282 - "POST /node/69b284334bce73bcf02346f3 HTTP/1.0" 202 Accepted 2026-03-12 09:16:36,126 - lava_callback - INFO - Starting async job submit for node 69b284334bce73bcf02346f3 2026-03-12 09:16:36,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:36,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284334bce73bcf02346f3 HTTP/1.1" 200 None 2026-03-12 09:16:36,163 - lava_callback - DEBUG - Processing node 69b284334bce73bcf02346f3: baseline-arm64-foundriesio 2026-03-12 09:16:36,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:36,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:36,499 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b284334bce73bcf02346f3/log.txt.gz INFO: 172.20.0.1:46288 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:16:36,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:36,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:36,829 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b284334bce73bcf02346f3/lava-logs.yaml 2026-03-12 09:16:36,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:36,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:36,982 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b284334bce73bcf02346f3/lava_callback.json.gz 2026-03-12 09:16:37,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:37,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284334bce73bcf02346f3 HTTP/1.1" 200 None 2026-03-12 09:16:37,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:37,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:37,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:37,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284334bce73bcf02346f3 HTTP/1.1" 200 None 2026-03-12 09:16:37,071 - lava_callback - INFO - Completed processing callback for node 69b284334bce73bcf02346f3 2026-03-12 09:16:38,209 - lava_callback - INFO - Received callback for node 69b284304bce73bcf02346e9 INFO: 172.20.0.1:46290 - "POST /node/69b284304bce73bcf02346e9 HTTP/1.0" 202 Accepted 2026-03-12 09:16:38,215 - lava_callback - INFO - Starting async job submit for node 69b284304bce73bcf02346e9 2026-03-12 09:16:38,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:38,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284304bce73bcf02346e9 HTTP/1.1" 200 None 2026-03-12 09:16:38,250 - lava_callback - DEBUG - Processing node 69b284304bce73bcf02346e9: baseline-arm64-pengutronix 2026-03-12 09:16:38,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:38,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:38,575 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b284304bce73bcf02346e9/log.txt.gz 2026-03-12 09:16:38,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:38,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:38,889 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b284304bce73bcf02346e9/lava-logs.yaml 2026-03-12 09:16:38,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:39,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:39,058 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b284304bce73bcf02346e9/lava_callback.json.gz 2026-03-12 09:16:39,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:39,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284304bce73bcf02346e9 HTTP/1.1" 200 None 2026-03-12 09:16:39,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:39,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:39,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:39,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284304bce73bcf02346e9 HTTP/1.1" 200 None 2026-03-12 09:16:39,135 - lava_callback - INFO - Completed processing callback for node 69b284304bce73bcf02346e9 2026-03-12 09:16:39,144 - lava_callback - INFO - Received callback for node 69b283fd4bce73bcf023464f INFO: 172.20.0.1:46306 - "POST /node/69b283fd4bce73bcf023464f HTTP/1.0" 202 Accepted 2026-03-12 09:16:39,181 - lava_callback - INFO - Starting async job submit for node 69b283fd4bce73bcf023464f 2026-03-12 09:16:39,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:39,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283fd4bce73bcf023464f HTTP/1.1" 200 None 2026-03-12 09:16:39,243 - lava_callback - DEBUG - Processing node 69b283fd4bce73bcf023464f: baseline-arm64 2026-03-12 09:16:39,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:40,589 - lava_callback - INFO - Received callback for node 69b283ef4bce73bcf0234614 INFO: 172.20.0.1:47176 - "POST /node/69b283ef4bce73bcf0234614 HTTP/1.0" 202 Accepted 2026-03-12 09:16:40,612 - lava_callback - INFO - Starting async job submit for node 69b283ef4bce73bcf0234614 2026-03-12 09:16:40,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:40,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ef4bce73bcf0234614 HTTP/1.1" 200 None 2026-03-12 09:16:40,656 - lava_callback - DEBUG - Processing node 69b283ef4bce73bcf0234614: baseline-arm64 2026-03-12 09:16:40,801 - lava_callback - WARNING - Failed to upload log 2026-03-12 09:16:40,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:40,954 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:40,956 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fd4bce73bcf023464f/log.txt.gz 2026-03-12 09:16:41,044 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:41,078 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ef4bce73bcf0234614/lava-logs.yaml 2026-03-12 09:16:41,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:41,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:41,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:41,279 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ef4bce73bcf0234614/lava_callback.json.gz 2026-03-12 09:16:41,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:41,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ef4bce73bcf0234614 HTTP/1.1" 200 None 2026-03-12 09:16:41,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:41,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:41,344 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fd4bce73bcf023464f/lava-logs.yaml 2026-03-12 09:16:41,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:41,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:41,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:41,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283ef4bce73bcf0234614 HTTP/1.1" 200 None 2026-03-12 09:16:41,474 - lava_callback - INFO - Completed processing callback for node 69b283ef4bce73bcf0234614 2026-03-12 09:16:41,564 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:41,566 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fd4bce73bcf023464f/lava_callback.json.gz 2026-03-12 09:16:41,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:41,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283fd4bce73bcf023464f HTTP/1.1" 200 None 2026-03-12 09:16:41,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:41,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:41,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:41,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283fd4bce73bcf023464f HTTP/1.1" 200 None 2026-03-12 09:16:41,683 - lava_callback - INFO - Completed processing callback for node 69b283fd4bce73bcf023464f 2026-03-12 09:16:41,836 - lava_callback - INFO - Received callback for node 69b283f94bce73bcf023462c INFO: 172.20.0.1:47186 - "POST /node/69b283f94bce73bcf023462c HTTP/1.0" 202 Accepted 2026-03-12 09:16:41,846 - lava_callback - INFO - Starting async job submit for node 69b283f94bce73bcf023462c 2026-03-12 09:16:41,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:41,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f94bce73bcf023462c HTTP/1.1" 200 None 2026-03-12 09:16:41,900 - lava_callback - DEBUG - Processing node 69b283f94bce73bcf023462c: baseline-arm64 2026-03-12 09:16:42,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:42,283 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:42,285 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f94bce73bcf023462c/log.txt.gz 2026-03-12 09:16:42,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:42,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:42,597 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f94bce73bcf023462c/lava-logs.yaml 2026-03-12 09:16:42,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:42,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:42,755 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f94bce73bcf023462c/lava_callback.json.gz 2026-03-12 09:16:42,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:42,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f94bce73bcf023462c HTTP/1.1" 200 None 2026-03-12 09:16:42,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:42,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:42,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:42,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283f94bce73bcf023462c HTTP/1.1" 200 None 2026-03-12 09:16:42,838 - lava_callback - INFO - Completed processing callback for node 69b283f94bce73bcf023462c 2026-03-12 09:16:42,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:42,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-12 09:16:43,281 - lava_callback - INFO - Received callback for node 69b283f34bce73bcf023461d INFO: 172.20.0.1:47196 - "POST /node/69b283f34bce73bcf023461d HTTP/1.0" 202 Accepted 2026-03-12 09:16:43,288 - lava_callback - INFO - Starting async job submit for node 69b283f34bce73bcf023461d 2026-03-12 09:16:43,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:43,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f34bce73bcf023461d HTTP/1.1" 200 None 2026-03-12 09:16:43,328 - lava_callback - DEBUG - Processing node 69b283f34bce73bcf023461d: baseline-arm64 2026-03-12 09:16:43,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:43,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:43,792 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f34bce73bcf023461d/log.txt.gz 2026-03-12 09:16:43,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:44,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:44,128 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f34bce73bcf023461d/lava-logs.yaml 2026-03-12 09:16:44,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:44,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:44,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f34bce73bcf023461d/lava_callback.json.gz 2026-03-12 09:16:44,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:44,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f34bce73bcf023461d HTTP/1.1" 200 None 2026-03-12 09:16:44,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:44,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:44,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:44,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283f34bce73bcf023461d HTTP/1.1" 200 None 2026-03-12 09:16:44,346 - lava_callback - INFO - Completed processing callback for node 69b283f34bce73bcf023461d INFO: 172.20.0.1:47212 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:16:46,911 - lava_callback - INFO - Received callback for node 69b283fb4bce73bcf0234632 INFO: 172.20.0.1:47216 - "POST /node/69b283fb4bce73bcf0234632 HTTP/1.0" 202 Accepted 2026-03-12 09:16:46,917 - lava_callback - INFO - Starting async job submit for node 69b283fb4bce73bcf0234632 2026-03-12 09:16:46,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:46,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283fb4bce73bcf0234632 HTTP/1.1" 200 None 2026-03-12 09:16:46,954 - lava_callback - DEBUG - Processing node 69b283fb4bce73bcf0234632: baseline-arm64 2026-03-12 09:16:47,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:47,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:47,376 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fb4bce73bcf0234632/log.txt.gz 2026-03-12 09:16:47,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:47,738 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:47,740 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fb4bce73bcf0234632/lava-logs.yaml 2026-03-12 09:16:47,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:47,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:47,887 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fb4bce73bcf0234632/lava_callback.json.gz 2026-03-12 09:16:47,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:47,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283fb4bce73bcf0234632 HTTP/1.1" 200 None 2026-03-12 09:16:47,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:47,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:47,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:47,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283fb4bce73bcf0234632 HTTP/1.1" 200 None 2026-03-12 09:16:47,977 - lava_callback - INFO - Completed processing callback for node 69b283fb4bce73bcf0234632 2026-03-12 09:16:48,275 - lava_callback - INFO - Received callback for node 69b284194bce73bcf0234697 INFO: 172.20.0.1:47220 - "POST /node/69b284194bce73bcf0234697 HTTP/1.0" 202 Accepted 2026-03-12 09:16:48,285 - lava_callback - INFO - Starting async job submit for node 69b284194bce73bcf0234697 2026-03-12 09:16:48,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:48,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284194bce73bcf0234697 HTTP/1.1" 200 None 2026-03-12 09:16:48,335 - lava_callback - DEBUG - Processing node 69b284194bce73bcf0234697: baseline-arm64-broonie 2026-03-12 09:16:48,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:48,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:48,685 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284194bce73bcf0234697/log.txt.gz 2026-03-12 09:16:48,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:49,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:49,018 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284194bce73bcf0234697/lava-logs.yaml 2026-03-12 09:16:49,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:49,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:49,161 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284194bce73bcf0234697/lava_callback.json.gz 2026-03-12 09:16:49,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:49,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284194bce73bcf0234697 HTTP/1.1" 200 None 2026-03-12 09:16:49,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:49,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:49,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:49,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284194bce73bcf0234697 HTTP/1.1" 200 None 2026-03-12 09:16:49,243 - lava_callback - INFO - Completed processing callback for node 69b284194bce73bcf0234697 2026-03-12 09:16:49,738 - lava_callback - INFO - Received callback for node 69b284324bce73bcf02346f0 INFO: 172.20.0.1:41480 - "POST /node/69b284324bce73bcf02346f0 HTTP/1.0" 202 Accepted 2026-03-12 09:16:49,745 - lava_callback - INFO - Starting async job submit for node 69b284324bce73bcf02346f0 2026-03-12 09:16:49,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:49,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284324bce73bcf02346f0 HTTP/1.1" 200 None 2026-03-12 09:16:49,782 - lava_callback - DEBUG - Processing node 69b284324bce73bcf02346f0: baseline-arm64-pengutronix 2026-03-12 09:16:50,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:50,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:50,145 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b284324bce73bcf02346f0/log.txt.gz 2026-03-12 09:16:50,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:50,346 - lava_callback - INFO - Received callback for node 69b284214bce73bcf02346c5 INFO: 172.20.0.1:41494 - "POST /node/69b284214bce73bcf02346c5 HTTP/1.0" 202 Accepted 2026-03-12 09:16:50,353 - lava_callback - INFO - Starting async job submit for node 69b284214bce73bcf02346c5 2026-03-12 09:16:50,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:50,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284214bce73bcf02346c5 HTTP/1.1" 200 None 2026-03-12 09:16:50,401 - lava_callback - DEBUG - Processing node 69b284214bce73bcf02346c5: baseline-arm64-broonie 2026-03-12 09:16:50,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:50,511 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b284324bce73bcf02346f0/lava-logs.yaml 2026-03-12 09:16:50,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:50,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:50,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:50,790 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284214bce73bcf02346c5/log.txt.gz 2026-03-12 09:16:50,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:50,853 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69b284324bce73bcf02346f0/lava_callback.json.gz 2026-03-12 09:16:50,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:50,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284324bce73bcf02346f0 HTTP/1.1" 200 None 2026-03-12 09:16:50,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:50,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:51,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:51,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:51,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284324bce73bcf02346f0 HTTP/1.1" 200 None 2026-03-12 09:16:51,045 - lava_callback - INFO - Completed processing callback for node 69b284324bce73bcf02346f0 2026-03-12 09:16:51,225 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:51,227 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284214bce73bcf02346c5/lava-logs.yaml 2026-03-12 09:16:51,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:51,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:51,394 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284214bce73bcf02346c5/lava_callback.json.gz 2026-03-12 09:16:51,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:51,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284214bce73bcf02346c5 HTTP/1.1" 200 None 2026-03-12 09:16:51,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:51,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:51,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:51,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284214bce73bcf02346c5 HTTP/1.1" 200 None 2026-03-12 09:16:51,494 - lava_callback - INFO - Completed processing callback for node 69b284214bce73bcf02346c5 INFO: 172.20.0.1:41496 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:16:54,279 - lava_callback - INFO - Received callback for node 69b284064bce73bcf023466a INFO: 172.20.0.1:41502 - "POST /node/69b284064bce73bcf023466a HTTP/1.0" 202 Accepted 2026-03-12 09:16:54,286 - lava_callback - INFO - Starting async job submit for node 69b284064bce73bcf023466a 2026-03-12 09:16:54,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:54,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284064bce73bcf023466a HTTP/1.1" 200 None 2026-03-12 09:16:54,324 - lava_callback - DEBUG - Processing node 69b284064bce73bcf023466a: baseline-nfs-arm64 2026-03-12 09:16:54,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:54,926 - lava_callback - INFO - Received callback for node 69b283ee4bce73bcf0234611 INFO: 172.20.0.1:41506 - "POST /node/69b283ee4bce73bcf0234611 HTTP/1.0" 202 Accepted 2026-03-12 09:16:54,934 - lava_callback - INFO - Starting async job submit for node 69b283ee4bce73bcf0234611 2026-03-12 09:16:54,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:54,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:54,965 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284064bce73bcf023466a/log.txt.gz 2026-03-12 09:16:55,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ee4bce73bcf0234611 HTTP/1.1" 200 None 2026-03-12 09:16:55,019 - lava_callback - DEBUG - Processing node 69b283ee4bce73bcf0234611: baseline-arm64 2026-03-12 09:16:55,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:55,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:55,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:55,689 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ee4bce73bcf0234611/log.txt.gz 2026-03-12 09:16:55,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:55,810 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284064bce73bcf023466a/lava-logs.yaml 2026-03-12 09:16:55,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:55,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:56,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:56,023 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ee4bce73bcf0234611/lava-logs.yaml 2026-03-12 09:16:56,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:16:56,123 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:56,125 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284064bce73bcf023466a/lava_callback.json.gz 2026-03-12 09:16:56,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:56,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:16:56,163 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ee4bce73bcf0234611/lava_callback.json.gz 2026-03-12 09:16:56,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284064bce73bcf023466a HTTP/1.1" 200 None 2026-03-12 09:16:56,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:56,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:56,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:56,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ee4bce73bcf0234611 HTTP/1.1" 200 None 2026-03-12 09:16:56,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:56,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:56,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284064bce73bcf023466a HTTP/1.1" 200 None 2026-03-12 09:16:56,238 - lava_callback - INFO - Completed processing callback for node 69b284064bce73bcf023466a 2026-03-12 09:16:56,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:16:56,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:16:56,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283ee4bce73bcf0234611 HTTP/1.1" 200 None 2026-03-12 09:16:56,330 - lava_callback - INFO - Completed processing callback for node 69b283ee4bce73bcf0234611 INFO: 172.20.0.1:41520 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:17:01,735 - lava_callback - INFO - Received callback for node 69b2840c4bce73bcf0234679 INFO: 172.20.0.1:38774 - "POST /node/69b2840c4bce73bcf0234679 HTTP/1.0" 202 Accepted 2026-03-12 09:17:01,742 - lava_callback - INFO - Starting async job submit for node 69b2840c4bce73bcf0234679 2026-03-12 09:17:01,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:01,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2840c4bce73bcf0234679 HTTP/1.1" 200 None 2026-03-12 09:17:01,780 - lava_callback - DEBUG - Processing node 69b2840c4bce73bcf0234679: baseline-nfs-arm64 2026-03-12 09:17:02,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:02,230 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:02,232 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840c4bce73bcf0234679/log.txt.gz 2026-03-12 09:17:02,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:02,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:02,626 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840c4bce73bcf0234679/lava-logs.yaml 2026-03-12 09:17:02,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:02,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:02,813 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840c4bce73bcf0234679/lava_callback.json.gz 2026-03-12 09:17:02,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:02,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2840c4bce73bcf0234679 HTTP/1.1" 200 None 2026-03-12 09:17:02,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:02,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:02,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:02,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2840c4bce73bcf0234679 HTTP/1.1" 200 None 2026-03-12 09:17:02,926 - lava_callback - INFO - Completed processing callback for node 69b2840c4bce73bcf0234679 2026-03-12 09:17:03,251 - lava_callback - INFO - Received callback for node 69b284054bce73bcf0234667 INFO: 172.20.0.1:38782 - "POST /node/69b284054bce73bcf0234667 HTTP/1.0" 202 Accepted 2026-03-12 09:17:03,259 - lava_callback - INFO - Starting async job submit for node 69b284054bce73bcf0234667 2026-03-12 09:17:03,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:03,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284054bce73bcf0234667 HTTP/1.1" 200 None 2026-03-12 09:17:03,301 - lava_callback - DEBUG - Processing node 69b284054bce73bcf0234667: baseline-nfs-arm64 2026-03-12 09:17:03,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:03,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:03,900 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284054bce73bcf0234667/log.txt.gz 2026-03-12 09:17:04,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:04,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:04,363 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284054bce73bcf0234667/lava-logs.yaml 2026-03-12 09:17:04,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:04,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:04,534 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284054bce73bcf0234667/lava_callback.json.gz 2026-03-12 09:17:04,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:04,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284054bce73bcf0234667 HTTP/1.1" 200 None 2026-03-12 09:17:04,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:04,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:04,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:04,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284054bce73bcf0234667 HTTP/1.1" 200 None 2026-03-12 09:17:04,625 - lava_callback - INFO - Completed processing callback for node 69b284054bce73bcf0234667 INFO: 172.20.0.1:38792 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:17:07,510 - lava_callback - INFO - Received callback for node 69b284094bce73bcf0234670 INFO: 172.20.0.1:38794 - "POST /node/69b284094bce73bcf0234670 HTTP/1.0" 202 Accepted 2026-03-12 09:17:07,517 - lava_callback - INFO - Starting async job submit for node 69b284094bce73bcf0234670 2026-03-12 09:17:07,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:07,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284094bce73bcf0234670 HTTP/1.1" 200 None 2026-03-12 09:17:07,558 - lava_callback - DEBUG - Processing node 69b284094bce73bcf0234670: baseline-nfs-arm64 2026-03-12 09:17:07,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:07,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:07,987 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284094bce73bcf0234670/log.txt.gz 2026-03-12 09:17:08,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:08,382 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:08,384 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284094bce73bcf0234670/lava-logs.yaml 2026-03-12 09:17:08,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:08,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:08,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284094bce73bcf0234670/lava_callback.json.gz 2026-03-12 09:17:08,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:08,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284094bce73bcf0234670 HTTP/1.1" 200 None 2026-03-12 09:17:08,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:08,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:08,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:08,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284094bce73bcf0234670 HTTP/1.1" 200 None 2026-03-12 09:17:08,709 - lava_callback - INFO - Completed processing callback for node 69b284094bce73bcf0234670 2026-03-12 09:17:11,415 - lava_callback - INFO - Received callback for node 69b2840e4bce73bcf023467c INFO: 172.20.0.1:46126 - "POST /node/69b2840e4bce73bcf023467c HTTP/1.0" 202 Accepted 2026-03-12 09:17:11,421 - lava_callback - INFO - Starting async job submit for node 69b2840e4bce73bcf023467c 2026-03-12 09:17:11,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:11,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2840e4bce73bcf023467c HTTP/1.1" 200 None 2026-03-12 09:17:11,477 - lava_callback - DEBUG - Processing node 69b2840e4bce73bcf023467c: baseline-nfs-arm64 2026-03-12 09:17:11,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:12,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:12,011 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840e4bce73bcf023467c/log.txt.gz 2026-03-12 09:17:12,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:12,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:12,458 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840e4bce73bcf023467c/lava-logs.yaml 2026-03-12 09:17:12,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:12,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:12,680 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840e4bce73bcf023467c/lava_callback.json.gz 2026-03-12 09:17:12,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:12,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2840e4bce73bcf023467c HTTP/1.1" 200 None 2026-03-12 09:17:12,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:12,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:12,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:12,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2840e4bce73bcf023467c HTTP/1.1" 200 None 2026-03-12 09:17:12,775 - lava_callback - INFO - Completed processing callback for node 69b2840e4bce73bcf023467c 2026-03-12 09:17:12,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:12,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-12 09:17:13,967 - lava_callback - INFO - Received callback for node 69b2841f4bce73bcf02346be INFO: 172.20.0.1:46140 - "POST /node/69b2841f4bce73bcf02346be HTTP/1.0" 202 Accepted 2026-03-12 09:17:13,973 - lava_callback - INFO - Starting async job submit for node 69b2841f4bce73bcf02346be 2026-03-12 09:17:13,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:14,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2841f4bce73bcf02346be HTTP/1.1" 200 None 2026-03-12 09:17:14,011 - lava_callback - DEBUG - Processing node 69b2841f4bce73bcf02346be: baseline-arm64-broonie 2026-03-12 09:17:14,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:14,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:14,407 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841f4bce73bcf02346be/log.txt.gz 2026-03-12 09:17:14,463 - lava_callback - INFO - Received callback for node 69b2840b4bce73bcf0234676 INFO: 172.20.0.1:46152 - "POST /node/69b2840b4bce73bcf0234676 HTTP/1.0" 202 Accepted 2026-03-12 09:17:14,487 - lava_callback - INFO - Starting async job submit for node 69b2840b4bce73bcf0234676 2026-03-12 09:17:14,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:14,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:14,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2840b4bce73bcf0234676 HTTP/1.1" 200 None 2026-03-12 09:17:14,609 - lava_callback - DEBUG - Processing node 69b2840b4bce73bcf0234676: baseline-nfs-arm64 2026-03-12 09:17:14,683 - lava_callback - INFO - Received callback for node 69b283fe4bce73bcf0234652 2026-03-12 09:17:14,705 - lava_callback - INFO - Starting async job submit for node 69b283fe4bce73bcf0234652 INFO: 172.20.0.1:46154 - "POST /node/69b283fe4bce73bcf0234652 HTTP/1.0" 202 Accepted 2026-03-12 09:17:14,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:14,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:14,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283fe4bce73bcf0234652 HTTP/1.1" 200 None 2026-03-12 09:17:14,853 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841f4bce73bcf02346be/lava-logs.yaml 2026-03-12 09:17:14,859 - lava_callback - DEBUG - Processing node 69b283fe4bce73bcf0234652: baseline-arm64 INFO: 172.20.0.1:46164 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 09:17:15,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:15,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:15,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:15,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:15,444 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fe4bce73bcf0234652/log.txt.gz 2026-03-12 09:17:15,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:15,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:15,533 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841f4bce73bcf02346be/lava_callback.json.gz 2026-03-12 09:17:15,533 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840b4bce73bcf0234676/log.txt.gz 2026-03-12 09:17:15,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:15,672 - lava_callback - INFO - Received callback for node 69b284254bce73bcf02346cb 2026-03-12 09:17:15,760 - lava_callback - INFO - Starting async job submit for node 69b284254bce73bcf02346cb INFO: 172.20.0.1:46166 - "POST /node/69b284254bce73bcf02346cb HTTP/1.0" 202 Accepted 2026-03-12 09:17:15,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:15,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:15,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2841f4bce73bcf02346be HTTP/1.1" 200 None 2026-03-12 09:17:15,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:15,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284254bce73bcf02346cb HTTP/1.1" 200 None 2026-03-12 09:17:15,922 - lava_callback - DEBUG - Processing node 69b284254bce73bcf02346cb: baseline-arm64-broonie 2026-03-12 09:17:15,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:16,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:16,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:16,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:16,148 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fe4bce73bcf0234652/lava-logs.yaml 2026-03-12 09:17:16,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2841f4bce73bcf02346be HTTP/1.1" 200 None 2026-03-12 09:17:16,237 - lava_callback - INFO - Completed processing callback for node 69b2841f4bce73bcf02346be 2026-03-12 09:17:16,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:16,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:16,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:16,299 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840b4bce73bcf0234676/lava-logs.yaml 2026-03-12 09:17:16,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:16,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:16,497 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fe4bce73bcf0234652/lava_callback.json.gz 2026-03-12 09:17:16,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:16,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:46176 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:17:16,532 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284254bce73bcf02346cb/log.txt.gz 2026-03-12 09:17:16,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:16,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283fe4bce73bcf0234652 HTTP/1.1" 200 None 2026-03-12 09:17:16,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:16,614 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840b4bce73bcf0234676/lava_callback.json.gz 2026-03-12 09:17:16,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:16,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:16,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:16,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:16,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2840b4bce73bcf0234676 HTTP/1.1" 200 None 2026-03-12 09:17:16,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:16,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283fe4bce73bcf0234652 HTTP/1.1" 200 None 2026-03-12 09:17:16,789 - lava_callback - INFO - Completed processing callback for node 69b283fe4bce73bcf0234652 2026-03-12 09:17:16,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:16,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:16,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2840b4bce73bcf0234676 HTTP/1.1" 200 None 2026-03-12 09:17:16,897 - lava_callback - INFO - Completed processing callback for node 69b2840b4bce73bcf0234676 2026-03-12 09:17:17,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:17,073 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284254bce73bcf02346cb/lava-logs.yaml 2026-03-12 09:17:17,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:17,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:17,259 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284254bce73bcf02346cb/lava_callback.json.gz 2026-03-12 09:17:17,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:17,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284254bce73bcf02346cb HTTP/1.1" 200 None 2026-03-12 09:17:17,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:17,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:17,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:17,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284254bce73bcf02346cb HTTP/1.1" 200 None 2026-03-12 09:17:17,340 - lava_callback - INFO - Completed processing callback for node 69b284254bce73bcf02346cb 2026-03-12 09:17:19,091 - lava_callback - INFO - Received callback for node 69b284034bce73bcf0234661 INFO: 172.20.0.1:46178 - "POST /node/69b284034bce73bcf0234661 HTTP/1.0" 202 Accepted 2026-03-12 09:17:19,097 - lava_callback - INFO - Starting async job submit for node 69b284034bce73bcf0234661 2026-03-12 09:17:19,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:19,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284034bce73bcf0234661 HTTP/1.1" 200 None 2026-03-12 09:17:19,141 - lava_callback - DEBUG - Processing node 69b284034bce73bcf0234661: baseline-nfs-arm64 2026-03-12 09:17:19,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:19,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:19,700 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284034bce73bcf0234661/log.txt.gz 2026-03-12 09:17:19,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:20,135 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:20,137 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284034bce73bcf0234661/lava-logs.yaml 2026-03-12 09:17:20,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:20,279 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:20,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284034bce73bcf0234661/lava_callback.json.gz 2026-03-12 09:17:20,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:20,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284034bce73bcf0234661 HTTP/1.1" 200 None 2026-03-12 09:17:20,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:20,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:20,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:20,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284034bce73bcf0234661 HTTP/1.1" 200 None 2026-03-12 09:17:20,361 - lava_callback - INFO - Completed processing callback for node 69b284034bce73bcf0234661 INFO: 172.20.0.1:34182 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:17:28,179 - lava_callback - INFO - Received callback for node 69b284264bce73bcf02346ce INFO: 172.20.0.1:34186 - "POST /node/69b284264bce73bcf02346ce HTTP/1.0" 202 Accepted 2026-03-12 09:17:28,185 - lava_callback - INFO - Starting async job submit for node 69b284264bce73bcf02346ce 2026-03-12 09:17:28,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:28,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284264bce73bcf02346ce HTTP/1.1" 200 None 2026-03-12 09:17:28,223 - lava_callback - DEBUG - Processing node 69b284264bce73bcf02346ce: baseline-arm64-broonie 2026-03-12 09:17:28,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:28,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:28,637 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284264bce73bcf02346ce/log.txt.gz 2026-03-12 09:17:28,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:28,963 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:28,966 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284264bce73bcf02346ce/lava-logs.yaml 2026-03-12 09:17:28,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:29,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:29,112 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284264bce73bcf02346ce/lava_callback.json.gz 2026-03-12 09:17:29,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:29,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284264bce73bcf02346ce HTTP/1.1" 200 None 2026-03-12 09:17:29,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:29,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:29,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:29,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284264bce73bcf02346ce HTTP/1.1" 200 None 2026-03-12 09:17:29,223 - lava_callback - INFO - Completed processing callback for node 69b284264bce73bcf02346ce 2026-03-12 09:17:34,003 - lava_callback - INFO - Received callback for node 69b284234bce73bcf02346c8 INFO: 172.20.0.1:43050 - "POST /node/69b284234bce73bcf02346c8 HTTP/1.0" 202 Accepted 2026-03-12 09:17:34,009 - lava_callback - INFO - Starting async job submit for node 69b284234bce73bcf02346c8 2026-03-12 09:17:34,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:34,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284234bce73bcf02346c8 HTTP/1.1" 200 None 2026-03-12 09:17:34,046 - lava_callback - DEBUG - Processing node 69b284234bce73bcf02346c8: baseline-arm64-broonie 2026-03-12 09:17:34,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:34,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:34,459 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284234bce73bcf02346c8/log.txt.gz 2026-03-12 09:17:34,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:34,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:34,814 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284234bce73bcf02346c8/lava-logs.yaml 2026-03-12 09:17:34,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:34,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:34,952 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284234bce73bcf02346c8/lava_callback.json.gz 2026-03-12 09:17:34,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:34,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284234bce73bcf02346c8 HTTP/1.1" 200 None 2026-03-12 09:17:34,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:35,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:35,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:35,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284234bce73bcf02346c8 HTTP/1.1" 200 None 2026-03-12 09:17:35,035 - lava_callback - INFO - Completed processing callback for node 69b284234bce73bcf02346c8 INFO: 172.20.0.1:43064 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:17:36,698 - lava_callback - INFO - Received callback for node 69b284004bce73bcf023465a INFO: 172.20.0.1:43066 - "POST /node/69b284004bce73bcf023465a HTTP/1.0" 202 Accepted 2026-03-12 09:17:36,705 - lava_callback - INFO - Starting async job submit for node 69b284004bce73bcf023465a 2026-03-12 09:17:36,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:36,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284004bce73bcf023465a HTTP/1.1" 200 None 2026-03-12 09:17:36,742 - lava_callback - DEBUG - Processing node 69b284004bce73bcf023465a: baseline-nfs-arm64 2026-03-12 09:17:37,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:37,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:37,159 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284004bce73bcf023465a/log.txt.gz 2026-03-12 09:17:37,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:37,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:37,546 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284004bce73bcf023465a/lava-logs.yaml 2026-03-12 09:17:37,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:37,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:37,690 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284004bce73bcf023465a/lava_callback.json.gz 2026-03-12 09:17:37,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:37,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284004bce73bcf023465a HTTP/1.1" 200 None 2026-03-12 09:17:37,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:37,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:37,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:37,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284004bce73bcf023465a HTTP/1.1" 200 None 2026-03-12 09:17:37,800 - lava_callback - INFO - Completed processing callback for node 69b284004bce73bcf023465a 2026-03-12 09:17:42,825 - lava_callback - INFO - Received callback for node 69b283fa4bce73bcf023462f INFO: 172.20.0.1:34496 - "POST /node/69b283fa4bce73bcf023462f HTTP/1.0" 202 Accepted 2026-03-12 09:17:42,831 - lava_callback - INFO - Starting async job submit for node 69b283fa4bce73bcf023462f 2026-03-12 09:17:42,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:42,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283fa4bce73bcf023462f HTTP/1.1" 200 None 2026-03-12 09:17:42,866 - lava_callback - DEBUG - Processing node 69b283fa4bce73bcf023462f: baseline-arm64 2026-03-12 09:17:42,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:43,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-12 09:17:43,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:43,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:43,729 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fa4bce73bcf023462f/log.txt.gz 2026-03-12 09:17:44,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:44,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:44,340 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fa4bce73bcf023462f/lava-logs.yaml 2026-03-12 09:17:44,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:44,509 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:44,511 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283fa4bce73bcf023462f/lava_callback.json.gz 2026-03-12 09:17:44,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:44,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283fa4bce73bcf023462f HTTP/1.1" 200 None 2026-03-12 09:17:44,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:44,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:44,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:44,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283fa4bce73bcf023462f HTTP/1.1" 200 None 2026-03-12 09:17:44,593 - lava_callback - INFO - Completed processing callback for node 69b283fa4bce73bcf023462f 2026-03-12 09:17:44,800 - lava_callback - INFO - Received callback for node 69b283f74bce73bcf0234626 INFO: 172.20.0.1:34510 - "POST /node/69b283f74bce73bcf0234626 HTTP/1.0" 202 Accepted 2026-03-12 09:17:44,807 - lava_callback - INFO - Starting async job submit for node 69b283f74bce73bcf0234626 2026-03-12 09:17:44,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:44,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f74bce73bcf0234626 HTTP/1.1" 200 None 2026-03-12 09:17:44,854 - lava_callback - DEBUG - Processing node 69b283f74bce73bcf0234626: baseline-arm64 2026-03-12 09:17:45,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:45,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:45,506 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f74bce73bcf0234626/log.txt.gz 2026-03-12 09:17:45,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:45,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:45,966 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f74bce73bcf0234626/lava-logs.yaml 2026-03-12 09:17:45,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:46,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:46,165 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f74bce73bcf0234626/lava_callback.json.gz 2026-03-12 09:17:46,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:46,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f74bce73bcf0234626 HTTP/1.1" 200 None 2026-03-12 09:17:46,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:46,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:46,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:46,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283f74bce73bcf0234626 HTTP/1.1" 200 None 2026-03-12 09:17:46,328 - lava_callback - INFO - Completed processing callback for node 69b283f74bce73bcf0234626 INFO: 172.20.0.1:34514 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:17:48,588 - lava_callback - INFO - Received callback for node 69b2841b4bce73bcf02346b5 INFO: 172.20.0.1:34518 - "POST /node/69b2841b4bce73bcf02346b5 HTTP/1.0" 202 Accepted 2026-03-12 09:17:48,594 - lava_callback - INFO - Starting async job submit for node 69b2841b4bce73bcf02346b5 2026-03-12 09:17:48,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:48,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2841b4bce73bcf02346b5 HTTP/1.1" 200 None 2026-03-12 09:17:48,641 - lava_callback - DEBUG - Processing node 69b2841b4bce73bcf02346b5: baseline-arm64-broonie 2026-03-12 09:17:48,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:48,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:48,978 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841b4bce73bcf02346b5/log.txt.gz 2026-03-12 09:17:49,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:49,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:49,290 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841b4bce73bcf02346b5/lava-logs.yaml 2026-03-12 09:17:49,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:17:49,437 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:17:49,439 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841b4bce73bcf02346b5/lava_callback.json.gz 2026-03-12 09:17:49,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:49,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2841b4bce73bcf02346b5 HTTP/1.1" 200 None 2026-03-12 09:17:49,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:49,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:17:49,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:49,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2841b4bce73bcf02346b5 HTTP/1.1" 200 None 2026-03-12 09:17:49,528 - lava_callback - INFO - Completed processing callback for node 69b2841b4bce73bcf02346b5 INFO: 172.20.0.1:45684 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:17:54,799 - lava_callback - INFO - Received callback for node 69afe1adae557f2600f0fbce INFO: 172.20.0.1:45700 - "POST /node/69afe1adae557f2600f0fbce HTTP/1.0" 202 Accepted 2026-03-12 09:17:54,804 - lava_callback - INFO - Starting async job submit for node 69afe1adae557f2600f0fbce 2026-03-12 09:17:54,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:54,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1adae557f2600f0fbce HTTP/1.1" 200 None 2026-03-12 09:17:54,823 - lava_callback - DEBUG - Processing node 69afe1adae557f2600f0fbce: baseline-arm64-baylibre 2026-03-12 09:17:54,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1adae557f2600f0fbce 2026-03-12 09:17:54,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:54,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1adae557f2600f0fbce HTTP/1.1" 200 None 2026-03-12 09:17:54,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:54,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 09:17:54,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:17:54,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1adae557f2600f0fbce HTTP/1.1" 200 None 2026-03-12 09:17:54,881 - lava_callback - ERROR - Error processing callback for node 69afe1adae557f2600f0fbce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44438 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:18:13,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:13,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60910 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:18:34,817 - lava_callback - INFO - Received callback for node 69afe1d8ae557f2600f0fc59 INFO: 172.20.0.1:47640 - "POST /node/69afe1d8ae557f2600f0fc59 HTTP/1.0" 202 Accepted 2026-03-12 09:18:34,823 - lava_callback - INFO - Starting async job submit for node 69afe1d8ae557f2600f0fc59 2026-03-12 09:18:34,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:34,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1d8ae557f2600f0fc59 HTTP/1.1" 200 None 2026-03-12 09:18:34,839 - lava_callback - DEBUG - Processing node 69afe1d8ae557f2600f0fc59: baseline-arm64-baylibre 2026-03-12 09:18:34,839 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1d8ae557f2600f0fc59 2026-03-12 09:18:34,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:34,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1d8ae557f2600f0fc59 HTTP/1.1" 200 None 2026-03-12 09:18:34,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:34,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:18:34,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:34,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1d8ae557f2600f0fc59 HTTP/1.1" 200 None 2026-03-12 09:18:34,895 - lava_callback - ERROR - Error processing callback for node 69afe1d8ae557f2600f0fc59: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:18:35,488 - lava_callback - INFO - Received callback for node 69afe1daae557f2600f0fc5c INFO: 172.20.0.1:47656 - "POST /node/69afe1daae557f2600f0fc5c HTTP/1.0" 202 Accepted 2026-03-12 09:18:35,494 - lava_callback - INFO - Starting async job submit for node 69afe1daae557f2600f0fc5c 2026-03-12 09:18:35,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:35,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1daae557f2600f0fc5c HTTP/1.1" 200 None 2026-03-12 09:18:35,513 - lava_callback - DEBUG - Processing node 69afe1daae557f2600f0fc5c: baseline-arm64-baylibre 2026-03-12 09:18:35,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1daae557f2600f0fc5c 2026-03-12 09:18:35,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:35,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1daae557f2600f0fc5c HTTP/1.1" 200 None 2026-03-12 09:18:35,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:35,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:18:35,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:35,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1daae557f2600f0fc5c HTTP/1.1" 200 None 2026-03-12 09:18:35,569 - lava_callback - ERROR - Error processing callback for node 69afe1daae557f2600f0fc5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:18:36,176 - lava_callback - INFO - Received callback for node 69afe1dbae557f2600f0fc76 INFO: 172.20.0.1:47658 - "POST /node/69afe1dbae557f2600f0fc76 HTTP/1.0" 202 Accepted 2026-03-12 09:18:36,182 - lava_callback - INFO - Starting async job submit for node 69afe1dbae557f2600f0fc76 2026-03-12 09:18:36,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:36,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1dbae557f2600f0fc76 HTTP/1.1" 200 None 2026-03-12 09:18:36,201 - lava_callback - DEBUG - Processing node 69afe1dbae557f2600f0fc76: baseline-arm64-baylibre 2026-03-12 09:18:36,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1dbae557f2600f0fc76 2026-03-12 09:18:36,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:36,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1dbae557f2600f0fc76 HTTP/1.1" 200 None 2026-03-12 09:18:36,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:36,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:18:36,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:36,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1dbae557f2600f0fc76 HTTP/1.1" 200 None 2026-03-12 09:18:36,268 - lava_callback - ERROR - Error processing callback for node 69afe1dbae557f2600f0fc76: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47670 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:18:36,819 - lava_callback - INFO - Received callback for node 69afe1deae557f2600f0fc80 INFO: 172.20.0.1:47682 - "POST /node/69afe1deae557f2600f0fc80 HTTP/1.0" 202 Accepted 2026-03-12 09:18:36,825 - lava_callback - INFO - Starting async job submit for node 69afe1deae557f2600f0fc80 2026-03-12 09:18:36,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:36,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1deae557f2600f0fc80 HTTP/1.1" 200 None 2026-03-12 09:18:36,840 - lava_callback - DEBUG - Processing node 69afe1deae557f2600f0fc80: baseline-arm64-baylibre 2026-03-12 09:18:36,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1deae557f2600f0fc80 2026-03-12 09:18:36,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:36,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1deae557f2600f0fc80 HTTP/1.1" 200 None 2026-03-12 09:18:36,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:36,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:18:36,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:36,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1deae557f2600f0fc80 HTTP/1.1" 200 None 2026-03-12 09:18:36,911 - lava_callback - ERROR - Error processing callback for node 69afe1deae557f2600f0fc80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:18:37,492 - lava_callback - INFO - Received callback for node 69afe1dfae557f2600f0fc83 INFO: 172.20.0.1:47688 - "POST /node/69afe1dfae557f2600f0fc83 HTTP/1.0" 202 Accepted 2026-03-12 09:18:37,498 - lava_callback - INFO - Starting async job submit for node 69afe1dfae557f2600f0fc83 2026-03-12 09:18:37,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:37,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1dfae557f2600f0fc83 HTTP/1.1" 200 None 2026-03-12 09:18:37,513 - lava_callback - DEBUG - Processing node 69afe1dfae557f2600f0fc83: baseline-arm64-baylibre 2026-03-12 09:18:37,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1dfae557f2600f0fc83 2026-03-12 09:18:37,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:37,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1dfae557f2600f0fc83 HTTP/1.1" 200 None 2026-03-12 09:18:37,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:37,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:18:37,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:37,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1dfae557f2600f0fc83 HTTP/1.1" 200 None 2026-03-12 09:18:37,573 - lava_callback - ERROR - Error processing callback for node 69afe1dfae557f2600f0fc83: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:18:38,152 - lava_callback - INFO - Received callback for node 69afe1e1ae557f2600f0fc86 INFO: 172.20.0.1:47696 - "POST /node/69afe1e1ae557f2600f0fc86 HTTP/1.0" 202 Accepted 2026-03-12 09:18:38,158 - lava_callback - INFO - Starting async job submit for node 69afe1e1ae557f2600f0fc86 2026-03-12 09:18:38,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:38,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1e1ae557f2600f0fc86 HTTP/1.1" 200 None 2026-03-12 09:18:38,174 - lava_callback - DEBUG - Processing node 69afe1e1ae557f2600f0fc86: baseline-arm64-baylibre 2026-03-12 09:18:38,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1e1ae557f2600f0fc86 2026-03-12 09:18:38,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:38,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1e1ae557f2600f0fc86 HTTP/1.1" 200 None 2026-03-12 09:18:38,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:38,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:18:38,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:38,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1e1ae557f2600f0fc86 HTTP/1.1" 200 None 2026-03-12 09:18:38,235 - lava_callback - ERROR - Error processing callback for node 69afe1e1ae557f2600f0fc86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:18:43,834 - lava_callback - INFO - Received callback for node 69afe1f1ae557f2600f0fcb5 INFO: 172.20.0.1:34116 - "POST /node/69afe1f1ae557f2600f0fcb5 HTTP/1.0" 202 Accepted 2026-03-12 09:18:43,839 - lava_callback - INFO - Starting async job submit for node 69afe1f1ae557f2600f0fcb5 2026-03-12 09:18:43,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:43,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f1ae557f2600f0fcb5 HTTP/1.1" 200 None 2026-03-12 09:18:43,857 - lava_callback - DEBUG - Processing node 69afe1f1ae557f2600f0fcb5: baseline-arm64-clabbe 2026-03-12 09:18:43,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1f1ae557f2600f0fcb5 2026-03-12 09:18:43,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:43,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f1ae557f2600f0fcb5 HTTP/1.1" 200 None 2026-03-12 09:18:43,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:43,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:18:43,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:43,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1f1ae557f2600f0fcb5 HTTP/1.1" 200 None 2026-03-12 09:18:43,939 - lava_callback - ERROR - Error processing callback for node 69afe1f1ae557f2600f0fcb5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34130 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:18:48,340 - lava_callback - INFO - Received callback for node 69b284204bce73bcf02346c1 INFO: 172.20.0.1:34144 - "POST /node/69b284204bce73bcf02346c1 HTTP/1.0" 202 Accepted 2026-03-12 09:18:48,346 - lava_callback - INFO - Starting async job submit for node 69b284204bce73bcf02346c1 2026-03-12 09:18:48,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:48,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284204bce73bcf02346c1 HTTP/1.1" 200 None 2026-03-12 09:18:48,384 - lava_callback - DEBUG - Processing node 69b284204bce73bcf02346c1: baseline-arm64-broonie 2026-03-12 09:18:48,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:18:48,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:18:48,809 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284204bce73bcf02346c1/log.txt.gz 2026-03-12 09:18:48,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:18:49,189 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:18:49,191 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284204bce73bcf02346c1/lava-logs.yaml 2026-03-12 09:18:49,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:18:49,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:18:49,366 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b284204bce73bcf02346c1/lava_callback.json.gz 2026-03-12 09:18:49,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:49,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284204bce73bcf02346c1 HTTP/1.1" 200 None 2026-03-12 09:18:49,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:49,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:18:49,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:49,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284204bce73bcf02346c1 HTTP/1.1" 200 None 2026-03-12 09:18:49,471 - lava_callback - INFO - Completed processing callback for node 69b284204bce73bcf02346c1 2026-03-12 09:18:53,098 - lava_callback - INFO - Received callback for node 69b284e74bce73bcf023482b INFO: 172.20.0.1:52354 - "POST /node/69b284e74bce73bcf023482b HTTP/1.0" 202 Accepted 2026-03-12 09:18:53,104 - lava_callback - INFO - Starting async job submit for node 69b284e74bce73bcf023482b 2026-03-12 09:18:53,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:53,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284e74bce73bcf023482b HTTP/1.1" 200 None 2026-03-12 09:18:53,122 - lava_callback - DEBUG - Processing node 69b284e74bce73bcf023482b: baseline-arm 2026-03-12 09:18:53,127 - lava_callback - WARNING - Failed to upload log 2026-03-12 09:18:53,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:18:53,274 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:18:53,276 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b284e74bce73bcf023482b/lava-logs.yaml 2026-03-12 09:18:53,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:18:53,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:18:53,405 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b284e74bce73bcf023482b/lava_callback.json.gz 2026-03-12 09:18:53,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:53,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284e74bce73bcf023482b HTTP/1.1" 200 None 2026-03-12 09:18:53,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:53,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 09:18:53,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:18:53,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284e74bce73bcf023482b HTTP/1.1" 200 None 2026-03-12 09:18:53,466 - lava_callback - INFO - Completed processing callback for node 69b284e74bce73bcf023482b INFO: 172.20.0.1:52364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52370 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:19:03,909 - lava_callback - INFO - Received callback for node 69afe1f3ae557f2600f0fcbb INFO: 172.20.0.1:53178 - "POST /node/69afe1f3ae557f2600f0fcbb HTTP/1.0" 202 Accepted 2026-03-12 09:19:03,915 - lava_callback - INFO - Starting async job submit for node 69afe1f3ae557f2600f0fcbb 2026-03-12 09:19:03,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:03,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f3ae557f2600f0fcbb HTTP/1.1" 200 None 2026-03-12 09:19:03,937 - lava_callback - DEBUG - Processing node 69afe1f3ae557f2600f0fcbb: baseline-arm64-clabbe 2026-03-12 09:19:03,937 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1f3ae557f2600f0fcbb 2026-03-12 09:19:03,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:03,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f3ae557f2600f0fcbb HTTP/1.1" 200 None 2026-03-12 09:19:03,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:03,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:19:03,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:04,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1f3ae557f2600f0fcbb HTTP/1.1" 200 None 2026-03-12 09:19:04,012 - lava_callback - ERROR - Error processing callback for node 69afe1f3ae557f2600f0fcbb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:19:04,617 - lava_callback - INFO - Received callback for node 69afe1f3ae557f2600f0fcbe INFO: 172.20.0.1:53180 - "POST /node/69afe1f3ae557f2600f0fcbe HTTP/1.0" 202 Accepted 2026-03-12 09:19:04,623 - lava_callback - INFO - Starting async job submit for node 69afe1f3ae557f2600f0fcbe 2026-03-12 09:19:04,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:04,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f3ae557f2600f0fcbe HTTP/1.1" 200 None 2026-03-12 09:19:04,639 - lava_callback - DEBUG - Processing node 69afe1f3ae557f2600f0fcbe: baseline-arm64-clabbe 2026-03-12 09:19:04,639 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1f3ae557f2600f0fcbe 2026-03-12 09:19:04,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:04,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f3ae557f2600f0fcbe HTTP/1.1" 200 None 2026-03-12 09:19:04,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:04,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:19:04,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:04,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1f3ae557f2600f0fcbe HTTP/1.1" 200 None 2026-03-12 09:19:04,691 - lava_callback - ERROR - Error processing callback for node 69afe1f3ae557f2600f0fcbe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:19:05,297 - lava_callback - INFO - Received callback for node 69afe1f4ae557f2600f0fcc1 INFO: 172.20.0.1:53190 - "POST /node/69afe1f4ae557f2600f0fcc1 HTTP/1.0" 202 Accepted 2026-03-12 09:19:05,302 - lava_callback - INFO - Starting async job submit for node 69afe1f4ae557f2600f0fcc1 2026-03-12 09:19:05,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:05,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f4ae557f2600f0fcc1 HTTP/1.1" 200 None 2026-03-12 09:19:05,323 - lava_callback - DEBUG - Processing node 69afe1f4ae557f2600f0fcc1: baseline-arm64-clabbe 2026-03-12 09:19:05,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1f4ae557f2600f0fcc1 2026-03-12 09:19:05,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:05,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f4ae557f2600f0fcc1 HTTP/1.1" 200 None 2026-03-12 09:19:05,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:05,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:19:05,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:05,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1f4ae557f2600f0fcc1 HTTP/1.1" 200 None 2026-03-12 09:19:05,378 - lava_callback - ERROR - Error processing callback for node 69afe1f4ae557f2600f0fcc1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:19:05,954 - lava_callback - INFO - Received callback for node 69afe1f7ae557f2600f0fcca INFO: 172.20.0.1:53202 - "POST /node/69afe1f7ae557f2600f0fcca HTTP/1.0" 202 Accepted 2026-03-12 09:19:05,960 - lava_callback - INFO - Starting async job submit for node 69afe1f7ae557f2600f0fcca 2026-03-12 09:19:05,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:05,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f7ae557f2600f0fcca HTTP/1.1" 200 None 2026-03-12 09:19:05,976 - lava_callback - DEBUG - Processing node 69afe1f7ae557f2600f0fcca: baseline-arm64-clabbe 2026-03-12 09:19:05,976 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1f7ae557f2600f0fcca 2026-03-12 09:19:05,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:05,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f7ae557f2600f0fcca HTTP/1.1" 200 None 2026-03-12 09:19:05,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:06,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:19:06,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:06,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1f7ae557f2600f0fcca HTTP/1.1" 200 None 2026-03-12 09:19:06,034 - lava_callback - ERROR - Error processing callback for node 69afe1f7ae557f2600f0fcca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53210 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:19:06,609 - lava_callback - INFO - Received callback for node 69afe1f6ae557f2600f0fcc7 INFO: 172.20.0.1:53226 - "POST /node/69afe1f6ae557f2600f0fcc7 HTTP/1.0" 202 Accepted 2026-03-12 09:19:06,615 - lava_callback - INFO - Starting async job submit for node 69afe1f6ae557f2600f0fcc7 2026-03-12 09:19:06,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:06,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f6ae557f2600f0fcc7 HTTP/1.1" 200 None 2026-03-12 09:19:06,630 - lava_callback - DEBUG - Processing node 69afe1f6ae557f2600f0fcc7: baseline-arm64-clabbe 2026-03-12 09:19:06,631 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe1f6ae557f2600f0fcc7 2026-03-12 09:19:06,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:06,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe1f6ae557f2600f0fcc7 HTTP/1.1" 200 None 2026-03-12 09:19:06,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:06,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd839ae557f2600f0f045 HTTP/1.1" 200 None 2026-03-12 09:19:06,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:06,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe1f6ae557f2600f0fcc7 HTTP/1.1" 200 None 2026-03-12 09:19:06,697 - lava_callback - ERROR - Error processing callback for node 69afe1f6ae557f2600f0fcc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:19:08,623 - lava_callback - INFO - Received callback for node 69b283f24bce73bcf023461a INFO: 172.20.0.1:53234 - "POST /node/69b283f24bce73bcf023461a HTTP/1.0" 202 Accepted 2026-03-12 09:19:08,628 - lava_callback - INFO - Starting async job submit for node 69b283f24bce73bcf023461a 2026-03-12 09:19:08,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:08,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f24bce73bcf023461a HTTP/1.1" 200 None 2026-03-12 09:19:08,675 - lava_callback - DEBUG - Processing node 69b283f24bce73bcf023461a: baseline-arm64 2026-03-12 09:19:08,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:19:09,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:19:09,141 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f24bce73bcf023461a/log.txt.gz 2026-03-12 09:19:09,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:19:09,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:19:09,555 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f24bce73bcf023461a/lava-logs.yaml 2026-03-12 09:19:09,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:19:09,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:19:09,694 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283f24bce73bcf023461a/lava_callback.json.gz 2026-03-12 09:19:09,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:09,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283f24bce73bcf023461a HTTP/1.1" 200 None 2026-03-12 09:19:09,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:09,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:19:09,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:09,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283f24bce73bcf023461a HTTP/1.1" 200 None 2026-03-12 09:19:09,775 - lava_callback - INFO - Completed processing callback for node 69b283f24bce73bcf023461a 2026-03-12 09:19:13,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:19:13,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58390 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:20:13,017 - lava_callback - INFO - Received callback for node 69b284024bce73bcf023465d INFO: 172.20.0.1:51356 - "POST /node/69b284024bce73bcf023465d HTTP/1.0" 202 Accepted 2026-03-12 09:20:13,023 - lava_callback - INFO - Starting async job submit for node 69b284024bce73bcf023465d 2026-03-12 09:20:13,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:20:13,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284024bce73bcf023465d HTTP/1.1" 200 None 2026-03-12 09:20:13,057 - lava_callback - DEBUG - Processing node 69b284024bce73bcf023465d: baseline-nfs-arm64 2026-03-12 09:20:13,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:20:13,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:20:13,638 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284024bce73bcf023465d/log.txt.gz 2026-03-12 09:20:13,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:20:14,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:20:14,098 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284024bce73bcf023465d/lava-logs.yaml 2026-03-12 09:20:14,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:20:14,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:20:14,259 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284024bce73bcf023465d/lava_callback.json.gz 2026-03-12 09:20:14,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:20:14,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284024bce73bcf023465d HTTP/1.1" 200 None 2026-03-12 09:20:14,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:20:14,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:20:14,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:20:14,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284024bce73bcf023465d HTTP/1.1" 200 None 2026-03-12 09:20:14,334 - lava_callback - INFO - Completed processing callback for node 69b284024bce73bcf023465d INFO: 172.20.0.1:51366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59670 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:20:34,177 - lava_callback - INFO - Received callback for node 69b284334bce73bcf02346f6 INFO: 172.20.0.1:33322 - "POST /node/69b284334bce73bcf02346f6 HTTP/1.0" 202 Accepted 2026-03-12 09:20:34,187 - lava_callback - INFO - Starting async job submit for node 69b284334bce73bcf02346f6 2026-03-12 09:20:34,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:20:34,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284334bce73bcf02346f6 HTTP/1.1" 200 None 2026-03-12 09:20:34,251 - lava_callback - DEBUG - Processing node 69b284334bce73bcf02346f6: baseline-arm64-foundriesio 2026-03-12 09:20:35,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:20:35,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:20:35,507 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b284334bce73bcf02346f6/log.txt.gz 2026-03-12 09:20:36,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33334 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:20:36,529 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:20:36,532 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b284334bce73bcf02346f6/lava-logs.yaml 2026-03-12 09:20:36,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:20:36,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:20:36,751 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b284334bce73bcf02346f6/lava_callback.json.gz DEBUG: 69b284334bce73bcf02346f6 Transitting job node result: pass -> incomplete 2026-03-12 09:20:36,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:20:36,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284334bce73bcf02346f6 HTTP/1.1" 200 None 2026-03-12 09:20:36,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:20:36,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:20:36,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:20:36,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284334bce73bcf02346f6 HTTP/1.1" 200 None 2026-03-12 09:20:36,887 - lava_callback - INFO - Completed processing callback for node 69b284334bce73bcf02346f6 2026-03-12 09:20:43,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:20:43,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47086 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:21:18,962 - lava_callback - INFO - Received callback for node 69b2840a4bce73bcf0234673 INFO: 172.20.0.1:47102 - "POST /node/69b2840a4bce73bcf0234673 HTTP/1.0" 202 Accepted 2026-03-12 09:21:18,969 - lava_callback - INFO - Starting async job submit for node 69b2840a4bce73bcf0234673 2026-03-12 09:21:18,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:21:19,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2840a4bce73bcf0234673 HTTP/1.1" 200 None 2026-03-12 09:21:19,008 - lava_callback - DEBUG - Processing node 69b2840a4bce73bcf0234673: baseline-nfs-arm64 2026-03-12 09:21:19,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:21:19,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:21:19,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840a4bce73bcf0234673/log.txt.gz 2026-03-12 09:21:20,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:21:20,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:21:20,735 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840a4bce73bcf0234673/lava-logs.yaml 2026-03-12 09:21:20,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:21:20,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:21:20,922 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2840a4bce73bcf0234673/lava_callback.json.gz 2026-03-12 09:21:20,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:21:20,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2840a4bce73bcf0234673 HTTP/1.1" 200 None 2026-03-12 09:21:20,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:21:20,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:21:20,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:21:21,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2840a4bce73bcf0234673 HTTP/1.1" 200 None 2026-03-12 09:21:21,015 - lava_callback - INFO - Completed processing callback for node 69b2840a4bce73bcf0234673 INFO: 172.20.0.1:44610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35094 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:21:43,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:21:43,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59522 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:22:14,435 - lava_callback - INFO - Received callback for node 69b2841d4bce73bcf02346bb INFO: 172.20.0.1:40614 - "POST /node/69b2841d4bce73bcf02346bb HTTP/1.0" 202 Accepted 2026-03-12 09:22:14,441 - lava_callback - INFO - Starting async job submit for node 69b2841d4bce73bcf02346bb 2026-03-12 09:22:14,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:22:14,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2841d4bce73bcf02346bb HTTP/1.1" 200 None 2026-03-12 09:22:14,495 - lava_callback - DEBUG - Processing node 69b2841d4bce73bcf02346bb: baseline-arm64-broonie 2026-03-12 09:22:15,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:40620 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 09:22:15,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:22:15,201 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841d4bce73bcf02346bb/log.txt.gz 2026-03-12 09:22:15,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:22:15,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:22:15,722 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841d4bce73bcf02346bb/lava-logs.yaml 2026-03-12 09:22:15,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:22:15,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:22:15,890 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841d4bce73bcf02346bb/lava_callback.json.gz DEBUG: 69b2841d4bce73bcf02346bb Transitting job node result: pass -> incomplete 2026-03-12 09:22:15,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:22:15,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2841d4bce73bcf02346bb HTTP/1.1" 200 None 2026-03-12 09:22:15,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:22:15,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:22:15,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:22:15,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2841d4bce73bcf02346bb HTTP/1.1" 200 None 2026-03-12 09:22:15,990 - lava_callback - INFO - Completed processing callback for node 69b2841d4bce73bcf02346bb INFO: 172.20.0.1:40636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48524 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:22:43,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:22:43,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35818 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:23:25,053 - lava_callback - INFO - Received callback for node 69b283ff4bce73bcf0234655 INFO: 172.20.0.1:44030 - "POST /node/69b283ff4bce73bcf0234655 HTTP/1.0" 202 Accepted 2026-03-12 09:23:25,060 - lava_callback - INFO - Starting async job submit for node 69b283ff4bce73bcf0234655 2026-03-12 09:23:25,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:23:25,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ff4bce73bcf0234655 HTTP/1.1" 200 None 2026-03-12 09:23:25,110 - lava_callback - DEBUG - Processing node 69b283ff4bce73bcf0234655: baseline-arm64 2026-03-12 09:23:25,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:23:25,650 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:23:25,652 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ff4bce73bcf0234655/log.txt.gz 2026-03-12 09:23:25,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:23:26,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:23:26,092 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ff4bce73bcf0234655/lava-logs.yaml 2026-03-12 09:23:26,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:23:26,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:23:26,328 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b283ff4bce73bcf0234655/lava_callback.json.gz DEBUG: Setup failed for 69b283ff4bce73bcf0234655. Transitting job node result: incomplete -> fail 2026-03-12 09:23:26,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:23:26,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283ff4bce73bcf0234655 HTTP/1.1" 200 None 2026-03-12 09:23:26,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:23:26,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:23:26,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:23:26,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283ff4bce73bcf0234655 HTTP/1.1" 200 None INFO: 172.20.0.1:44040 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:23:26,558 - lava_callback - INFO - Completed processing callback for node 69b283ff4bce73bcf0234655 INFO: 172.20.0.1:53050 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:23:43,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:23:43,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51680 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:24:24,167 - lava_callback - INFO - Received callback for node 69afe33fae557f2600f0fe12 INFO: 172.20.0.1:47694 - "POST /node/69afe33fae557f2600f0fe12 HTTP/1.0" 202 Accepted 2026-03-12 09:24:24,172 - lava_callback - INFO - Starting async job submit for node 69afe33fae557f2600f0fe12 2026-03-12 09:24:24,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:24:24,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe33fae557f2600f0fe12 HTTP/1.1" 200 None 2026-03-12 09:24:24,203 - lava_callback - DEBUG - Processing node 69afe33fae557f2600f0fe12: baseline-arm64-clabbe 2026-03-12 09:24:24,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe33fae557f2600f0fe12 2026-03-12 09:24:24,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:24:24,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe33fae557f2600f0fe12 HTTP/1.1" 200 None 2026-03-12 09:24:24,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:24:24,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 09:24:24,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:24:24,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe33fae557f2600f0fe12 HTTP/1.1" 200 None 2026-03-12 09:24:24,261 - lava_callback - ERROR - Error processing callback for node 69afe33fae557f2600f0fe12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50546 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:25:44,300 - lava_callback - INFO - Received callback for node 69afe390ae557f2600f0fe3a INFO: 172.20.0.1:51254 - "POST /node/69afe390ae557f2600f0fe3a HTTP/1.0" 202 Accepted 2026-03-12 09:25:44,306 - lava_callback - INFO - Starting async job submit for node 69afe390ae557f2600f0fe3a 2026-03-12 09:25:44,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:25:44,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe390ae557f2600f0fe3a HTTP/1.1" 200 None 2026-03-12 09:25:44,322 - lava_callback - DEBUG - Processing node 69afe390ae557f2600f0fe3a: baseline-arm64-clabbe 2026-03-12 09:25:44,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe390ae557f2600f0fe3a 2026-03-12 09:25:44,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:25:44,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe390ae557f2600f0fe3a HTTP/1.1" 200 None 2026-03-12 09:25:44,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:25:44,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 09:25:44,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:25:44,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe390ae557f2600f0fe3a HTTP/1.1" 200 None 2026-03-12 09:25:44,383 - lava_callback - ERROR - Error processing callback for node 69afe390ae557f2600f0fe3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54652 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:26:55,432 - lava_callback - INFO - Received callback for node 69afe3d8ae557f2600f0fe58 INFO: 172.20.0.1:54658 - "POST /node/69afe3d8ae557f2600f0fe58 HTTP/1.0" 202 Accepted 2026-03-12 09:26:55,438 - lava_callback - INFO - Starting async job submit for node 69afe3d8ae557f2600f0fe58 2026-03-12 09:26:55,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:26:55,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe3d8ae557f2600f0fe58 HTTP/1.1" 200 None 2026-03-12 09:26:55,453 - lava_callback - DEBUG - Processing node 69afe3d8ae557f2600f0fe58: baseline-arm64-baylibre 2026-03-12 09:26:55,454 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe3d8ae557f2600f0fe58 2026-03-12 09:26:55,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:26:55,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe3d8ae557f2600f0fe58 HTTP/1.1" 200 None 2026-03-12 09:26:55,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:26:55,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7ea227c78d2166ff857f2 HTTP/1.1" 200 None 2026-03-12 09:26:55,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:26:55,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe3d8ae557f2600f0fe58 HTTP/1.1" 200 None 2026-03-12 09:26:55,526 - lava_callback - ERROR - Error processing callback for node 69afe3d8ae557f2600f0fe58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 09:28:15,392 - lava_callback - INFO - Received callback for node 69afe42cae557f2600f0fe77 INFO: 172.20.0.1:41830 - "POST /node/69afe42cae557f2600f0fe77 HTTP/1.0" 202 Accepted 2026-03-12 09:28:15,398 - lava_callback - INFO - Starting async job submit for node 69afe42cae557f2600f0fe77 2026-03-12 09:28:15,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:28:15,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe42cae557f2600f0fe77 HTTP/1.1" 200 None 2026-03-12 09:28:15,413 - lava_callback - DEBUG - Processing node 69afe42cae557f2600f0fe77: baseline-arm64-baylibre 2026-03-12 09:28:15,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe42cae557f2600f0fe77 2026-03-12 09:28:15,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:28:15,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe42cae557f2600f0fe77 HTTP/1.1" 200 None 2026-03-12 09:28:15,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:28:15,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7ea227c78d2166ff857f2 HTTP/1.1" 200 None 2026-03-12 09:28:15,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:28:15,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe42cae557f2600f0fe77 HTTP/1.1" 200 None 2026-03-12 09:28:15,470 - lava_callback - ERROR - Error processing callback for node 69afe42cae557f2600f0fe77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57228 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:29:35,421 - lava_callback - INFO - Received callback for node 69afe47cae557f2600f0fe96 INFO: 172.20.0.1:34784 - "POST /node/69afe47cae557f2600f0fe96 HTTP/1.0" 202 Accepted 2026-03-12 09:29:35,426 - lava_callback - INFO - Starting async job submit for node 69afe47cae557f2600f0fe96 2026-03-12 09:29:35,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:29:35,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe47cae557f2600f0fe96 HTTP/1.1" 200 None 2026-03-12 09:29:35,442 - lava_callback - DEBUG - Processing node 69afe47cae557f2600f0fe96: baseline-arm64-baylibre 2026-03-12 09:29:35,442 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe47cae557f2600f0fe96 2026-03-12 09:29:35,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:29:35,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe47cae557f2600f0fe96 HTTP/1.1" 200 None 2026-03-12 09:29:35,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:29:35,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7ea227c78d2166ff857f2 HTTP/1.1" 200 None 2026-03-12 09:29:35,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:29:35,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe47cae557f2600f0fe96 HTTP/1.1" 200 None 2026-03-12 09:29:35,508 - lava_callback - ERROR - Error processing callback for node 69afe47cae557f2600f0fe96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46126 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:31:08,926 - lava_callback - INFO - Received callback for node 69b283cb4bce73bcf02345e6 INFO: 172.20.0.1:46138 - "POST /node/69b283cb4bce73bcf02345e6 HTTP/1.0" 202 Accepted 2026-03-12 09:31:08,932 - lava_callback - INFO - Starting async job submit for node 69b283cb4bce73bcf02345e6 2026-03-12 09:31:08,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:31:09,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283cb4bce73bcf02345e6 HTTP/1.1" 200 None 2026-03-12 09:31:09,004 - lava_callback - DEBUG - Processing node 69b283cb4bce73bcf02345e6: ltp-smoke 2026-03-12 09:31:09,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:31:09,804 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:31:09,807 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b283cb4bce73bcf02345e6/log.txt.gz 2026-03-12 09:31:10,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:31:10,343 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:31:10,346 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b283cb4bce73bcf02345e6/lava-logs.yaml 2026-03-12 09:31:10,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:31:10,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:31:10,549 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b283cb4bce73bcf02345e6/lava_callback.json.gz 2026-03-12 09:31:10,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:31:10,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b283cb4bce73bcf02345e6 HTTP/1.1" 200 None 2026-03-12 09:31:10,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:31:10,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2766c4bce73bcf0233d1a HTTP/1.1" 200 None 2026-03-12 09:31:10,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:31:10,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b283cb4bce73bcf02345e6 HTTP/1.1" 200 None 2026-03-12 09:31:10,830 - lava_callback - INFO - Completed processing callback for node 69b283cb4bce73bcf02345e6 2026-03-12 09:31:13,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:31:13,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33626 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 09:34:15,799 - lava_callback - INFO - Received callback for node 69afe58cae557f2600f0feec INFO: 172.20.0.1:33634 - "POST /node/69afe58cae557f2600f0feec HTTP/1.0" 202 Accepted 2026-03-12 09:34:15,806 - lava_callback - INFO - Starting async job submit for node 69afe58cae557f2600f0feec 2026-03-12 09:34:15,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:34:15,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe58cae557f2600f0feec HTTP/1.1" 200 None 2026-03-12 09:34:15,824 - lava_callback - DEBUG - Processing node 69afe58cae557f2600f0feec: baseline-arm64-baylibre 2026-03-12 09:34:15,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe58cae557f2600f0feec 2026-03-12 09:34:15,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:34:15,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe58cae557f2600f0feec HTTP/1.1" 200 None 2026-03-12 09:34:15,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:34:15,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-12 09:34:15,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:34:15,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe58cae557f2600f0feec HTTP/1.1" 200 None 2026-03-12 09:34:15,898 - lava_callback - ERROR - Error processing callback for node 69afe58cae557f2600f0feec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41388 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:35:23,354 - lava_callback - INFO - Received callback for node 69b284044bce73bcf0234664 INFO: 172.20.0.1:43602 - "POST /node/69b284044bce73bcf0234664 HTTP/1.0" 202 Accepted 2026-03-12 09:35:23,365 - lava_callback - INFO - Starting async job submit for node 69b284044bce73bcf0234664 2026-03-12 09:35:23,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:35:23,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284044bce73bcf0234664 HTTP/1.1" 200 None 2026-03-12 09:35:23,434 - lava_callback - DEBUG - Processing node 69b284044bce73bcf0234664: baseline-nfs-arm64 2026-03-12 09:35:25,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:35:25,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:35:25,483 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284044bce73bcf0234664/log.txt.gz 2026-03-12 09:35:26,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:43618 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:35:26,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:35:26,824 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284044bce73bcf0234664/lava-logs.yaml 2026-03-12 09:35:26,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:35:27,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:35:27,098 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b284044bce73bcf0234664/lava_callback.json.gz DEBUG: Setup failed for 69b284044bce73bcf0234664. Transitting job node result: incomplete -> fail 2026-03-12 09:35:27,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:35:27,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b284044bce73bcf0234664 HTTP/1.1" 200 None 2026-03-12 09:35:27,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:35:27,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:35:27,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:35:27,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b284044bce73bcf0234664 HTTP/1.1" 200 None 2026-03-12 09:35:27,413 - lava_callback - INFO - Completed processing callback for node 69b284044bce73bcf0234664 INFO: 172.20.0.1:42012 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:35:43,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:35:43,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:36:25,864 - lava_callback - INFO - Received callback for node 69b2842b4bce73bcf02346dd INFO: 172.20.0.1:56160 - "POST /node/69b2842b4bce73bcf02346dd HTTP/1.0" 202 Accepted 2026-03-12 09:36:25,870 - lava_callback - INFO - Starting async job submit for node 69b2842b4bce73bcf02346dd 2026-03-12 09:36:25,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:36:25,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2842b4bce73bcf02346dd HTTP/1.1" 200 None 2026-03-12 09:36:25,927 - lava_callback - DEBUG - Processing node 69b2842b4bce73bcf02346dd: baseline-arm64-clabbe 2026-03-12 09:36:26,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:56166 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:36:26,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:36:26,643 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b2842b4bce73bcf02346dd/log.txt.gz 2026-03-12 09:36:26,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:36:27,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:36:27,214 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b2842b4bce73bcf02346dd/lava-logs.yaml 2026-03-12 09:36:27,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:36:27,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:36:27,385 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b2842b4bce73bcf02346dd/lava_callback.json.gz DEBUG: Setup failed for 69b2842b4bce73bcf02346dd. Transitting job node result: incomplete -> fail 2026-03-12 09:36:27,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:36:27,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2842b4bce73bcf02346dd HTTP/1.1" 200 None 2026-03-12 09:36:27,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:36:27,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:36:27,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:36:27,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2842b4bce73bcf02346dd HTTP/1.1" 200 None 2026-03-12 09:36:27,618 - lava_callback - INFO - Completed processing callback for node 69b2842b4bce73bcf02346dd INFO: 172.20.0.1:37074 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:36:43,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:36:43,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57862 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:36:55,898 - lava_callback - INFO - Received callback for node 69afe62eae557f2600f0ff0d INFO: 172.20.0.1:57874 - "POST /node/69afe62eae557f2600f0ff0d HTTP/1.0" 202 Accepted 2026-03-12 09:36:55,903 - lava_callback - INFO - Starting async job submit for node 69afe62eae557f2600f0ff0d 2026-03-12 09:36:55,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:36:55,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe62eae557f2600f0ff0d HTTP/1.1" 200 None 2026-03-12 09:36:55,920 - lava_callback - DEBUG - Processing node 69afe62eae557f2600f0ff0d: baseline-arm64-baylibre 2026-03-12 09:36:55,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe62eae557f2600f0ff0d 2026-03-12 09:36:55,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:36:55,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe62eae557f2600f0ff0d HTTP/1.1" 200 None 2026-03-12 09:36:55,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:36:55,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-12 09:36:55,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:36:55,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe62eae557f2600f0ff0d HTTP/1.1" 200 None 2026-03-12 09:36:55,972 - lava_callback - ERROR - Error processing callback for node 69afe62eae557f2600f0ff0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48906 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 09:38:15,883 - lava_callback - INFO - Received callback for node 69afe67eae557f2600f0ff20 INFO: 172.20.0.1:48918 - "POST /node/69afe67eae557f2600f0ff20 HTTP/1.0" 202 Accepted 2026-03-12 09:38:15,889 - lava_callback - INFO - Starting async job submit for node 69afe67eae557f2600f0ff20 2026-03-12 09:38:15,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:38:15,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe67eae557f2600f0ff20 HTTP/1.1" 200 None 2026-03-12 09:38:15,905 - lava_callback - DEBUG - Processing node 69afe67eae557f2600f0ff20: baseline-arm64-baylibre 2026-03-12 09:38:15,906 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe67eae557f2600f0ff20 2026-03-12 09:38:15,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:38:15,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe67eae557f2600f0ff20 HTTP/1.1" 200 None 2026-03-12 09:38:15,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:38:15,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-12 09:38:15,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:38:15,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe67eae557f2600f0ff20 HTTP/1.1" 200 None 2026-03-12 09:38:15,961 - lava_callback - ERROR - Error processing callback for node 69afe67eae557f2600f0ff20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41870 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:39:35,969 - lava_callback - INFO - Received callback for node 69afe6cfae557f2600f0ff4a INFO: 172.20.0.1:53582 - "POST /node/69afe6cfae557f2600f0ff4a HTTP/1.0" 202 Accepted 2026-03-12 09:39:35,974 - lava_callback - INFO - Starting async job submit for node 69afe6cfae557f2600f0ff4a 2026-03-12 09:39:35,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:39:35,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe6cfae557f2600f0ff4a HTTP/1.1" 200 None 2026-03-12 09:39:35,995 - lava_callback - DEBUG - Processing node 69afe6cfae557f2600f0ff4a: baseline-arm64-baylibre 2026-03-12 09:39:35,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe6cfae557f2600f0ff4a 2026-03-12 09:39:35,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:39:36,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe6cfae557f2600f0ff4a HTTP/1.1" 200 None 2026-03-12 09:39:36,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:39:36,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-12 09:39:36,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:39:36,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe6cfae557f2600f0ff4a HTTP/1.1" 200 None 2026-03-12 09:39:36,052 - lava_callback - ERROR - Error processing callback for node 69afe6cfae557f2600f0ff4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53616 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:40:56,006 - lava_callback - INFO - Received callback for node 69afe721ae557f2600f0ff5b INFO: 172.20.0.1:53626 - "POST /node/69afe721ae557f2600f0ff5b HTTP/1.0" 202 Accepted 2026-03-12 09:40:56,012 - lava_callback - INFO - Starting async job submit for node 69afe721ae557f2600f0ff5b 2026-03-12 09:40:56,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:40:56,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe721ae557f2600f0ff5b HTTP/1.1" 200 None 2026-03-12 09:40:56,028 - lava_callback - DEBUG - Processing node 69afe721ae557f2600f0ff5b: baseline-arm64-baylibre 2026-03-12 09:40:56,028 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe721ae557f2600f0ff5b 2026-03-12 09:40:56,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:40:56,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe721ae557f2600f0ff5b HTTP/1.1" 200 None 2026-03-12 09:40:56,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:40:56,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-12 09:40:56,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:40:56,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe721ae557f2600f0ff5b HTTP/1.1" 200 None 2026-03-12 09:40:56,091 - lava_callback - ERROR - Error processing callback for node 69afe721ae557f2600f0ff5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35836 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 09:42:16,119 - lava_callback - INFO - Received callback for node 69afe76cae557f2600f0ff69 INFO: 172.20.0.1:35852 - "POST /node/69afe76cae557f2600f0ff69 HTTP/1.0" 202 Accepted 2026-03-12 09:42:16,124 - lava_callback - INFO - Starting async job submit for node 69afe76cae557f2600f0ff69 2026-03-12 09:42:16,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:42:16,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe76cae557f2600f0ff69 HTTP/1.1" 200 None 2026-03-12 09:42:16,140 - lava_callback - DEBUG - Processing node 69afe76cae557f2600f0ff69: baseline-arm64-baylibre 2026-03-12 09:42:16,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe76cae557f2600f0ff69 2026-03-12 09:42:16,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:42:16,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe76cae557f2600f0ff69 HTTP/1.1" 200 None 2026-03-12 09:42:16,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:42:16,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7ea227c78d2166ff857f2 HTTP/1.1" 200 None 2026-03-12 09:42:16,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:42:16,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe76cae557f2600f0ff69 HTTP/1.1" 200 None 2026-03-12 09:42:16,205 - lava_callback - ERROR - Error processing callback for node 69afe76cae557f2600f0ff69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:43:45,208 - lava_callback - INFO - Received callback for node 69afe7beae557f2600f0ff7e INFO: 172.20.0.1:47326 - "POST /node/69afe7beae557f2600f0ff7e HTTP/1.0" 202 Accepted 2026-03-12 09:43:45,214 - lava_callback - INFO - Starting async job submit for node 69afe7beae557f2600f0ff7e 2026-03-12 09:43:45,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:43:45,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe7beae557f2600f0ff7e HTTP/1.1" 200 None 2026-03-12 09:43:45,236 - lava_callback - DEBUG - Processing node 69afe7beae557f2600f0ff7e: baseline-arm64-clabbe 2026-03-12 09:43:45,236 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe7beae557f2600f0ff7e 2026-03-12 09:43:45,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:43:45,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe7beae557f2600f0ff7e HTTP/1.1" 200 None 2026-03-12 09:43:45,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:43:45,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-12 09:43:45,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:43:45,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe7beae557f2600f0ff7e HTTP/1.1" 200 None 2026-03-12 09:43:45,297 - lava_callback - ERROR - Error processing callback for node 69afe7beae557f2600f0ff7e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36308 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:45:05,243 - lava_callback - INFO - Received callback for node 69afe80cae557f2600f0ff8c INFO: 172.20.0.1:34212 - "POST /node/69afe80cae557f2600f0ff8c HTTP/1.0" 202 Accepted 2026-03-12 09:45:05,249 - lava_callback - INFO - Starting async job submit for node 69afe80cae557f2600f0ff8c 2026-03-12 09:45:05,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:05,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe80cae557f2600f0ff8c HTTP/1.1" 200 None 2026-03-12 09:45:05,264 - lava_callback - DEBUG - Processing node 69afe80cae557f2600f0ff8c: baseline-arm64-clabbe 2026-03-12 09:45:05,265 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe80cae557f2600f0ff8c 2026-03-12 09:45:05,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:05,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe80cae557f2600f0ff8c HTTP/1.1" 200 None 2026-03-12 09:45:05,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:05,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30c9ed37758c525e15e4 HTTP/1.1" 200 None 2026-03-12 09:45:05,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:05,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe80cae557f2600f0ff8c HTTP/1.1" 200 None 2026-03-12 09:45:05,316 - lava_callback - ERROR - Error processing callback for node 69afe80cae557f2600f0ff8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46634 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:45:44,917 - lava_callback - INFO - Received callback for node 69b28b234bce73bcf0234a54 INFO: 172.20.0.1:47934 - "POST /node/69b28b234bce73bcf0234a54 HTTP/1.0" 202 Accepted 2026-03-12 09:45:44,923 - lava_callback - INFO - Starting async job submit for node 69b28b234bce73bcf0234a54 2026-03-12 09:45:44,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:44,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28b234bce73bcf0234a54 HTTP/1.1" 200 None 2026-03-12 09:45:44,956 - lava_callback - DEBUG - Processing node 69b28b234bce73bcf0234a54: baseline-arm64 2026-03-12 09:45:45,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:45:45,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:45:45,268 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28b234bce73bcf0234a54/log.txt.gz 2026-03-12 09:45:45,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:45:45,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:45:45,616 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28b234bce73bcf0234a54/lava-logs.yaml 2026-03-12 09:45:45,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:45:45,775 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:45:45,777 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28b234bce73bcf0234a54/lava_callback.json.gz 2026-03-12 09:45:45,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:45,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28b234bce73bcf0234a54 HTTP/1.1" 200 None 2026-03-12 09:45:45,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:45,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:45:45,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:45,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b28b234bce73bcf0234a54 HTTP/1.1" 200 None 2026-03-12 09:45:45,863 - lava_callback - INFO - Completed processing callback for node 69b28b234bce73bcf0234a54 INFO: 172.20.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60174 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:45:54,741 - lava_callback - INFO - Received callback for node 69b28b1f4bce73bcf0234a4f INFO: 172.20.0.1:60182 - "POST /node/69b28b1f4bce73bcf0234a4f HTTP/1.0" 202 Accepted 2026-03-12 09:45:54,748 - lava_callback - INFO - Starting async job submit for node 69b28b1f4bce73bcf0234a4f 2026-03-12 09:45:54,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:54,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28b1f4bce73bcf0234a4f HTTP/1.1" 200 None 2026-03-12 09:45:54,780 - lava_callback - DEBUG - Processing node 69b28b1f4bce73bcf0234a4f: baseline-arm64 2026-03-12 09:45:54,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:45:55,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:45:55,055 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28b1f4bce73bcf0234a4f/log.txt.gz 2026-03-12 09:45:55,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:45:55,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:45:55,323 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28b1f4bce73bcf0234a4f/lava-logs.yaml 2026-03-12 09:45:55,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:45:55,450 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:45:55,452 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28b1f4bce73bcf0234a4f/lava_callback.json.gz 2026-03-12 09:45:55,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:55,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28b1f4bce73bcf0234a4f HTTP/1.1" 200 None 2026-03-12 09:45:55,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:55,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:45:55,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:45:55,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b28b1f4bce73bcf0234a4f HTTP/1.1" 200 None 2026-03-12 09:45:55,567 - lava_callback - INFO - Completed processing callback for node 69b28b1f4bce73bcf0234a4f INFO: 172.20.0.1:60196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:46:13,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:46:13,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34638 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:48:06,211 - lava_callback - INFO - Received callback for node 69b28bbd4bce73bcf0234a79 INFO: 172.20.0.1:36146 - "POST /node/69b28bbd4bce73bcf0234a79 HTTP/1.0" 202 Accepted 2026-03-12 09:48:06,216 - lava_callback - INFO - Starting async job submit for node 69b28bbd4bce73bcf0234a79 2026-03-12 09:48:06,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:48:06,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28bbd4bce73bcf0234a79 HTTP/1.1" 200 None 2026-03-12 09:48:06,233 - lava_callback - DEBUG - Processing node 69b28bbd4bce73bcf0234a79: baseline-nfs-arm64 2026-03-12 09:48:06,234 - lava_callback - WARNING - Failed to upload log 2026-03-12 09:48:06,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:48:06,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:48:06,381 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b28bbd4bce73bcf0234a79/lava-logs.yaml 2026-03-12 09:48:06,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:36154 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:48:06,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:48:06,522 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b28bbd4bce73bcf0234a79/lava_callback.json.gz 2026-03-12 09:48:06,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:48:06,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28bbd4bce73bcf0234a79 HTTP/1.1" 200 None 2026-03-12 09:48:06,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:48:06,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:48:06,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:48:06,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b28bbd4bce73bcf0234a79 HTTP/1.1" 200 None 2026-03-12 09:48:06,584 - lava_callback - INFO - Completed processing callback for node 69b28bbd4bce73bcf0234a79 2026-03-12 09:48:13,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:48:13,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39930 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:48:29,502 - lava_callback - INFO - Received callback for node 69b2841c4bce73bcf02346b8 INFO: 172.20.0.1:39934 - "POST /node/69b2841c4bce73bcf02346b8 HTTP/1.0" 202 Accepted 2026-03-12 09:48:29,507 - lava_callback - INFO - Starting async job submit for node 69b2841c4bce73bcf02346b8 2026-03-12 09:48:29,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:48:29,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2841c4bce73bcf02346b8 HTTP/1.1" 200 None 2026-03-12 09:48:29,544 - lava_callback - DEBUG - Processing node 69b2841c4bce73bcf02346b8: baseline-arm64-broonie 2026-03-12 09:48:29,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:48:29,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:48:29,764 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841c4bce73bcf02346b8/log.txt.gz 2026-03-12 09:48:29,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:48:29,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:48:29,950 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841c4bce73bcf02346b8/lava-logs.yaml 2026-03-12 09:48:29,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:48:30,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:48:30,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2841c4bce73bcf02346b8/lava_callback.json.gz 2026-03-12 09:48:30,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:48:30,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2841c4bce73bcf02346b8 HTTP/1.1" 200 None 2026-03-12 09:48:30,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:48:30,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:48:30,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:48:30,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2841c4bce73bcf02346b8 HTTP/1.1" 200 None 2026-03-12 09:48:30,193 - lava_callback - INFO - Completed processing callback for node 69b2841c4bce73bcf02346b8 INFO: 172.20.0.1:44542 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:48:43,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:48:43,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:49:08,995 - lava_callback - INFO - Received callback for node 69b287fd4bce73bcf0234957 INFO: 172.20.0.1:58086 - "POST /node/69b287fd4bce73bcf0234957 HTTP/1.0" 202 Accepted 2026-03-12 09:49:09,001 - lava_callback - INFO - Starting async job submit for node 69b287fd4bce73bcf0234957 2026-03-12 09:49:09,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:49:09,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b287fd4bce73bcf0234957 HTTP/1.1" 200 None 2026-03-12 09:49:09,077 - lava_callback - DEBUG - Processing node 69b287fd4bce73bcf0234957: baseline-arm 2026-03-12 09:49:09,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:49:09,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:49:09,830 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b287fd4bce73bcf0234957/log.txt.gz 2026-03-12 09:49:10,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:49:10,399 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:49:10,401 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b287fd4bce73bcf0234957/lava-logs.yaml 2026-03-12 09:49:10,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:49:10,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:49:10,559 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b287fd4bce73bcf0234957/lava_callback.json.gz DEBUG: Setup failed for 69b287fd4bce73bcf0234957. Transitting job node result: incomplete -> fail 2026-03-12 09:49:10,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:49:10,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b287fd4bce73bcf0234957 HTTP/1.1" 200 None 2026-03-12 09:49:10,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:49:10,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 09:49:10,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:49:10,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b287fd4bce73bcf0234957 HTTP/1.1" 200 None 2026-03-12 09:49:10,836 - lava_callback - INFO - Completed processing callback for node 69b287fd4bce73bcf0234957 2026-03-12 09:49:13,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:49:13,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33990 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:50:23,963 - lava_callback - INFO - Received callback for node 69b28c254bce73bcf0234a9b INFO: 172.20.0.1:45088 - "POST /node/69b28c254bce73bcf0234a9b HTTP/1.0" 202 Accepted 2026-03-12 09:50:23,970 - lava_callback - INFO - Starting async job submit for node 69b28c254bce73bcf0234a9b 2026-03-12 09:50:23,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:50:24,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28c254bce73bcf0234a9b HTTP/1.1" 200 None 2026-03-12 09:50:24,015 - lava_callback - DEBUG - Processing node 69b28c254bce73bcf0234a9b: baseline-arm64 2026-03-12 09:50:24,150 - lava_callback - WARNING - Failed to upload log 2026-03-12 09:50:24,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:50:24,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:50:24,421 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28c254bce73bcf0234a9b/lava-logs.yaml 2026-03-12 09:50:24,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:50:24,589 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:50:24,590 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28c254bce73bcf0234a9b/lava_callback.json.gz 2026-03-12 09:50:24,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:50:24,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28c254bce73bcf0234a9b HTTP/1.1" 200 None 2026-03-12 09:50:24,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:50:24,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:50:24,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:50:24,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b28c254bce73bcf0234a9b HTTP/1.1" 200 None 2026-03-12 09:50:24,778 - lava_callback - INFO - Completed processing callback for node 69b28c254bce73bcf0234a9b INFO: 172.20.0.1:45104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33286 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:50:36,573 - lava_callback - INFO - Received callback for node 69afe964ae557f2600f0fff4 INFO: 172.20.0.1:33300 - "POST /node/69afe964ae557f2600f0fff4 HTTP/1.0" 202 Accepted 2026-03-12 09:50:36,578 - lava_callback - INFO - Starting async job submit for node 69afe964ae557f2600f0fff4 2026-03-12 09:50:36,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:50:36,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe964ae557f2600f0fff4 HTTP/1.1" 200 None 2026-03-12 09:50:36,597 - lava_callback - DEBUG - Processing node 69afe964ae557f2600f0fff4: baseline-arm64-baylibre 2026-03-12 09:50:36,597 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe964ae557f2600f0fff4 2026-03-12 09:50:36,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:50:36,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe964ae557f2600f0fff4 HTTP/1.1" 200 None 2026-03-12 09:50:36,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:50:36,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7ea227c78d2166ff857f2 HTTP/1.1" 200 None 2026-03-12 09:50:36,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:50:36,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe964ae557f2600f0fff4 HTTP/1.1" 200 None 2026-03-12 09:50:36,677 - lava_callback - ERROR - Error processing callback for node 69afe964ae557f2600f0fff4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 09:50:43,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:50:43,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38406 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:51:33,815 - lava_callback - INFO - Received callback for node 69b28c704bce73bcf0234aa7 INFO: 172.20.0.1:38932 - "POST /node/69b28c704bce73bcf0234aa7 HTTP/1.0" 202 Accepted 2026-03-12 09:51:33,821 - lava_callback - INFO - Starting async job submit for node 69b28c704bce73bcf0234aa7 2026-03-12 09:51:33,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:51:33,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28c704bce73bcf0234aa7 HTTP/1.1" 200 None 2026-03-12 09:51:33,866 - lava_callback - DEBUG - Processing node 69b28c704bce73bcf0234aa7: baseline-arm64 2026-03-12 09:51:33,993 - lava_callback - WARNING - Failed to upload log 2026-03-12 09:51:34,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:51:34,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:51:34,267 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28c704bce73bcf0234aa7/lava-logs.yaml 2026-03-12 09:51:34,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:51:34,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:51:34,413 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28c704bce73bcf0234aa7/lava_callback.json.gz 2026-03-12 09:51:34,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:51:34,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28c704bce73bcf0234aa7 HTTP/1.1" 200 None 2026-03-12 09:51:34,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:51:34,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:51:34,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:51:34,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b28c704bce73bcf0234aa7 HTTP/1.1" 200 None 2026-03-12 09:51:34,590 - lava_callback - INFO - Completed processing callback for node 69b28c704bce73bcf0234aa7 INFO: 172.20.0.1:38942 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:51:43,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:51:43,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40798 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:52:36,726 - lava_callback - INFO - Received callback for node 69afe9d7ae557f2600f1000c INFO: 172.20.0.1:40800 - "POST /node/69afe9d7ae557f2600f1000c HTTP/1.0" 202 Accepted 2026-03-12 09:52:36,732 - lava_callback - INFO - Starting async job submit for node 69afe9d7ae557f2600f1000c 2026-03-12 09:52:36,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:52:36,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe9d7ae557f2600f1000c HTTP/1.1" 200 None 2026-03-12 09:52:36,749 - lava_callback - DEBUG - Processing node 69afe9d7ae557f2600f1000c: baseline-arm64-baylibre 2026-03-12 09:52:36,750 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afe9d7ae557f2600f1000c 2026-03-12 09:52:36,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:52:36,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afe9d7ae557f2600f1000c HTTP/1.1" 200 None 2026-03-12 09:52:36,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:52:36,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69a7ea227c78d2166ff857f2 HTTP/1.1" 200 None 2026-03-12 09:52:36,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:52:36,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afe9d7ae557f2600f1000c HTTP/1.1" 200 None 2026-03-12 09:52:36,830 - lava_callback - ERROR - Error processing callback for node 69afe9d7ae557f2600f1000c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48664 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:53:46,424 - lava_callback - INFO - Received callback for node 69b2889b4bce73bcf0234973 INFO: 172.20.0.1:33696 - "POST /node/69b2889b4bce73bcf0234973 HTTP/1.0" 202 Accepted 2026-03-12 09:53:46,473 - lava_callback - INFO - Starting async job submit for node 69b2889b4bce73bcf0234973 2026-03-12 09:53:46,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:33712 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:53:46,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2889b4bce73bcf0234973 HTTP/1.1" 200 None 2026-03-12 09:53:46,513 - lava_callback - DEBUG - Processing node 69b2889b4bce73bcf0234973: baseline-arm-pengutronix 2026-03-12 09:53:46,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:53:46,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:53:46,773 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b2889b4bce73bcf0234973/log.txt.gz 2026-03-12 09:53:46,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:53:47,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:53:47,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b2889b4bce73bcf0234973/lava-logs.yaml 2026-03-12 09:53:47,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:53:47,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:53:47,177 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b2889b4bce73bcf0234973/lava_callback.json.gz DEBUG: Setup failed for 69b2889b4bce73bcf0234973. Transitting job node result: incomplete -> fail 2026-03-12 09:53:47,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:53:47,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2889b4bce73bcf0234973 HTTP/1.1" 200 None 2026-03-12 09:53:47,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:53:47,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 09:53:47,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:53:47,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2889b4bce73bcf0234973 HTTP/1.1" 200 None 2026-03-12 09:53:47,355 - lava_callback - INFO - Completed processing callback for node 69b2889b4bce73bcf0234973 INFO: 172.20.0.1:53662 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:53:55,604 - lava_callback - INFO - Received callback for node 69b28c1f4bce73bcf0234a98 INFO: 172.20.0.1:53670 - "POST /node/69b28c1f4bce73bcf0234a98 HTTP/1.0" 202 Accepted 2026-03-12 09:53:55,610 - lava_callback - INFO - Starting async job submit for node 69b28c1f4bce73bcf0234a98 2026-03-12 09:53:55,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:53:55,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28c1f4bce73bcf0234a98 HTTP/1.1" 200 None 2026-03-12 09:53:55,650 - lava_callback - DEBUG - Processing node 69b28c1f4bce73bcf0234a98: ltp-sched 2026-03-12 09:53:56,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:53:56,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:53:56,205 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69b28c1f4bce73bcf0234a98/log.txt.gz 2026-03-12 09:53:56,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:53672 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:53:56,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:53:56,669 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69b28c1f4bce73bcf0234a98/lava-logs.yaml 2026-03-12 09:53:56,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:53:56,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:53:56,840 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69b28c1f4bce73bcf0234a98/lava_callback.json.gz 2026-03-12 09:53:56,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:53:56,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28c1f4bce73bcf0234a98 HTTP/1.1" 200 None 2026-03-12 09:53:56,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:53:56,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 09:53:56,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:53:56,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b28c1f4bce73bcf0234a98 HTTP/1.1" 200 None 2026-03-12 09:53:56,921 - lava_callback - INFO - Completed processing callback for node 69b28c1f4bce73bcf0234a98 2026-03-12 09:54:03,021 - lava_callback - INFO - Received callback for node 69b28cbc4bce73bcf0234ab2 INFO: 172.20.0.1:56084 - "POST /node/69b28cbc4bce73bcf0234ab2 HTTP/1.0" 202 Accepted 2026-03-12 09:54:03,028 - lava_callback - INFO - Starting async job submit for node 69b28cbc4bce73bcf0234ab2 2026-03-12 09:54:03,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:54:03,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28cbc4bce73bcf0234ab2 HTTP/1.1" 200 None 2026-03-12 09:54:03,066 - lava_callback - DEBUG - Processing node 69b28cbc4bce73bcf0234ab2: baseline-arm64 2026-03-12 09:54:03,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:54:03,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:54:03,617 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28cbc4bce73bcf0234ab2/log.txt.gz 2026-03-12 09:54:03,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:54:04,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:54:04,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28cbc4bce73bcf0234ab2/lava-logs.yaml 2026-03-12 09:54:04,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 09:54:04,210 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 09:54:04,212 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b28cbc4bce73bcf0234ab2/lava_callback.json.gz 2026-03-12 09:54:04,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:54:04,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28cbc4bce73bcf0234ab2 HTTP/1.1" 200 None 2026-03-12 09:54:04,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:54:04,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 09:54:04,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:54:04,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b28cbc4bce73bcf0234ab2 HTTP/1.1" 200 None 2026-03-12 09:54:04,408 - lava_callback - INFO - Completed processing callback for node 69b28cbc4bce73bcf0234ab2 INFO: 172.20.0.1:56086 - "GET / HTTP/1.0" 200 OK 2026-03-12 09:54:13,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 09:54:13,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59754 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:03:25,035 - lava_callback - INFO - Received callback for node 69b28f554bce73bcf0234b1b INFO: 172.20.0.1:56278 - "POST /node/69b28f554bce73bcf0234b1b HTTP/1.0" 202 Accepted 2026-03-12 10:03:25,041 - lava_callback - INFO - Starting async job submit for node 69b28f554bce73bcf0234b1b 2026-03-12 10:03:25,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:03:25,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28f554bce73bcf0234b1b HTTP/1.1" 200 None 2026-03-12 10:03:25,063 - lava_callback - DEBUG - Processing node 69b28f554bce73bcf0234b1b: baseline-arm 2026-03-12 10:03:25,068 - lava_callback - WARNING - Failed to upload log 2026-03-12 10:03:25,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:03:25,231 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:03:25,233 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b28f554bce73bcf0234b1b/lava-logs.yaml 2026-03-12 10:03:25,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:03:25,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:03:25,383 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b28f554bce73bcf0234b1b/lava_callback.json.gz 2026-03-12 10:03:25,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:03:25,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b28f554bce73bcf0234b1b HTTP/1.1" 200 None 2026-03-12 10:03:25,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:03:25,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 10:03:25,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:03:25,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b28f554bce73bcf0234b1b HTTP/1.1" 200 None 2026-03-12 10:03:25,461 - lava_callback - INFO - Completed processing callback for node 69b28f554bce73bcf0234b1b INFO: 172.20.0.1:56284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55502 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:03:43,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:03:43,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59242 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:03:57,321 - lava_callback - INFO - Received callback for node 69afec78ae557f2600f100bd INFO: 172.20.0.1:59246 - "POST /node/69afec78ae557f2600f100bd HTTP/1.0" 202 Accepted 2026-03-12 10:03:57,327 - lava_callback - INFO - Starting async job submit for node 69afec78ae557f2600f100bd 2026-03-12 10:03:57,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:03:57,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afec78ae557f2600f100bd HTTP/1.1" 200 None 2026-03-12 10:03:57,351 - lava_callback - DEBUG - Processing node 69afec78ae557f2600f100bd: baseline-arm-baylibre 2026-03-12 10:03:57,351 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afec78ae557f2600f100bd 2026-03-12 10:03:57,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:03:57,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afec78ae557f2600f100bd HTTP/1.1" 200 None 2026-03-12 10:03:57,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:03:57,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba33feb96b28aa34a775 HTTP/1.1" 200 None 2026-03-12 10:03:57,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:03:57,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afec78ae557f2600f100bd HTTP/1.1" 200 None 2026-03-12 10:03:57,429 - lava_callback - ERROR - Error processing callback for node 69afec78ae557f2600f100bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60154 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:04:17,319 - lava_callback - INFO - Received callback for node 69afec9dae557f2600f100c7 INFO: 172.20.0.1:60162 - "POST /node/69afec9dae557f2600f100c7 HTTP/1.0" 202 Accepted 2026-03-12 10:04:17,325 - lava_callback - INFO - Starting async job submit for node 69afec9dae557f2600f100c7 2026-03-12 10:04:17,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:04:17,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afec9dae557f2600f100c7 HTTP/1.1" 200 None 2026-03-12 10:04:17,341 - lava_callback - DEBUG - Processing node 69afec9dae557f2600f100c7: baseline-arm-baylibre 2026-03-12 10:04:17,341 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afec9dae557f2600f100c7 2026-03-12 10:04:17,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:04:17,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afec9dae557f2600f100c7 HTTP/1.1" 200 None 2026-03-12 10:04:17,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:04:17,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba33feb96b28aa34a775 HTTP/1.1" 200 None 2026-03-12 10:04:17,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:04:17,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afec9dae557f2600f100c7 HTTP/1.1" 200 None 2026-03-12 10:04:17,401 - lava_callback - ERROR - Error processing callback for node 69afec9dae557f2600f100c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55744 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:04:57,356 - lava_callback - INFO - Received callback for node 69afecc2ae557f2600f100d2 INFO: 172.20.0.1:55750 - "POST /node/69afecc2ae557f2600f100d2 HTTP/1.0" 202 Accepted 2026-03-12 10:04:57,362 - lava_callback - INFO - Starting async job submit for node 69afecc2ae557f2600f100d2 2026-03-12 10:04:57,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:04:57,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afecc2ae557f2600f100d2 HTTP/1.1" 200 None 2026-03-12 10:04:57,386 - lava_callback - DEBUG - Processing node 69afecc2ae557f2600f100d2: baseline-arm-baylibre 2026-03-12 10:04:57,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afecc2ae557f2600f100d2 2026-03-12 10:04:57,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:04:57,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afecc2ae557f2600f100d2 HTTP/1.1" 200 None 2026-03-12 10:04:57,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:04:57,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba33feb96b28aa34a775 HTTP/1.1" 200 None 2026-03-12 10:04:57,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:04:57,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afecc2ae557f2600f100d2 HTTP/1.1" 200 None 2026-03-12 10:04:57,478 - lava_callback - ERROR - Error processing callback for node 69afecc2ae557f2600f100d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48864 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:13:36,087 - lava_callback - INFO - Received callback for node 69b291564bce73bcf0234b73 INFO: 172.20.0.1:33026 - "POST /node/69b291564bce73bcf0234b73 HTTP/1.0" 202 Accepted 2026-03-12 10:13:36,093 - lava_callback - INFO - Starting async job submit for node 69b291564bce73bcf0234b73 2026-03-12 10:13:36,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:13:36,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b291564bce73bcf0234b73 HTTP/1.1" 200 None 2026-03-12 10:13:36,132 - lava_callback - DEBUG - Processing node 69b291564bce73bcf0234b73: baseline-arm64-broonie 2026-03-12 10:13:36,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33040 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:13:36,559 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:13:36,560 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b291564bce73bcf0234b73/log.txt.gz 2026-03-12 10:13:36,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:13:36,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:13:36,949 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b291564bce73bcf0234b73/lava-logs.yaml 2026-03-12 10:13:36,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:13:37,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:13:37,133 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b291564bce73bcf0234b73/lava_callback.json.gz 2026-03-12 10:13:37,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:13:37,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b291564bce73bcf0234b73 HTTP/1.1" 200 None 2026-03-12 10:13:37,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:13:37,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:13:37,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:13:37,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b291564bce73bcf0234b73 HTTP/1.1" 200 None 2026-03-12 10:13:37,219 - lava_callback - INFO - Completed processing callback for node 69b291564bce73bcf0234b73 2026-03-12 10:13:37,892 - lava_callback - INFO - Received callback for node 69afeeceae557f2600f10152 INFO: 172.20.0.1:33050 - "POST /node/69afeeceae557f2600f10152 HTTP/1.0" 202 Accepted 2026-03-12 10:13:37,898 - lava_callback - INFO - Starting async job submit for node 69afeeceae557f2600f10152 2026-03-12 10:13:37,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:13:37,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afeeceae557f2600f10152 HTTP/1.1" 200 None 2026-03-12 10:13:37,917 - lava_callback - DEBUG - Processing node 69afeeceae557f2600f10152: baseline-arm-baylibre 2026-03-12 10:13:37,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afeeceae557f2600f10152 2026-03-12 10:13:37,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:13:37,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afeeceae557f2600f10152 HTTP/1.1" 200 None 2026-03-12 10:13:37,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:13:37,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba33feb96b28aa34a775 HTTP/1.1" 200 None 2026-03-12 10:13:37,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:13:37,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afeeceae557f2600f10152 HTTP/1.1" 200 None 2026-03-12 10:13:37,989 - lava_callback - ERROR - Error processing callback for node 69afeeceae557f2600f10152: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 10:13:43,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:13:43,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41490 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:15:38,009 - lava_callback - INFO - Received callback for node 69afef3eae557f2600f10165 INFO: 172.20.0.1:41494 - "POST /node/69afef3eae557f2600f10165 HTTP/1.0" 202 Accepted 2026-03-12 10:15:38,015 - lava_callback - INFO - Starting async job submit for node 69afef3eae557f2600f10165 2026-03-12 10:15:38,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:15:38,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afef3eae557f2600f10165 HTTP/1.1" 200 None 2026-03-12 10:15:38,038 - lava_callback - DEBUG - Processing node 69afef3eae557f2600f10165: baseline-arm-baylibre 2026-03-12 10:15:38,038 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afef3eae557f2600f10165 2026-03-12 10:15:38,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:15:38,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afef3eae557f2600f10165 HTTP/1.1" 200 None 2026-03-12 10:15:38,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:15:38,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba33feb96b28aa34a775 HTTP/1.1" 200 None 2026-03-12 10:15:38,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:15:38,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afef3eae557f2600f10165 HTTP/1.1" 200 None 2026-03-12 10:15:38,117 - lava_callback - ERROR - Error processing callback for node 69afef3eae557f2600f10165: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 10:15:43,535 - lava_callback - INFO - Received callback for node 69b2910f4bce73bcf0234b6b INFO: 172.20.0.1:54532 - "POST /node/69b2910f4bce73bcf0234b6b HTTP/1.0" 202 Accepted 2026-03-12 10:15:43,544 - lava_callback - INFO - Starting async job submit for node 69b2910f4bce73bcf0234b6b 2026-03-12 10:15:43,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:15:43,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2910f4bce73bcf0234b6b HTTP/1.1" 200 None 2026-03-12 10:15:43,606 - lava_callback - DEBUG - Processing node 69b2910f4bce73bcf0234b6b: baseline-arm64-foundriesio 2026-03-12 10:15:44,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:15:45,032 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:15:45,033 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b2910f4bce73bcf0234b6b/log.txt.gz 2026-03-12 10:15:45,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:15:46,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:15:46,048 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b2910f4bce73bcf0234b6b/lava-logs.yaml 2026-03-12 10:15:46,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:15:46,304 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:15:46,306 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b2910f4bce73bcf0234b6b/lava_callback.json.gz DEBUG: 69b2910f4bce73bcf0234b6b Transitting job node result: pass -> incomplete 2026-03-12 10:15:46,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:15:46,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2910f4bce73bcf0234b6b HTTP/1.1" 200 None 2026-03-12 10:15:46,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:15:46,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:15:46,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:15:46,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2910f4bce73bcf0234b6b HTTP/1.1" 200 None 2026-03-12 10:15:46,510 - lava_callback - INFO - Completed processing callback for node 69b2910f4bce73bcf0234b6b INFO: 172.20.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46248 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:16:13,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:16:13,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49516 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:18:46,750 - lava_callback - INFO - Received callback for node 69afeff8ae557f2600f10193 INFO: 172.20.0.1:49530 - "POST /node/69afeff8ae557f2600f10193 HTTP/1.0" 202 Accepted 2026-03-12 10:18:46,756 - lava_callback - INFO - Starting async job submit for node 69afeff8ae557f2600f10193 2026-03-12 10:18:46,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:18:46,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afeff8ae557f2600f10193 HTTP/1.1" 200 None 2026-03-12 10:18:46,771 - lava_callback - DEBUG - Processing node 69afeff8ae557f2600f10193: baseline-arm-clabbe 2026-03-12 10:18:46,772 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69afeff8ae557f2600f10193 2026-03-12 10:18:46,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:18:46,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afeff8ae557f2600f10193 HTTP/1.1" 200 None 2026-03-12 10:18:46,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:18:46,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba33feb96b28aa34a775 HTTP/1.1" 200 None 2026-03-12 10:18:46,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:18:46,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69afeff8ae557f2600f10193 HTTP/1.1" 200 None 2026-03-12 10:18:46,835 - lava_callback - ERROR - Error processing callback for node 69afeff8ae557f2600f10193: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42448 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:19:26,699 - lava_callback - INFO - Received callback for node 69aff01dae557f2600f1019c INFO: 172.20.0.1:42452 - "POST /node/69aff01dae557f2600f1019c HTTP/1.0" 202 Accepted 2026-03-12 10:19:26,704 - lava_callback - INFO - Starting async job submit for node 69aff01dae557f2600f1019c 2026-03-12 10:19:26,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:19:26,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff01dae557f2600f1019c HTTP/1.1" 200 None 2026-03-12 10:19:26,721 - lava_callback - DEBUG - Processing node 69aff01dae557f2600f1019c: baseline-arm-clabbe 2026-03-12 10:19:26,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff01dae557f2600f1019c 2026-03-12 10:19:26,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:19:26,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff01dae557f2600f1019c HTTP/1.1" 200 None 2026-03-12 10:19:26,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:19:26,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba33feb96b28aa34a775 HTTP/1.1" 200 None 2026-03-12 10:19:26,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:19:26,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff01dae557f2600f1019c HTTP/1.1" 200 None 2026-03-12 10:19:26,776 - lava_callback - ERROR - Error processing callback for node 69aff01dae557f2600f1019c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34144 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:20:06,804 - lava_callback - INFO - Received callback for node 69aff042ae557f2600f101a5 INFO: 172.20.0.1:34158 - "POST /node/69aff042ae557f2600f101a5 HTTP/1.0" 202 Accepted 2026-03-12 10:20:06,810 - lava_callback - INFO - Starting async job submit for node 69aff042ae557f2600f101a5 2026-03-12 10:20:06,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:06,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff042ae557f2600f101a5 HTTP/1.1" 200 None 2026-03-12 10:20:06,837 - lava_callback - DEBUG - Processing node 69aff042ae557f2600f101a5: baseline-arm-clabbe 2026-03-12 10:20:06,837 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff042ae557f2600f101a5 2026-03-12 10:20:06,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:06,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff042ae557f2600f101a5 HTTP/1.1" 200 None 2026-03-12 10:20:06,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:06,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba33feb96b28aa34a775 HTTP/1.1" 200 None 2026-03-12 10:20:06,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:06,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff042ae557f2600f101a5 HTTP/1.1" 200 None 2026-03-12 10:20:06,916 - lava_callback - ERROR - Error processing callback for node 69aff042ae557f2600f101a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59306 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:20:26,780 - lava_callback - INFO - Received callback for node 69aff068ae557f2600f101b3 INFO: 172.20.0.1:59314 - "POST /node/69aff068ae557f2600f101b3 HTTP/1.0" 202 Accepted 2026-03-12 10:20:26,785 - lava_callback - INFO - Starting async job submit for node 69aff068ae557f2600f101b3 2026-03-12 10:20:26,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:26,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff068ae557f2600f101b3 HTTP/1.1" 200 None 2026-03-12 10:20:26,803 - lava_callback - DEBUG - Processing node 69aff068ae557f2600f101b3: baseline-arm-clabbe 2026-03-12 10:20:26,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff068ae557f2600f101b3 2026-03-12 10:20:26,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:26,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff068ae557f2600f101b3 HTTP/1.1" 200 None 2026-03-12 10:20:26,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:26,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba33feb96b28aa34a775 HTTP/1.1" 200 None 2026-03-12 10:20:26,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:26,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff068ae557f2600f101b3 HTTP/1.1" 200 None 2026-03-12 10:20:26,862 - lava_callback - ERROR - Error processing callback for node 69aff068ae557f2600f101b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54964 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:20:39,757 - lava_callback - INFO - Received callback for node 69b291964bce73bcf0234b7d INFO: 172.20.0.1:48640 - "POST /node/69b291964bce73bcf0234b7d HTTP/1.0" 202 Accepted 2026-03-12 10:20:39,764 - lava_callback - INFO - Starting async job submit for node 69b291964bce73bcf0234b7d 2026-03-12 10:20:39,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:39,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b291964bce73bcf0234b7d HTTP/1.1" 200 None 2026-03-12 10:20:39,815 - lava_callback - DEBUG - Processing node 69b291964bce73bcf0234b7d: baseline-arm64 2026-03-12 10:20:40,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:20:40,350 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:20:40,352 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b291964bce73bcf0234b7d/log.txt.gz 2026-03-12 10:20:40,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:20:40,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:20:40,817 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b291964bce73bcf0234b7d/lava-logs.yaml 2026-03-12 10:20:40,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:20:41,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:20:41,022 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b291964bce73bcf0234b7d/lava_callback.json.gz DEBUG: Setup failed for 69b291964bce73bcf0234b7d. Transitting job node result: incomplete -> fail 2026-03-12 10:20:41,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:41,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b291964bce73bcf0234b7d HTTP/1.1" 200 None 2026-03-12 10:20:41,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:41,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:20:41,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:41,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b291964bce73bcf0234b7d HTTP/1.1" 200 None 2026-03-12 10:20:41,252 - lava_callback - INFO - Completed processing callback for node 69b291964bce73bcf0234b7d 2026-03-12 10:20:43,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:20:43,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38246 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:21:46,868 - lava_callback - INFO - Received callback for node 69aff0b0ae557f2600f101c4 INFO: 172.20.0.1:38252 - "POST /node/69aff0b0ae557f2600f101c4 HTTP/1.0" 202 Accepted 2026-03-12 10:21:46,874 - lava_callback - INFO - Starting async job submit for node 69aff0b0ae557f2600f101c4 2026-03-12 10:21:46,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:21:46,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff0b0ae557f2600f101c4 HTTP/1.1" 200 None 2026-03-12 10:21:46,890 - lava_callback - DEBUG - Processing node 69aff0b0ae557f2600f101c4: baseline-arm-clabbe 2026-03-12 10:21:46,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff0b0ae557f2600f101c4 2026-03-12 10:21:46,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:21:46,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff0b0ae557f2600f101c4 HTTP/1.1" 200 None 2026-03-12 10:21:46,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:21:46,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba33feb96b28aa34a775 HTTP/1.1" 200 None 2026-03-12 10:21:46,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:21:46,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff0b0ae557f2600f101c4 HTTP/1.1" 200 None 2026-03-12 10:21:46,944 - lava_callback - ERROR - Error processing callback for node 69aff0b0ae557f2600f101c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35846 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:30:57,908 - lava_callback - INFO - Received callback for node 69b293ed4bce73bcf0234c0f INFO: 172.20.0.1:35860 - "POST /node/69b293ed4bce73bcf0234c0f HTTP/1.0" 202 Accepted 2026-03-12 10:30:57,918 - lava_callback - INFO - Starting async job submit for node 69b293ed4bce73bcf0234c0f 2026-03-12 10:30:57,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:30:57,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b293ed4bce73bcf0234c0f HTTP/1.1" 200 None 2026-03-12 10:30:57,964 - lava_callback - DEBUG - Processing node 69b293ed4bce73bcf0234c0f: baseline-arm64-clabbe 2026-03-12 10:30:58,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:30:58,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:30:58,534 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b293ed4bce73bcf0234c0f/log.txt.gz 2026-03-12 10:30:58,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:30:59,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:30:59,026 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b293ed4bce73bcf0234c0f/lava-logs.yaml 2026-03-12 10:30:59,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:30:59,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:30:59,231 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69b293ed4bce73bcf0234c0f/lava_callback.json.gz 2026-03-12 10:30:59,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:30:59,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b293ed4bce73bcf0234c0f HTTP/1.1" 200 None 2026-03-12 10:30:59,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:30:59,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:30:59,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:30:59,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b293ed4bce73bcf0234c0f HTTP/1.1" 200 None 2026-03-12 10:30:59,343 - lava_callback - INFO - Completed processing callback for node 69b293ed4bce73bcf0234c0f INFO: 172.20.0.1:48178 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:31:13,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:31:13,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33028 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:32:47,716 - lava_callback - INFO - Received callback for node 69b2963a4bce73bcf0234c7b INFO: 172.20.0.1:33034 - "POST /node/69b2963a4bce73bcf0234c7b HTTP/1.0" 202 Accepted 2026-03-12 10:32:47,722 - lava_callback - INFO - Starting async job submit for node 69b2963a4bce73bcf0234c7b 2026-03-12 10:32:47,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:32:47,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2963a4bce73bcf0234c7b HTTP/1.1" 200 None 2026-03-12 10:32:47,738 - lava_callback - DEBUG - Processing node 69b2963a4bce73bcf0234c7b: baseline-nfs-arm64 2026-03-12 10:32:47,740 - lava_callback - WARNING - Failed to upload log 2026-03-12 10:32:47,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:32:47,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:32:47,906 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2963a4bce73bcf0234c7b/lava-logs.yaml 2026-03-12 10:32:47,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:32:48,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:32:48,055 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2963a4bce73bcf0234c7b/lava_callback.json.gz 2026-03-12 10:32:48,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:32:48,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2963a4bce73bcf0234c7b HTTP/1.1" 200 None 2026-03-12 10:32:48,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:32:48,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:32:48,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:32:48,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2963a4bce73bcf0234c7b HTTP/1.1" 200 None 2026-03-12 10:32:48,112 - lava_callback - INFO - Completed processing callback for node 69b2963a4bce73bcf0234c7b INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42452 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:33:13,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:33:13,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52398 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:36:27,911 - lava_callback - INFO - Received callback for node 69b296ee4bce73bcf0234ca0 INFO: 172.20.0.1:52404 - "POST /node/69b296ee4bce73bcf0234ca0 HTTP/1.0" 202 Accepted 2026-03-12 10:36:27,918 - lava_callback - INFO - Starting async job submit for node 69b296ee4bce73bcf0234ca0 2026-03-12 10:36:27,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:36:27,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b296ee4bce73bcf0234ca0 HTTP/1.1" 200 None 2026-03-12 10:36:27,963 - lava_callback - DEBUG - Processing node 69b296ee4bce73bcf0234ca0: baseline-arm64 2026-03-12 10:36:28,161 - lava_callback - WARNING - Failed to upload log 2026-03-12 10:36:28,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:36:28,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:36:28,461 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b296ee4bce73bcf0234ca0/lava-logs.yaml 2026-03-12 10:36:28,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:36:28,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:36:28,611 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b296ee4bce73bcf0234ca0/lava_callback.json.gz 2026-03-12 10:36:28,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:36:28,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b296ee4bce73bcf0234ca0 HTTP/1.1" 200 None 2026-03-12 10:36:28,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:36:28,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:36:28,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:36:28,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b296ee4bce73bcf0234ca0 HTTP/1.1" 200 None 2026-03-12 10:36:28,811 - lava_callback - INFO - Completed processing callback for node 69b296ee4bce73bcf0234ca0 INFO: 172.20.0.1:45906 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:36:43,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:36:43,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48994 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:37:31,461 - lava_callback - INFO - Received callback for node 69b2933e4bce73bcf0234be7 INFO: 172.20.0.1:45930 - "POST /node/69b2933e4bce73bcf0234be7 HTTP/1.0" 202 Accepted 2026-03-12 10:37:31,468 - lava_callback - INFO - Starting async job submit for node 69b2933e4bce73bcf0234be7 2026-03-12 10:37:31,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:37:31,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2933e4bce73bcf0234be7 HTTP/1.1" 200 None 2026-03-12 10:37:31,550 - lava_callback - DEBUG - Processing node 69b2933e4bce73bcf0234be7: ltp-smoke 2026-03-12 10:37:32,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:37:32,188 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:37:32,189 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b2933e4bce73bcf0234be7/log.txt.gz 2026-03-12 10:37:32,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:37:32,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:37:32,695 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b2933e4bce73bcf0234be7/lava-logs.yaml 2026-03-12 10:37:32,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:37:32,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:37:32,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b2933e4bce73bcf0234be7/lava_callback.json.gz 2026-03-12 10:37:33,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:37:33,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2933e4bce73bcf0234be7 HTTP/1.1" 200 None 2026-03-12 10:37:33,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:37:33,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2766c4bce73bcf0233d1a HTTP/1.1" 200 None 2026-03-12 10:37:33,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:37:33,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2933e4bce73bcf0234be7 HTTP/1.1" 200 None 2026-03-12 10:37:33,178 - lava_callback - INFO - Completed processing callback for node 69b2933e4bce73bcf0234be7 INFO: 172.20.0.1:45942 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:37:41,199 - lava_callback - INFO - Received callback for node 69b297384bce73bcf0234caf INFO: 172.20.0.1:60198 - "POST /node/69b297384bce73bcf0234caf HTTP/1.0" 202 Accepted 2026-03-12 10:37:41,206 - lava_callback - INFO - Starting async job submit for node 69b297384bce73bcf0234caf 2026-03-12 10:37:41,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:37:41,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b297384bce73bcf0234caf HTTP/1.1" 200 None 2026-03-12 10:37:41,289 - lava_callback - DEBUG - Processing node 69b297384bce73bcf0234caf: baseline-arm64 2026-03-12 10:37:41,418 - lava_callback - WARNING - Failed to upload log 2026-03-12 10:37:41,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:37:41,662 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:37:41,664 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b297384bce73bcf0234caf/lava-logs.yaml 2026-03-12 10:37:41,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:37:41,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:37:41,785 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b297384bce73bcf0234caf/lava_callback.json.gz 2026-03-12 10:37:41,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:37:41,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b297384bce73bcf0234caf HTTP/1.1" 200 None 2026-03-12 10:37:41,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:37:41,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:37:41,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:37:41,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b297384bce73bcf0234caf HTTP/1.1" 200 None 2026-03-12 10:37:41,957 - lava_callback - INFO - Completed processing callback for node 69b297384bce73bcf0234caf 2026-03-12 10:37:43,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:37:43,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46530 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:39:36,977 - lava_callback - INFO - Received callback for node 69b297ab4bce73bcf0234cd0 INFO: 172.20.0.1:46532 - "POST /node/69b297ab4bce73bcf0234cd0 HTTP/1.0" 202 Accepted 2026-03-12 10:39:36,983 - lava_callback - INFO - Starting async job submit for node 69b297ab4bce73bcf0234cd0 2026-03-12 10:39:37,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:39:37,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b297ab4bce73bcf0234cd0 HTTP/1.1" 200 None 2026-03-12 10:39:37,025 - lava_callback - DEBUG - Processing node 69b297ab4bce73bcf0234cd0: baseline-arm64 2026-03-12 10:39:37,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:39:37,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:39:37,599 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b297ab4bce73bcf0234cd0/log.txt.gz 2026-03-12 10:39:37,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:39:38,078 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:39:38,080 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b297ab4bce73bcf0234cd0/lava-logs.yaml 2026-03-12 10:39:38,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:39:38,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:39:38,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b297ab4bce73bcf0234cd0/lava_callback.json.gz 2026-03-12 10:39:38,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:39:38,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b297ab4bce73bcf0234cd0 HTTP/1.1" 200 None 2026-03-12 10:39:38,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:39:38,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:39:38,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:39:38,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b297ab4bce73bcf0234cd0 HTTP/1.1" 200 None 2026-03-12 10:39:38,323 - lava_callback - INFO - Completed processing callback for node 69b297ab4bce73bcf0234cd0 2026-03-12 10:39:43,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:39:43,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48888 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:40:53,004 - lava_callback - INFO - Received callback for node 69b293944bce73bcf0234c00 INFO: 172.20.0.1:51938 - "POST /node/69b293944bce73bcf0234c00 HTTP/1.0" 202 Accepted 2026-03-12 10:40:53,013 - lava_callback - INFO - Starting async job submit for node 69b293944bce73bcf0234c00 2026-03-12 10:40:53,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:40:53,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b293944bce73bcf0234c00 HTTP/1.1" 200 None 2026-03-12 10:40:53,072 - lava_callback - DEBUG - Processing node 69b293944bce73bcf0234c00: baseline-nfs-arm64 2026-03-12 10:40:54,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:40:54,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:40:54,335 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b293944bce73bcf0234c00/log.txt.gz 2026-03-12 10:40:54,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:40:55,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:40:55,207 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b293944bce73bcf0234c00/lava-logs.yaml 2026-03-12 10:40:55,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:40:55,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:40:55,433 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b293944bce73bcf0234c00/lava_callback.json.gz INFO: 172.20.0.1:51954 - "GET / HTTP/1.0" 200 OK DEBUG: Setup failed for 69b293944bce73bcf0234c00. Transitting job node result: incomplete -> fail 2026-03-12 10:40:55,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:40:55,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b293944bce73bcf0234c00 HTTP/1.1" 200 None 2026-03-12 10:40:55,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:40:55,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:40:55,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:40:55,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b293944bce73bcf0234c00 HTTP/1.1" 200 None 2026-03-12 10:40:55,680 - lava_callback - INFO - Completed processing callback for node 69b293944bce73bcf0234c00 INFO: 172.20.0.1:51964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52328 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:41:13,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:41:13,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53088 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:43:15,123 - lava_callback - INFO - Received callback for node 69b2980b4bce73bcf0234cea INFO: 172.20.0.1:51094 - "POST /node/69b2980b4bce73bcf0234cea HTTP/1.0" 202 Accepted 2026-03-12 10:43:15,130 - lava_callback - INFO - Starting async job submit for node 69b2980b4bce73bcf0234cea INFO: 172.20.0.1:51102 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 10:43:15,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:43:15,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2980b4bce73bcf0234cea HTTP/1.1" 200 None 2026-03-12 10:43:15,189 - lava_callback - DEBUG - Processing node 69b2980b4bce73bcf0234cea: baseline-arm64-foundriesio 2026-03-12 10:43:15,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:43:16,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:43:16,269 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b2980b4bce73bcf0234cea/log.txt.gz INFO: 172.20.0.1:51116 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:43:16,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:43:16,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:43:16,990 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b2980b4bce73bcf0234cea/lava-logs.yaml 2026-03-12 10:43:17,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:43:17,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:43:17,219 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b2980b4bce73bcf0234cea/lava_callback.json.gz DEBUG: 69b2980b4bce73bcf0234cea Transitting job node result: pass -> incomplete 2026-03-12 10:43:17,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:43:17,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2980b4bce73bcf0234cea HTTP/1.1" 200 None 2026-03-12 10:43:17,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:43:17,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:43:17,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:43:17,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2980b4bce73bcf0234cea HTTP/1.1" 200 None 2026-03-12 10:43:17,335 - lava_callback - INFO - Completed processing callback for node 69b2980b4bce73bcf0234cea INFO: 172.20.0.1:39430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36472 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:43:43,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:43:43,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44860 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:43:59,650 - lava_callback - INFO - Received callback for node 69aff5e3ae557f2600f102e9 INFO: 172.20.0.1:44866 - "POST /node/69aff5e3ae557f2600f102e9 HTTP/1.0" 202 Accepted 2026-03-12 10:43:59,655 - lava_callback - INFO - Starting async job submit for node 69aff5e3ae557f2600f102e9 2026-03-12 10:43:59,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:43:59,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff5e3ae557f2600f102e9 HTTP/1.1" 200 None 2026-03-12 10:43:59,671 - lava_callback - DEBUG - Processing node 69aff5e3ae557f2600f102e9: baseline-arm64-baylibre 2026-03-12 10:43:59,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff5e3ae557f2600f102e9 2026-03-12 10:43:59,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:43:59,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff5e3ae557f2600f102e9 HTTP/1.1" 200 None 2026-03-12 10:43:59,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:43:59,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 10:43:59,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:43:59,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff5e3ae557f2600f102e9 HTTP/1.1" 200 None 2026-03-12 10:43:59,726 - lava_callback - ERROR - Error processing callback for node 69aff5e3ae557f2600f102e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36784 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:45:08,597 - lava_callback - INFO - Received callback for node 69b2991d4bce73bcf0234d28 INFO: 172.20.0.1:36790 - "POST /node/69b2991d4bce73bcf0234d28 HTTP/1.0" 202 Accepted 2026-03-12 10:45:08,603 - lava_callback - INFO - Starting async job submit for node 69b2991d4bce73bcf0234d28 2026-03-12 10:45:08,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:45:08,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2991d4bce73bcf0234d28 HTTP/1.1" 200 None 2026-03-12 10:45:08,620 - lava_callback - DEBUG - Processing node 69b2991d4bce73bcf0234d28: baseline-nfs-arm64 2026-03-12 10:45:08,621 - lava_callback - WARNING - Failed to upload log 2026-03-12 10:45:08,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:45:08,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:45:08,770 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2991d4bce73bcf0234d28/lava-logs.yaml 2026-03-12 10:45:08,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:45:08,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:45:08,922 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b2991d4bce73bcf0234d28/lava_callback.json.gz 2026-03-12 10:45:08,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:45:08,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2991d4bce73bcf0234d28 HTTP/1.1" 200 None 2026-03-12 10:45:08,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:45:08,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:45:08,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:45:08,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2991d4bce73bcf0234d28 HTTP/1.1" 200 None 2026-03-12 10:45:08,988 - lava_callback - INFO - Completed processing callback for node 69b2991d4bce73bcf0234d28 2026-03-12 10:45:13,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:45:13,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60654 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:46:47,144 - lava_callback - INFO - Received callback for node 69b299604bce73bcf0234d34 INFO: 172.20.0.1:60666 - "POST /node/69b299604bce73bcf0234d34 HTTP/1.0" 202 Accepted 2026-03-12 10:46:47,151 - lava_callback - INFO - Starting async job submit for node 69b299604bce73bcf0234d34 2026-03-12 10:46:47,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:46:47,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b299604bce73bcf0234d34 HTTP/1.1" 200 None 2026-03-12 10:46:47,201 - lava_callback - DEBUG - Processing node 69b299604bce73bcf0234d34: baseline-arm64 2026-03-12 10:46:47,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:46:47,541 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:46:47,543 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b299604bce73bcf0234d34/log.txt.gz 2026-03-12 10:46:47,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:46:47,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:46:47,873 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b299604bce73bcf0234d34/lava-logs.yaml 2026-03-12 10:46:47,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:46:48,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:46:48,010 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b299604bce73bcf0234d34/lava_callback.json.gz 2026-03-12 10:46:48,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:46:48,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b299604bce73bcf0234d34 HTTP/1.1" 200 None 2026-03-12 10:46:48,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:46:48,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:46:48,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:46:48,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b299604bce73bcf0234d34 HTTP/1.1" 200 None 2026-03-12 10:46:48,190 - lava_callback - INFO - Completed processing callback for node 69b299604bce73bcf0234d34 INFO: 172.20.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32856 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:47:13,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:47:13,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41646 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:48:20,533 - lava_callback - INFO - Received callback for node 69b299ac4bce73bcf0234d4a INFO: 172.20.0.1:49374 - "POST /node/69b299ac4bce73bcf0234d4a HTTP/1.0" 202 Accepted 2026-03-12 10:48:20,540 - lava_callback - INFO - Starting async job submit for node 69b299ac4bce73bcf0234d4a 2026-03-12 10:48:20,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:48:20,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b299ac4bce73bcf0234d4a HTTP/1.1" 200 None 2026-03-12 10:48:20,584 - lava_callback - DEBUG - Processing node 69b299ac4bce73bcf0234d4a: baseline-arm64 2026-03-12 10:48:20,714 - lava_callback - WARNING - Failed to upload log 2026-03-12 10:48:20,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:48:20,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:48:20,996 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b299ac4bce73bcf0234d4a/lava-logs.yaml 2026-03-12 10:48:20,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:48:21,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:48:21,166 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b299ac4bce73bcf0234d4a/lava_callback.json.gz 2026-03-12 10:48:21,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:48:21,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b299ac4bce73bcf0234d4a HTTP/1.1" 200 None 2026-03-12 10:48:21,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:48:21,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:48:21,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:48:21,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b299ac4bce73bcf0234d4a HTTP/1.1" 200 None 2026-03-12 10:48:21,334 - lava_callback - INFO - Completed processing callback for node 69b299ac4bce73bcf0234d4a INFO: 172.20.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33088 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:48:43,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:48:43,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33282 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:48:59,902 - lava_callback - INFO - Received callback for node 69aff718ae557f2600f10322 INFO: 172.20.0.1:43186 - "POST /node/69aff718ae557f2600f10322 HTTP/1.0" 202 Accepted 2026-03-12 10:48:59,908 - lava_callback - INFO - Starting async job submit for node 69aff718ae557f2600f10322 2026-03-12 10:48:59,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:48:59,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff718ae557f2600f10322 HTTP/1.1" 200 None 2026-03-12 10:48:59,924 - lava_callback - DEBUG - Processing node 69aff718ae557f2600f10322: baseline-arm64-baylibre 2026-03-12 10:48:59,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff718ae557f2600f10322 2026-03-12 10:48:59,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:48:59,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff718ae557f2600f10322 HTTP/1.1" 200 None 2026-03-12 10:48:59,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:48:59,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 10:48:59,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:49:00,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff718ae557f2600f10322 HTTP/1.1" 200 None 2026-03-12 10:49:00,010 - lava_callback - ERROR - Error processing callback for node 69aff718ae557f2600f10322: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43712 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:49:55,424 - lava_callback - INFO - Received callback for node 69b298854bce73bcf0234d04 INFO: 172.20.0.1:51294 - "POST /node/69b298854bce73bcf0234d04 HTTP/1.0" 202 Accepted 2026-03-12 10:49:55,431 - lava_callback - INFO - Starting async job submit for node 69b298854bce73bcf0234d04 2026-03-12 10:49:55,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:49:55,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b298854bce73bcf0234d04 HTTP/1.1" 200 None 2026-03-12 10:49:55,480 - lava_callback - DEBUG - Processing node 69b298854bce73bcf0234d04: baseline-arm64 INFO: 172.20.0.1:51302 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:49:55,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:49:56,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:49:56,024 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b298854bce73bcf0234d04/log.txt.gz 2026-03-12 10:49:56,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:49:56,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:49:56,472 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b298854bce73bcf0234d04/lava-logs.yaml 2026-03-12 10:49:56,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:51308 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:49:56,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:49:56,634 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b298854bce73bcf0234d04/lava_callback.json.gz DEBUG: Setup failed for 69b298854bce73bcf0234d04. Transitting job node result: incomplete -> fail 2026-03-12 10:49:56,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:49:56,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b298854bce73bcf0234d04 HTTP/1.1" 200 None 2026-03-12 10:49:56,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:49:56,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:49:56,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:49:56,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b298854bce73bcf0234d04 HTTP/1.1" 200 None 2026-03-12 10:49:56,842 - lava_callback - INFO - Completed processing callback for node 69b298854bce73bcf0234d04 INFO: 172.20.0.1:43142 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:50:13,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:50:13,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43166 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:50:20,010 - lava_callback - INFO - Received callback for node 69aff767ae557f2600f1032b INFO: 172.20.0.1:55344 - "POST /node/69aff767ae557f2600f1032b HTTP/1.0" 202 Accepted 2026-03-12 10:50:20,016 - lava_callback - INFO - Starting async job submit for node 69aff767ae557f2600f1032b 2026-03-12 10:50:20,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:50:20,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff767ae557f2600f1032b HTTP/1.1" 200 None 2026-03-12 10:50:20,033 - lava_callback - DEBUG - Processing node 69aff767ae557f2600f1032b: baseline-arm64-baylibre 2026-03-12 10:50:20,033 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff767ae557f2600f1032b 2026-03-12 10:50:20,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:50:20,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff767ae557f2600f1032b HTTP/1.1" 200 None 2026-03-12 10:50:20,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:50:20,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 10:50:20,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:50:20,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff767ae557f2600f1032b HTTP/1.1" 200 None 2026-03-12 10:50:20,101 - lava_callback - ERROR - Error processing callback for node 69aff767ae557f2600f1032b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33026 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 10:52:15,262 - lava_callback - INFO - Received callback for node 69b296c84bce73bcf0234c96 INFO: 172.20.0.1:33042 - "POST /node/69b296c84bce73bcf0234c96 HTTP/1.0" 202 Accepted 2026-03-12 10:52:15,269 - lava_callback - INFO - Starting async job submit for node 69b296c84bce73bcf0234c96 2026-03-12 10:52:15,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:52:15,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b296c84bce73bcf0234c96 HTTP/1.1" 200 None 2026-03-12 10:52:15,336 - lava_callback - DEBUG - Processing node 69b296c84bce73bcf0234c96: baseline-arm 2026-03-12 10:52:15,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:52:16,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:52:16,102 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b296c84bce73bcf0234c96/log.txt.gz 2026-03-12 10:52:16,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33044 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:52:16,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:52:16,702 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b296c84bce73bcf0234c96/lava-logs.yaml 2026-03-12 10:52:16,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:52:16,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:52:16,911 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b296c84bce73bcf0234c96/lava_callback.json.gz DEBUG: Setup failed for 69b296c84bce73bcf0234c96. Transitting job node result: incomplete -> fail 2026-03-12 10:52:17,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:52:17,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b296c84bce73bcf0234c96 HTTP/1.1" 200 None 2026-03-12 10:52:17,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:52:17,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 10:52:17,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:52:17,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b296c84bce73bcf0234c96 HTTP/1.1" 200 None 2026-03-12 10:52:17,195 - lava_callback - INFO - Completed processing callback for node 69b296c84bce73bcf0234c96 INFO: 172.20.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59842 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:52:43,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:52:43,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47254 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:54:49,102 - lava_callback - INFO - Received callback for node 69b29ac34bce73bcf0234d9d INFO: 172.20.0.1:47264 - "POST /node/69b29ac34bce73bcf0234d9d HTTP/1.0" 202 Accepted 2026-03-12 10:54:49,110 - lava_callback - INFO - Starting async job submit for node 69b29ac34bce73bcf0234d9d 2026-03-12 10:54:49,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:54:49,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29ac34bce73bcf0234d9d HTTP/1.1" 200 None 2026-03-12 10:54:49,161 - lava_callback - DEBUG - Processing node 69b29ac34bce73bcf0234d9d: baseline-arm64-foundriesio 2026-03-12 10:54:49,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:54:50,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:54:50,178 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b29ac34bce73bcf0234d9d/log.txt.gz 2026-03-12 10:54:50,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:54:50,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:54:50,914 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b29ac34bce73bcf0234d9d/lava-logs.yaml 2026-03-12 10:54:50,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:54:51,107 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:54:51,109 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69b29ac34bce73bcf0234d9d/lava_callback.json.gz DEBUG: 69b29ac34bce73bcf0234d9d Transitting job node result: pass -> incomplete 2026-03-12 10:54:51,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:54:51,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29ac34bce73bcf0234d9d HTTP/1.1" 200 None 2026-03-12 10:54:51,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:54:51,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 10:54:51,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:54:51,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b29ac34bce73bcf0234d9d HTTP/1.1" 200 None 2026-03-12 10:54:51,209 - lava_callback - INFO - Completed processing callback for node 69b29ac34bce73bcf0234d9d INFO: 172.20.0.1:36618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:55:13,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:55:13,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57742 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:56:20,475 - lava_callback - INFO - Received callback for node 69aff8bfae557f2600f10381 INFO: 172.20.0.1:49910 - "POST /node/69aff8bfae557f2600f10381 HTTP/1.0" 202 Accepted 2026-03-12 10:56:20,481 - lava_callback - INFO - Starting async job submit for node 69aff8bfae557f2600f10381 2026-03-12 10:56:20,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:56:20,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff8bfae557f2600f10381 HTTP/1.1" 200 None 2026-03-12 10:56:20,498 - lava_callback - DEBUG - Processing node 69aff8bfae557f2600f10381: baseline-arm64-baylibre 2026-03-12 10:56:20,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff8bfae557f2600f10381 2026-03-12 10:56:20,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:56:20,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff8bfae557f2600f10381 HTTP/1.1" 200 None 2026-03-12 10:56:20,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:56:20,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 10:56:20,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:56:20,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff8bfae557f2600f10381 HTTP/1.1" 200 None 2026-03-12 10:56:20,555 - lava_callback - ERROR - Error processing callback for node 69aff8bfae557f2600f10381: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40324 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:57:20,232 - lava_callback - INFO - Received callback for node 69aff90cae557f2600f10391 INFO: 172.20.0.1:43062 - "POST /node/69aff90cae557f2600f10391 HTTP/1.0" 202 Accepted 2026-03-12 10:57:20,237 - lava_callback - INFO - Starting async job submit for node 69aff90cae557f2600f10391 2026-03-12 10:57:20,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:57:20,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff90cae557f2600f10391 HTTP/1.1" 200 None 2026-03-12 10:57:20,253 - lava_callback - DEBUG - Processing node 69aff90cae557f2600f10391: baseline-arm64-baylibre 2026-03-12 10:57:20,254 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff90cae557f2600f10391 2026-03-12 10:57:20,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:57:20,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff90cae557f2600f10391 HTTP/1.1" 200 None 2026-03-12 10:57:20,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:57:20,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 10:57:20,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:57:20,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff90cae557f2600f10391 HTTP/1.1" 200 None 2026-03-12 10:57:20,311 - lava_callback - ERROR - Error processing callback for node 69aff90cae557f2600f10391: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43068 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:57:29,015 - lava_callback - INFO - Received callback for node 69b2978e4bce73bcf0234cc5 INFO: 172.20.0.1:43076 - "POST /node/69b2978e4bce73bcf0234cc5 HTTP/1.0" 202 Accepted 2026-03-12 10:57:29,061 - lava_callback - INFO - Starting async job submit for node 69b2978e4bce73bcf0234cc5 2026-03-12 10:57:29,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:57:29,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2978e4bce73bcf0234cc5 HTTP/1.1" 200 None 2026-03-12 10:57:29,102 - lava_callback - DEBUG - Processing node 69b2978e4bce73bcf0234cc5: baseline-arm-pengutronix 2026-03-12 10:57:29,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:57:29,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:57:29,357 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b2978e4bce73bcf0234cc5/log.txt.gz 2026-03-12 10:57:29,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:57:29,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:57:29,736 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b2978e4bce73bcf0234cc5/lava-logs.yaml 2026-03-12 10:57:29,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 10:57:29,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 10:57:29,890 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b2978e4bce73bcf0234cc5/lava_callback.json.gz DEBUG: Setup failed for 69b2978e4bce73bcf0234cc5. Transitting job node result: incomplete -> fail 2026-03-12 10:57:29,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:57:29,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2978e4bce73bcf0234cc5 HTTP/1.1" 200 None 2026-03-12 10:57:29,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:57:29,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 10:57:29,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:57:30,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2978e4bce73bcf0234cc5 HTTP/1.1" 200 None 2026-03-12 10:57:30,081 - lava_callback - INFO - Completed processing callback for node 69b2978e4bce73bcf0234cc5 INFO: 172.20.0.1:53386 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:57:43,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:57:43,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52916 - "GET / HTTP/1.0" 200 OK 2026-03-12 10:58:40,341 - lava_callback - INFO - Received callback for node 69aff95aae557f2600f103a3 INFO: 172.20.0.1:45850 - "POST /node/69aff95aae557f2600f103a3 HTTP/1.0" 202 Accepted 2026-03-12 10:58:40,347 - lava_callback - INFO - Starting async job submit for node 69aff95aae557f2600f103a3 2026-03-12 10:58:40,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:58:40,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff95aae557f2600f103a3 HTTP/1.1" 200 None 2026-03-12 10:58:40,366 - lava_callback - DEBUG - Processing node 69aff95aae557f2600f103a3: baseline-arm64-baylibre 2026-03-12 10:58:40,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff95aae557f2600f103a3 2026-03-12 10:58:40,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:58:40,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff95aae557f2600f103a3 HTTP/1.1" 200 None 2026-03-12 10:58:40,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:58:40,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 10:58:40,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 10:58:40,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff95aae557f2600f103a3 HTTP/1.1" 200 None 2026-03-12 10:58:40,450 - lava_callback - ERROR - Error processing callback for node 69aff95aae557f2600f103a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36786 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:00:08,840 - lava_callback - INFO - Received callback for node 69aff9adae557f2600f103b3 INFO: 172.20.0.1:36788 - "POST /node/69aff9adae557f2600f103b3 HTTP/1.0" 202 Accepted 2026-03-12 11:00:08,846 - lava_callback - INFO - Starting async job submit for node 69aff9adae557f2600f103b3 2026-03-12 11:00:08,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:00:08,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff9adae557f2600f103b3 HTTP/1.1" 200 None 2026-03-12 11:00:08,869 - lava_callback - DEBUG - Processing node 69aff9adae557f2600f103b3: baseline-arm64-clabbe 2026-03-12 11:00:08,869 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69aff9adae557f2600f103b3 2026-03-12 11:00:08,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:00:08,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aff9adae557f2600f103b3 HTTP/1.1" 200 None 2026-03-12 11:00:08,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:00:08,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 11:00:08,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:00:08,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69aff9adae557f2600f103b3 HTTP/1.1" 200 None 2026-03-12 11:00:08,992 - lava_callback - ERROR - Error processing callback for node 69aff9adae557f2600f103b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33206 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:02:17,280 - lava_callback - INFO - Received callback for node 69b29bf84bce73bcf0234e00 INFO: 172.20.0.1:33210 - "POST /node/69b29bf84bce73bcf0234e00 HTTP/1.0" 202 Accepted 2026-03-12 11:02:17,288 - lava_callback - INFO - Starting async job submit for node 69b29bf84bce73bcf0234e00 2026-03-12 11:02:17,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:02:17,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29bf84bce73bcf0234e00 HTTP/1.1" 200 None 2026-03-12 11:02:17,354 - lava_callback - DEBUG - Processing node 69b29bf84bce73bcf0234e00: baseline-arm64 2026-03-12 11:02:17,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:02:17,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:02:17,855 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69b29bf84bce73bcf0234e00/log.txt.gz 2026-03-12 11:02:18,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:02:18,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:02:18,327 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69b29bf84bce73bcf0234e00/lava-logs.yaml 2026-03-12 11:02:18,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:02:18,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:02:18,522 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69b29bf84bce73bcf0234e00/lava_callback.json.gz DEBUG: Setup failed for 69b29bf84bce73bcf0234e00. Transitting job node result: incomplete -> fail 2026-03-12 11:02:18,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:02:18,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29bf84bce73bcf0234e00 HTTP/1.1" 200 None 2026-03-12 11:02:18,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:02:18,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 11:02:18,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:02:18,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b29bf84bce73bcf0234e00 HTTP/1.1" 200 None 2026-03-12 11:02:18,727 - lava_callback - INFO - Completed processing callback for node 69b29bf84bce73bcf0234e00 INFO: 172.20.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49574 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:02:43,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:02:43,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42146 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:02:48,788 - lava_callback - INFO - Received callback for node 69affa53ae557f2600f103d3 INFO: 172.20.0.1:42158 - "POST /node/69affa53ae557f2600f103d3 HTTP/1.0" 202 Accepted 2026-03-12 11:02:48,794 - lava_callback - INFO - Starting async job submit for node 69affa53ae557f2600f103d3 2026-03-12 11:02:48,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:02:48,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69affa53ae557f2600f103d3 HTTP/1.1" 200 None 2026-03-12 11:02:48,820 - lava_callback - DEBUG - Processing node 69affa53ae557f2600f103d3: baseline-arm64-clabbe 2026-03-12 11:02:48,820 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69affa53ae557f2600f103d3 2026-03-12 11:02:48,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:02:48,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69affa53ae557f2600f103d3 HTTP/1.1" 200 None 2026-03-12 11:02:48,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:02:48,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 11:02:48,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:02:48,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69affa53ae557f2600f103d3 HTTP/1.1" 200 None 2026-03-12 11:02:48,878 - lava_callback - ERROR - Error processing callback for node 69affa53ae557f2600f103d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47170 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:04:08,847 - lava_callback - INFO - Received callback for node 69affaa5ae557f2600f103e5 INFO: 172.20.0.1:47176 - "POST /node/69affaa5ae557f2600f103e5 HTTP/1.0" 202 Accepted 2026-03-12 11:04:08,853 - lava_callback - INFO - Starting async job submit for node 69affaa5ae557f2600f103e5 2026-03-12 11:04:08,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:04:08,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69affaa5ae557f2600f103e5 HTTP/1.1" 200 None 2026-03-12 11:04:08,869 - lava_callback - DEBUG - Processing node 69affaa5ae557f2600f103e5: baseline-arm64-clabbe 2026-03-12 11:04:08,869 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69affaa5ae557f2600f103e5 2026-03-12 11:04:08,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:04:08,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69affaa5ae557f2600f103e5 HTTP/1.1" 200 None 2026-03-12 11:04:08,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:04:08,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 11:04:08,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:04:08,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69affaa5ae557f2600f103e5 HTTP/1.1" 200 None 2026-03-12 11:04:08,932 - lava_callback - ERROR - Error processing callback for node 69affaa5ae557f2600f103e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43626 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:04:52,601 - lava_callback - INFO - Received callback for node 69b299aa4bce73bcf0234d47 INFO: 172.20.0.1:41234 - "POST /node/69b299aa4bce73bcf0234d47 HTTP/1.0" 202 Accepted 2026-03-12 11:04:52,608 - lava_callback - INFO - Starting async job submit for node 69b299aa4bce73bcf0234d47 2026-03-12 11:04:52,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:04:52,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b299aa4bce73bcf0234d47 HTTP/1.1" 200 None 2026-03-12 11:04:52,708 - lava_callback - DEBUG - Processing node 69b299aa4bce73bcf0234d47: ltp-smoke 2026-03-12 11:04:53,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:04:53,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:04:53,340 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69b299aa4bce73bcf0234d47/log.txt.gz 2026-03-12 11:04:53,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:04:53,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:04:53,884 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69b299aa4bce73bcf0234d47/lava-logs.yaml 2026-03-12 11:04:53,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:04:54,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:04:54,092 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69b299aa4bce73bcf0234d47/lava_callback.json.gz 2026-03-12 11:04:54,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:04:54,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b299aa4bce73bcf0234d47 HTTP/1.1" 200 None 2026-03-12 11:04:54,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:04:54,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2766c4bce73bcf0233d1a HTTP/1.1" 200 None 2026-03-12 11:04:54,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:04:54,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b299aa4bce73bcf0234d47 HTTP/1.1" 200 None 2026-03-12 11:04:54,387 - lava_callback - INFO - Completed processing callback for node 69b299aa4bce73bcf0234d47 INFO: 172.20.0.1:41246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51292 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:05:13,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:05:13,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32788 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:05:48,910 - lava_callback - INFO - Received callback for node 69affaf8ae557f2600f103f7 INFO: 172.20.0.1:32798 - "POST /node/69affaf8ae557f2600f103f7 HTTP/1.0" 202 Accepted 2026-03-12 11:05:48,915 - lava_callback - INFO - Starting async job submit for node 69affaf8ae557f2600f103f7 2026-03-12 11:05:48,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:05:48,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69affaf8ae557f2600f103f7 HTTP/1.1" 200 None 2026-03-12 11:05:48,946 - lava_callback - DEBUG - Processing node 69affaf8ae557f2600f103f7: baseline-arm64-clabbe 2026-03-12 11:05:48,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69affaf8ae557f2600f103f7 2026-03-12 11:05:48,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:05:48,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69affaf8ae557f2600f103f7 HTTP/1.1" 200 None 2026-03-12 11:05:48,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:05:48,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 11:05:48,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:05:49,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69affaf8ae557f2600f103f7 HTTP/1.1" 200 None 2026-03-12 11:05:49,024 - lava_callback - ERROR - Error processing callback for node 69affaf8ae557f2600f103f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34742 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:07:08,958 - lava_callback - INFO - Received callback for node 69affb4bae557f2600f10403 INFO: 172.20.0.1:34756 - "POST /node/69affb4bae557f2600f10403 HTTP/1.0" 202 Accepted 2026-03-12 11:07:08,963 - lava_callback - INFO - Starting async job submit for node 69affb4bae557f2600f10403 2026-03-12 11:07:08,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:07:08,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69affb4bae557f2600f10403 HTTP/1.1" 200 None 2026-03-12 11:07:08,980 - lava_callback - DEBUG - Processing node 69affb4bae557f2600f10403: baseline-arm64-clabbe 2026-03-12 11:07:08,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69affb4bae557f2600f10403 2026-03-12 11:07:08,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:07:08,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69affb4bae557f2600f10403 HTTP/1.1" 200 None 2026-03-12 11:07:08,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:07:09,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 11:07:09,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:07:09,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69affb4bae557f2600f10403 HTTP/1.1" 200 None 2026-03-12 11:07:09,057 - lava_callback - ERROR - Error processing callback for node 69affb4bae557f2600f10403: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46722 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:08:10,228 - lava_callback - INFO - Received callback for node 69b299fe4bce73bcf0234d60 INFO: 172.20.0.1:53344 - "POST /node/69b299fe4bce73bcf0234d60 HTTP/1.0" 202 Accepted 2026-03-12 11:08:10,235 - lava_callback - INFO - Starting async job submit for node 69b299fe4bce73bcf0234d60 2026-03-12 11:08:10,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:10,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b299fe4bce73bcf0234d60 HTTP/1.1" 200 None 2026-03-12 11:08:10,302 - lava_callback - DEBUG - Processing node 69b299fe4bce73bcf0234d60: baseline-nfs-arm64 2026-03-12 11:08:11,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:08:11,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:08:11,576 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b299fe4bce73bcf0234d60/log.txt.gz 2026-03-12 11:08:12,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:08:12,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:08:12,483 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b299fe4bce73bcf0234d60/lava-logs.yaml 2026-03-12 11:08:12,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:08:12,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:08:12,703 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b299fe4bce73bcf0234d60/lava_callback.json.gz DEBUG: Setup failed for 69b299fe4bce73bcf0234d60. Transitting job node result: incomplete -> fail 2026-03-12 11:08:12,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:12,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b299fe4bce73bcf0234d60 HTTP/1.1" 200 None 2026-03-12 11:08:12,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:12,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 11:08:12,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:12,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b299fe4bce73bcf0234d60 HTTP/1.1" 200 None 2026-03-12 11:08:12,969 - lava_callback - INFO - Completed processing callback for node 69b299fe4bce73bcf0234d60 2026-03-12 11:08:13,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:13,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52802 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:08:29,072 - lava_callback - INFO - Received callback for node 69affb9dae557f2600f10413 INFO: 172.20.0.1:52814 - "POST /node/69affb9dae557f2600f10413 HTTP/1.0" 202 Accepted 2026-03-12 11:08:29,078 - lava_callback - INFO - Starting async job submit for node 69affb9dae557f2600f10413 2026-03-12 11:08:29,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:29,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69affb9dae557f2600f10413 HTTP/1.1" 200 None 2026-03-12 11:08:29,095 - lava_callback - DEBUG - Processing node 69affb9dae557f2600f10413: baseline-arm64-clabbe 2026-03-12 11:08:29,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69affb9dae557f2600f10413 2026-03-12 11:08:29,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:29,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69affb9dae557f2600f10413 HTTP/1.1" 200 None 2026-03-12 11:08:29,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:29,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aeba41feb96b28aa34a78b HTTP/1.1" 200 None 2026-03-12 11:08:29,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:29,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69affb9dae557f2600f10413 HTTP/1.1" 200 None 2026-03-12 11:08:29,159 - lava_callback - ERROR - Error processing callback for node 69affb9dae557f2600f10413: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-12 11:08:33,467 - lava_callback - INFO - Received callback for node 69b2968f4bce73bcf0234c8c INFO: 172.20.0.1:56498 - "POST /node/69b2968f4bce73bcf0234c8c HTTP/1.0" 202 Accepted 2026-03-12 11:08:33,474 - lava_callback - INFO - Starting async job submit for node 69b2968f4bce73bcf0234c8c 2026-03-12 11:08:33,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:33,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2968f4bce73bcf0234c8c HTTP/1.1" 200 None 2026-03-12 11:08:33,510 - lava_callback - DEBUG - Processing node 69b2968f4bce73bcf0234c8c: baseline-arm64-broonie 2026-03-12 11:08:33,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:08:33,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:08:33,856 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2968f4bce73bcf0234c8c/log.txt.gz 2026-03-12 11:08:34,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:08:34,248 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:08:34,250 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2968f4bce73bcf0234c8c/lava-logs.yaml 2026-03-12 11:08:34,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:08:34,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:08:34,364 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2968f4bce73bcf0234c8c/lava_callback.json.gz 2026-03-12 11:08:34,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:34,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2968f4bce73bcf0234c8c HTTP/1.1" 200 None 2026-03-12 11:08:34,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:34,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 11:08:34,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:34,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2968f4bce73bcf0234c8c HTTP/1.1" 200 None 2026-03-12 11:08:34,516 - lava_callback - INFO - Completed processing callback for node 69b2968f4bce73bcf0234c8c INFO: 172.20.0.1:56506 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:08:43,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:08:43,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57072 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:14:01,306 - lava_callback - INFO - Received callback for node 69b29c3d4bce73bcf0234e11 INFO: 172.20.0.1:60760 - "POST /node/69b29c3d4bce73bcf0234e11 HTTP/1.0" 202 Accepted 2026-03-12 11:14:01,313 - lava_callback - INFO - Starting async job submit for node 69b29c3d4bce73bcf0234e11 2026-03-12 11:14:01,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:14:01,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29c3d4bce73bcf0234e11 HTTP/1.1" 200 None 2026-03-12 11:14:01,388 - lava_callback - DEBUG - Processing node 69b29c3d4bce73bcf0234e11: baseline-arm 2026-03-12 11:14:01,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:14:02,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:14:02,128 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69b29c3d4bce73bcf0234e11/log.txt.gz 2026-03-12 11:14:02,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:14:02,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:14:02,685 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69b29c3d4bce73bcf0234e11/lava-logs.yaml 2026-03-12 11:14:02,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:14:02,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:14:02,867 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69b29c3d4bce73bcf0234e11/lava_callback.json.gz DEBUG: Setup failed for 69b29c3d4bce73bcf0234e11. Transitting job node result: incomplete -> fail 2026-03-12 11:14:02,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:14:02,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29c3d4bce73bcf0234e11 HTTP/1.1" 200 None 2026-03-12 11:14:02,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:14:03,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 11:14:03,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:14:03,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b29c3d4bce73bcf0234e11 HTTP/1.1" 200 None 2026-03-12 11:14:03,139 - lava_callback - INFO - Completed processing callback for node 69b29c3d4bce73bcf0234e11 INFO: 172.20.0.1:60772 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:14:13,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:14:13,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58052 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:15:41,242 - lava_callback - INFO - Received callback for node 69b280604bce73bcf02344d5 INFO: 172.20.0.1:42946 - "POST /node/69b280604bce73bcf02344d5 HTTP/1.0" 202 Accepted 2026-03-12 11:15:41,248 - lava_callback - INFO - Starting async job submit for node 69b280604bce73bcf02344d5 2026-03-12 11:15:41,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:15:41,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b280604bce73bcf02344d5 HTTP/1.1" 200 None 2026-03-12 11:15:41,280 - lava_callback - DEBUG - Processing node 69b280604bce73bcf02344d5: baseline-x86 2026-03-12 11:15:41,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:15:41,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:15:41,725 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69b280604bce73bcf02344d5/log.txt.gz 2026-03-12 11:15:41,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:15:42,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:15:42,128 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69b280604bce73bcf02344d5/lava-logs.yaml 2026-03-12 11:15:42,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:15:42,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:15:42,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69b280604bce73bcf02344d5/lava_callback.json.gz 2026-03-12 11:15:42,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:15:42,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b280604bce73bcf02344d5 HTTP/1.1" 200 None 2026-03-12 11:15:42,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:15:42,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2768d4bce73bcf0233d3d HTTP/1.1" 200 None 2026-03-12 11:15:42,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:15:42,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b280604bce73bcf02344d5 HTTP/1.1" 200 None 2026-03-12 11:15:42,373 - lava_callback - INFO - Completed processing callback for node 69b280604bce73bcf02344d5 2026-03-12 11:15:43,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:15:43,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58026 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:18:10,276 - lava_callback - INFO - Received callback for node 69b29c6b4bce73bcf0234e21 INFO: 172.20.0.1:45378 - "POST /node/69b29c6b4bce73bcf0234e21 HTTP/1.0" 202 Accepted 2026-03-12 11:18:10,326 - lava_callback - INFO - Starting async job submit for node 69b29c6b4bce73bcf0234e21 2026-03-12 11:18:10,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:18:10,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29c6b4bce73bcf0234e21 HTTP/1.1" 200 None 2026-03-12 11:18:10,389 - lava_callback - DEBUG - Processing node 69b29c6b4bce73bcf0234e21: baseline-arm-pengutronix 2026-03-12 11:18:10,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:18:10,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:18:10,635 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b29c6b4bce73bcf0234e21/log.txt.gz 2026-03-12 11:18:10,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:18:10,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:18:10,879 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b29c6b4bce73bcf0234e21/lava-logs.yaml 2026-03-12 11:18:10,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:18:10,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:18:10,996 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69b29c6b4bce73bcf0234e21/lava_callback.json.gz DEBUG: Setup failed for 69b29c6b4bce73bcf0234e21. Transitting job node result: incomplete -> fail 2026-03-12 11:18:11,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:18:11,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29c6b4bce73bcf0234e21 HTTP/1.1" 200 None 2026-03-12 11:18:11,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:18:11,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276654bce73bcf0233d14 HTTP/1.1" 200 None 2026-03-12 11:18:11,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:18:11,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b29c6b4bce73bcf0234e21 HTTP/1.1" 200 None 2026-03-12 11:18:11,160 - lava_callback - INFO - Completed processing callback for node 69b29c6b4bce73bcf0234e21 2026-03-12 11:18:13,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:18:13,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34900 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47378 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:28:02,291 - lava_callback - INFO - Received callback for node 69b00040ae557f2600f10516 INFO: 172.20.0.1:36652 - "POST /node/69b00040ae557f2600f10516 HTTP/1.0" 202 Accepted 2026-03-12 11:28:02,296 - lava_callback - INFO - Starting async job submit for node 69b00040ae557f2600f10516 2026-03-12 11:28:02,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:02,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00040ae557f2600f10516 HTTP/1.1" 200 None 2026-03-12 11:28:02,318 - lava_callback - DEBUG - Processing node 69b00040ae557f2600f10516: baseline-arm-baylibre 2026-03-12 11:28:02,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00040ae557f2600f10516 2026-03-12 11:28:02,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:02,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00040ae557f2600f10516 HTTP/1.1" 200 None 2026-03-12 11:28:02,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:02,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:28:02,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:02,379 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00040ae557f2600f10516 HTTP/1.1" 200 None 2026-03-12 11:28:02,389 - lava_callback - ERROR - Error processing callback for node 69b00040ae557f2600f10516: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36664 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:28:11,195 - lava_callback - INFO - Received callback for node 69b29ea54bce73bcf0234e5d INFO: 172.20.0.1:45922 - "POST /node/69b29ea54bce73bcf0234e5d HTTP/1.0" 202 Accepted 2026-03-12 11:28:11,202 - lava_callback - INFO - Starting async job submit for node 69b29ea54bce73bcf0234e5d 2026-03-12 11:28:11,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:11,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29ea54bce73bcf0234e5d HTTP/1.1" 200 None 2026-03-12 11:28:11,260 - lava_callback - DEBUG - Processing node 69b29ea54bce73bcf0234e5d: baseline-nfs-arm64 2026-03-12 11:28:12,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:28:12,527 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:28:12,529 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b29ea54bce73bcf0234e5d/log.txt.gz 2026-03-12 11:28:13,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:28:13,390 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:28:13,393 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b29ea54bce73bcf0234e5d/lava-logs.yaml 2026-03-12 11:28:13,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:28:13,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:28:13,593 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69b29ea54bce73bcf0234e5d/lava_callback.json.gz DEBUG: Setup failed for 69b29ea54bce73bcf0234e5d. Transitting job node result: incomplete -> fail 2026-03-12 11:28:13,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:13,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29ea54bce73bcf0234e5d HTTP/1.1" 200 None 2026-03-12 11:28:13,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:13,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 11:28:13,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:13,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b29ea54bce73bcf0234e5d HTTP/1.1" 200 None 2026-03-12 11:28:13,829 - lava_callback - INFO - Completed processing callback for node 69b29ea54bce73bcf0234e5d 2026-03-12 11:28:13,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:14,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36760 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:28:42,005 - lava_callback - INFO - Received callback for node 69b00065ae557f2600f10525 INFO: 172.20.0.1:40148 - "POST /node/69b00065ae557f2600f10525 HTTP/1.0" 202 Accepted 2026-03-12 11:28:42,010 - lava_callback - INFO - Starting async job submit for node 69b00065ae557f2600f10525 2026-03-12 11:28:42,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:42,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00065ae557f2600f10525 HTTP/1.1" 200 None 2026-03-12 11:28:42,027 - lava_callback - DEBUG - Processing node 69b00065ae557f2600f10525: baseline-arm-baylibre 2026-03-12 11:28:42,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00065ae557f2600f10525 2026-03-12 11:28:42,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:42,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00065ae557f2600f10525 HTTP/1.1" 200 None 2026-03-12 11:28:42,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:42,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:28:42,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:28:42,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00065ae557f2600f10525 HTTP/1.1" 200 None 2026-03-12 11:28:42,096 - lava_callback - ERROR - Error processing callback for node 69b00065ae557f2600f10525: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59640 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:30:42,173 - lava_callback - INFO - Received callback for node 69b000d7ae557f2600f1053e INFO: 172.20.0.1:43050 - "POST /node/69b000d7ae557f2600f1053e HTTP/1.0" 202 Accepted 2026-03-12 11:30:42,178 - lava_callback - INFO - Starting async job submit for node 69b000d7ae557f2600f1053e 2026-03-12 11:30:42,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:30:42,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b000d7ae557f2600f1053e HTTP/1.1" 200 None 2026-03-12 11:30:42,231 - lava_callback - DEBUG - Processing node 69b000d7ae557f2600f1053e: baseline-arm-baylibre 2026-03-12 11:30:42,231 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b000d7ae557f2600f1053e 2026-03-12 11:30:42,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:30:42,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b000d7ae557f2600f1053e HTTP/1.1" 200 None 2026-03-12 11:30:42,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:30:42,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:30:42,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:30:42,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b000d7ae557f2600f1053e HTTP/1.1" 200 None 2026-03-12 11:30:42,315 - lava_callback - ERROR - Error processing callback for node 69b000d7ae557f2600f1053e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59712 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:31:22,146 - lava_callback - INFO - Received callback for node 69b000fcae557f2600f10548 INFO: 172.20.0.1:41844 - "POST /node/69b000fcae557f2600f10548 HTTP/1.0" 202 Accepted 2026-03-12 11:31:22,151 - lava_callback - INFO - Starting async job submit for node 69b000fcae557f2600f10548 2026-03-12 11:31:22,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:31:22,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b000fcae557f2600f10548 HTTP/1.1" 200 None 2026-03-12 11:31:22,174 - lava_callback - DEBUG - Processing node 69b000fcae557f2600f10548: baseline-arm-baylibre 2026-03-12 11:31:22,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b000fcae557f2600f10548 2026-03-12 11:31:22,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:31:22,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b000fcae557f2600f10548 HTTP/1.1" 200 None 2026-03-12 11:31:22,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:31:22,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:31:22,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:31:22,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b000fcae557f2600f10548 HTTP/1.1" 200 None 2026-03-12 11:31:22,256 - lava_callback - ERROR - Error processing callback for node 69b000fcae557f2600f10548: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37680 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:32:02,236 - lava_callback - INFO - Received callback for node 69b00122ae557f2600f1055a INFO: 172.20.0.1:44274 - "POST /node/69b00122ae557f2600f1055a HTTP/1.0" 202 Accepted 2026-03-12 11:32:02,241 - lava_callback - INFO - Starting async job submit for node 69b00122ae557f2600f1055a 2026-03-12 11:32:02,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:32:02,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00122ae557f2600f1055a HTTP/1.1" 200 None 2026-03-12 11:32:02,257 - lava_callback - DEBUG - Processing node 69b00122ae557f2600f1055a: baseline-arm-baylibre 2026-03-12 11:32:02,258 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00122ae557f2600f1055a 2026-03-12 11:32:02,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:32:02,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00122ae557f2600f1055a HTTP/1.1" 200 None 2026-03-12 11:32:02,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:32:02,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:32:02,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:32:02,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00122ae557f2600f1055a HTTP/1.1" 200 None 2026-03-12 11:32:02,309 - lava_callback - ERROR - Error processing callback for node 69b00122ae557f2600f1055a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45144 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:32:42,197 - lava_callback - INFO - Received callback for node 69b00146ae557f2600f10564 INFO: 172.20.0.1:60156 - "POST /node/69b00146ae557f2600f10564 HTTP/1.0" 202 Accepted 2026-03-12 11:32:42,202 - lava_callback - INFO - Starting async job submit for node 69b00146ae557f2600f10564 2026-03-12 11:32:42,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:32:42,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00146ae557f2600f10564 HTTP/1.1" 200 None 2026-03-12 11:32:42,219 - lava_callback - DEBUG - Processing node 69b00146ae557f2600f10564: baseline-arm-baylibre 2026-03-12 11:32:42,219 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00146ae557f2600f10564 2026-03-12 11:32:42,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:32:42,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00146ae557f2600f10564 HTTP/1.1" 200 None 2026-03-12 11:32:42,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:32:42,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:32:42,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:32:42,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00146ae557f2600f10564 HTTP/1.1" 200 None 2026-03-12 11:32:42,269 - lava_callback - ERROR - Error processing callback for node 69b00146ae557f2600f10564: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52054 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:35:50,229 - lava_callback - INFO - Received callback for node 69b00201ae557f2600f10594 INFO: 172.20.0.1:45728 - "POST /node/69b00201ae557f2600f10594 HTTP/1.0" 202 Accepted 2026-03-12 11:35:50,235 - lava_callback - INFO - Starting async job submit for node 69b00201ae557f2600f10594 2026-03-12 11:35:50,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:35:50,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00201ae557f2600f10594 HTTP/1.1" 200 None 2026-03-12 11:35:50,278 - lava_callback - DEBUG - Processing node 69b00201ae557f2600f10594: baseline-arm-clabbe 2026-03-12 11:35:50,279 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00201ae557f2600f10594 2026-03-12 11:35:50,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:35:50,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00201ae557f2600f10594 HTTP/1.1" 200 None 2026-03-12 11:35:50,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:35:50,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:35:50,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:35:50,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00201ae557f2600f10594 HTTP/1.1" 200 None 2026-03-12 11:35:50,363 - lava_callback - ERROR - Error processing callback for node 69b00201ae557f2600f10594: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57344 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:36:10,215 - lava_callback - INFO - Received callback for node 69b00226ae557f2600f1059e INFO: 172.20.0.1:45930 - "POST /node/69b00226ae557f2600f1059e HTTP/1.0" 202 Accepted 2026-03-12 11:36:10,220 - lava_callback - INFO - Starting async job submit for node 69b00226ae557f2600f1059e 2026-03-12 11:36:10,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:36:10,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00226ae557f2600f1059e HTTP/1.1" 200 None 2026-03-12 11:36:10,237 - lava_callback - DEBUG - Processing node 69b00226ae557f2600f1059e: baseline-arm-clabbe 2026-03-12 11:36:10,237 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00226ae557f2600f1059e 2026-03-12 11:36:10,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:36:10,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00226ae557f2600f1059e HTTP/1.1" 200 None 2026-03-12 11:36:10,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:36:10,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:36:10,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:36:10,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00226ae557f2600f1059e HTTP/1.1" 200 None 2026-03-12 11:36:10,320 - lava_callback - ERROR - Error processing callback for node 69b00226ae557f2600f1059e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41786 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:36:50,288 - lava_callback - INFO - Received callback for node 69b0024bae557f2600f105a8 INFO: 172.20.0.1:57206 - "POST /node/69b0024bae557f2600f105a8 HTTP/1.0" 202 Accepted 2026-03-12 11:36:50,293 - lava_callback - INFO - Starting async job submit for node 69b0024bae557f2600f105a8 2026-03-12 11:36:50,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:36:50,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0024bae557f2600f105a8 HTTP/1.1" 200 None 2026-03-12 11:36:50,329 - lava_callback - DEBUG - Processing node 69b0024bae557f2600f105a8: baseline-arm-clabbe 2026-03-12 11:36:50,329 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0024bae557f2600f105a8 2026-03-12 11:36:50,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:36:50,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0024bae557f2600f105a8 HTTP/1.1" 200 None 2026-03-12 11:36:50,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:36:50,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:36:50,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:36:50,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0024bae557f2600f105a8 HTTP/1.1" 200 None 2026-03-12 11:36:50,435 - lava_callback - ERROR - Error processing callback for node 69b0024bae557f2600f105a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51376 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:37:30,261 - lava_callback - INFO - Received callback for node 69b00273ae557f2600f105b4 INFO: 172.20.0.1:60194 - "POST /node/69b00273ae557f2600f105b4 HTTP/1.0" 202 Accepted 2026-03-12 11:37:30,266 - lava_callback - INFO - Starting async job submit for node 69b00273ae557f2600f105b4 2026-03-12 11:37:30,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:37:30,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00273ae557f2600f105b4 HTTP/1.1" 200 None 2026-03-12 11:37:30,283 - lava_callback - DEBUG - Processing node 69b00273ae557f2600f105b4: baseline-arm-clabbe 2026-03-12 11:37:30,283 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00273ae557f2600f105b4 2026-03-12 11:37:30,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:37:30,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00273ae557f2600f105b4 HTTP/1.1" 200 None 2026-03-12 11:37:30,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:37:30,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:37:30,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:37:30,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00273ae557f2600f105b4 HTTP/1.1" 200 None 2026-03-12 11:37:30,382 - lava_callback - ERROR - Error processing callback for node 69b00273ae557f2600f105b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56048 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:38:10,293 - lava_callback - INFO - Received callback for node 69b00299ae557f2600f105be INFO: 172.20.0.1:47596 - "POST /node/69b00299ae557f2600f105be HTTP/1.0" 202 Accepted 2026-03-12 11:38:10,299 - lava_callback - INFO - Starting async job submit for node 69b00299ae557f2600f105be 2026-03-12 11:38:10,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:38:10,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00299ae557f2600f105be HTTP/1.1" 200 None 2026-03-12 11:38:10,336 - lava_callback - DEBUG - Processing node 69b00299ae557f2600f105be: baseline-arm-clabbe 2026-03-12 11:38:10,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00299ae557f2600f105be 2026-03-12 11:38:10,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:38:10,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00299ae557f2600f105be HTTP/1.1" 200 None 2026-03-12 11:38:10,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:38:10,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef424ce46d8ec186ab80e HTTP/1.1" 200 None 2026-03-12 11:38:10,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:38:10,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00299ae557f2600f105be HTTP/1.1" 200 None 2026-03-12 11:38:10,426 - lava_callback - ERROR - Error processing callback for node 69b00299ae557f2600f105be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34098 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:44:33,460 - lava_callback - INFO - Received callback for node 69b29efb4bce73bcf0234e6f INFO: 172.20.0.1:43446 - "POST /node/69b29efb4bce73bcf0234e6f HTTP/1.0" 202 Accepted 2026-03-12 11:44:33,465 - lava_callback - INFO - Starting async job submit for node 69b29efb4bce73bcf0234e6f 2026-03-12 11:44:33,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:44:33,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29efb4bce73bcf0234e6f HTTP/1.1" 200 None 2026-03-12 11:44:33,522 - lava_callback - DEBUG - Processing node 69b29efb4bce73bcf0234e6f: baseline-arm64-broonie 2026-03-12 11:44:33,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:44:33,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:44:33,741 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b29efb4bce73bcf0234e6f/log.txt.gz 2026-03-12 11:44:33,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:44:33,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:44:33,978 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b29efb4bce73bcf0234e6f/lava-logs.yaml 2026-03-12 11:44:33,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 11:44:34,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 11:44:34,099 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b29efb4bce73bcf0234e6f/lava_callback.json.gz 2026-03-12 11:44:34,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:44:34,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b29efb4bce73bcf0234e6f HTTP/1.1" 200 None 2026-03-12 11:44:34,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:44:34,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 11:44:34,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:44:34,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b29efb4bce73bcf0234e6f HTTP/1.1" 200 None 2026-03-12 11:44:34,212 - lava_callback - INFO - Completed processing callback for node 69b29efb4bce73bcf0234e6f INFO: 172.20.0.1:43462 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:44:44,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:44:44,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48892 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:49:23,277 - lava_callback - INFO - Received callback for node 69b00530ae557f2600f10651 INFO: 172.20.0.1:46506 - "POST /node/69b00530ae557f2600f10651 HTTP/1.0" 202 Accepted 2026-03-12 11:49:23,283 - lava_callback - INFO - Starting async job submit for node 69b00530ae557f2600f10651 2026-03-12 11:49:23,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:49:23,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00530ae557f2600f10651 HTTP/1.1" 200 None 2026-03-12 11:49:23,319 - lava_callback - DEBUG - Processing node 69b00530ae557f2600f10651: baseline-arm64-baylibre 2026-03-12 11:49:23,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00530ae557f2600f10651 2026-03-12 11:49:23,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:49:23,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00530ae557f2600f10651 HTTP/1.1" 200 None 2026-03-12 11:49:23,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:49:23,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-12 11:49:23,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:49:23,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00530ae557f2600f10651 HTTP/1.1" 200 None 2026-03-12 11:49:23,420 - lava_callback - ERROR - Error processing callback for node 69b00530ae557f2600f10651: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44708 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:51:43,383 - lava_callback - INFO - Received callback for node 69b005c9ae557f2600f10669 INFO: 172.20.0.1:48774 - "POST /node/69b005c9ae557f2600f10669 HTTP/1.0" 202 Accepted 2026-03-12 11:51:43,388 - lava_callback - INFO - Starting async job submit for node 69b005c9ae557f2600f10669 2026-03-12 11:51:43,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:51:43,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b005c9ae557f2600f10669 HTTP/1.1" 200 None 2026-03-12 11:51:43,429 - lava_callback - DEBUG - Processing node 69b005c9ae557f2600f10669: baseline-arm64-baylibre 2026-03-12 11:51:43,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b005c9ae557f2600f10669 2026-03-12 11:51:43,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:51:43,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b005c9ae557f2600f10669 HTTP/1.1" 200 None 2026-03-12 11:51:43,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:51:43,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-12 11:51:43,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:51:43,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b005c9ae557f2600f10669 HTTP/1.1" 200 None 2026-03-12 11:51:43,533 - lava_callback - ERROR - Error processing callback for node 69b005c9ae557f2600f10669: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56354 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:54:23,496 - lava_callback - INFO - Received callback for node 69b00663ae557f2600f10685 INFO: 172.20.0.1:44102 - "POST /node/69b00663ae557f2600f10685 HTTP/1.0" 202 Accepted 2026-03-12 11:54:23,502 - lava_callback - INFO - Starting async job submit for node 69b00663ae557f2600f10685 2026-03-12 11:54:23,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:54:23,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00663ae557f2600f10685 HTTP/1.1" 200 None 2026-03-12 11:54:23,518 - lava_callback - DEBUG - Processing node 69b00663ae557f2600f10685: baseline-arm64-baylibre 2026-03-12 11:54:23,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00663ae557f2600f10685 2026-03-12 11:54:23,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:54:23,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00663ae557f2600f10685 HTTP/1.1" 200 None 2026-03-12 11:54:23,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:54:23,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-12 11:54:23,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:54:23,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00663ae557f2600f10685 HTTP/1.1" 200 None 2026-03-12 11:54:23,596 - lava_callback - ERROR - Error processing callback for node 69b00663ae557f2600f10685: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57642 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:57:03,698 - lava_callback - INFO - Received callback for node 69b006feae557f2600f1069a INFO: 172.20.0.1:35460 - "POST /node/69b006feae557f2600f1069a HTTP/1.0" 202 Accepted 2026-03-12 11:57:03,704 - lava_callback - INFO - Starting async job submit for node 69b006feae557f2600f1069a 2026-03-12 11:57:03,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:57:03,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b006feae557f2600f1069a HTTP/1.1" 200 None 2026-03-12 11:57:03,721 - lava_callback - DEBUG - Processing node 69b006feae557f2600f1069a: baseline-arm64-baylibre 2026-03-12 11:57:03,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b006feae557f2600f1069a 2026-03-12 11:57:03,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:57:03,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b006feae557f2600f1069a HTTP/1.1" 200 None 2026-03-12 11:57:03,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:57:03,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-12 11:57:03,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:57:03,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b006feae557f2600f1069a HTTP/1.1" 200 None 2026-03-12 11:57:03,825 - lava_callback - ERROR - Error processing callback for node 69b006feae557f2600f1069a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37810 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:58:11,228 - lava_callback - INFO - Received callback for node 69b0074fae557f2600f106a9 INFO: 172.20.0.1:47082 - "POST /node/69b0074fae557f2600f106a9 HTTP/1.0" 202 Accepted 2026-03-12 11:58:11,233 - lava_callback - INFO - Starting async job submit for node 69b0074fae557f2600f106a9 2026-03-12 11:58:11,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:58:11,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0074fae557f2600f106a9 HTTP/1.1" 200 None 2026-03-12 11:58:11,272 - lava_callback - DEBUG - Processing node 69b0074fae557f2600f106a9: baseline-arm64-clabbe 2026-03-12 11:58:11,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0074fae557f2600f106a9 2026-03-12 11:58:11,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:58:11,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0074fae557f2600f106a9 HTTP/1.1" 200 None 2026-03-12 11:58:11,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:58:11,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-12 11:58:11,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:58:11,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0074fae557f2600f106a9 HTTP/1.1" 200 None 2026-03-12 11:58:11,363 - lava_callback - ERROR - Error processing callback for node 69b0074fae557f2600f106a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59946 - "GET / HTTP/1.0" 200 OK 2026-03-12 11:59:31,261 - lava_callback - INFO - Received callback for node 69b0079cae557f2600f106be INFO: 172.20.0.1:53976 - "POST /node/69b0079cae557f2600f106be HTTP/1.0" 202 Accepted 2026-03-12 11:59:31,267 - lava_callback - INFO - Starting async job submit for node 69b0079cae557f2600f106be 2026-03-12 11:59:31,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:59:31,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0079cae557f2600f106be HTTP/1.1" 200 None 2026-03-12 11:59:31,284 - lava_callback - DEBUG - Processing node 69b0079cae557f2600f106be: baseline-arm64-clabbe 2026-03-12 11:59:31,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0079cae557f2600f106be 2026-03-12 11:59:31,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:59:31,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0079cae557f2600f106be HTTP/1.1" 200 None 2026-03-12 11:59:31,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:59:31,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-12 11:59:31,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 11:59:31,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0079cae557f2600f106be HTTP/1.1" 200 None 2026-03-12 11:59:31,348 - lava_callback - ERROR - Error processing callback for node 69b0079cae557f2600f106be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44154 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:01:31,494 - lava_callback - INFO - Received callback for node 69b00813ae557f2600f106cc INFO: 172.20.0.1:47454 - "POST /node/69b00813ae557f2600f106cc HTTP/1.0" 202 Accepted 2026-03-12 12:01:31,499 - lava_callback - INFO - Starting async job submit for node 69b00813ae557f2600f106cc 2026-03-12 12:01:31,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:01:31,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00813ae557f2600f106cc HTTP/1.1" 200 None 2026-03-12 12:01:31,520 - lava_callback - DEBUG - Processing node 69b00813ae557f2600f106cc: baseline-arm64-clabbe 2026-03-12 12:01:31,520 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00813ae557f2600f106cc 2026-03-12 12:01:31,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:01:31,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00813ae557f2600f106cc HTTP/1.1" 200 None 2026-03-12 12:01:31,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:01:31,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-12 12:01:31,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:01:31,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00813ae557f2600f106cc HTTP/1.1" 200 None 2026-03-12 12:01:31,576 - lava_callback - ERROR - Error processing callback for node 69b00813ae557f2600f106cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60806 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:02:51,418 - lava_callback - INFO - Received callback for node 69b00868ae557f2600f106d8 INFO: 172.20.0.1:36198 - "POST /node/69b00868ae557f2600f106d8 HTTP/1.0" 202 Accepted 2026-03-12 12:02:51,424 - lava_callback - INFO - Starting async job submit for node 69b00868ae557f2600f106d8 2026-03-12 12:02:51,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:02:51,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00868ae557f2600f106d8 HTTP/1.1" 200 None 2026-03-12 12:02:51,458 - lava_callback - DEBUG - Processing node 69b00868ae557f2600f106d8: baseline-arm64-clabbe 2026-03-12 12:02:51,458 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00868ae557f2600f106d8 2026-03-12 12:02:51,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:02:51,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00868ae557f2600f106d8 HTTP/1.1" 200 None 2026-03-12 12:02:51,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:02:51,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-12 12:02:51,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:02:51,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00868ae557f2600f106d8 HTTP/1.1" 200 None 2026-03-12 12:02:51,539 - lava_callback - ERROR - Error processing callback for node 69b00868ae557f2600f106d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33490 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:04:11,468 - lava_callback - INFO - Received callback for node 69b008b7ae557f2600f106ed INFO: 172.20.0.1:36218 - "POST /node/69b008b7ae557f2600f106ed HTTP/1.0" 202 Accepted 2026-03-12 12:04:11,473 - lava_callback - INFO - Starting async job submit for node 69b008b7ae557f2600f106ed 2026-03-12 12:04:11,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:04:11,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b008b7ae557f2600f106ed HTTP/1.1" 200 None 2026-03-12 12:04:11,497 - lava_callback - DEBUG - Processing node 69b008b7ae557f2600f106ed: baseline-arm64-clabbe 2026-03-12 12:04:11,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b008b7ae557f2600f106ed 2026-03-12 12:04:11,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:04:11,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b008b7ae557f2600f106ed HTTP/1.1" 200 None 2026-03-12 12:04:11,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:04:11,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-12 12:04:11,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:04:11,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b008b7ae557f2600f106ed HTTP/1.1" 200 None 2026-03-12 12:04:11,600 - lava_callback - ERROR - Error processing callback for node 69b008b7ae557f2600f106ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57500 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:05:31,594 - lava_callback - INFO - Received callback for node 69b00907ae557f2600f106fb INFO: 172.20.0.1:52722 - "POST /node/69b00907ae557f2600f106fb HTTP/1.0" 202 Accepted 2026-03-12 12:05:31,601 - lava_callback - INFO - Starting async job submit for node 69b00907ae557f2600f106fb 2026-03-12 12:05:31,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:05:31,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00907ae557f2600f106fb HTTP/1.1" 200 None 2026-03-12 12:05:31,617 - lava_callback - DEBUG - Processing node 69b00907ae557f2600f106fb: baseline-arm64-clabbe 2026-03-12 12:05:31,617 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b00907ae557f2600f106fb 2026-03-12 12:05:31,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:05:31,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b00907ae557f2600f106fb HTTP/1.1" 200 None 2026-03-12 12:05:31,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:05:31,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aef431ce46d8ec186ab81d HTTP/1.1" 200 None 2026-03-12 12:05:31,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:05:31,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b00907ae557f2600f106fb HTTP/1.1" 200 None 2026-03-12 12:05:31,689 - lava_callback - ERROR - Error processing callback for node 69b00907ae557f2600f106fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40404 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:17:35,358 - lava_callback - INFO - Received callback for node 69b2a7354bce73bcf0234eb3 INFO: 172.20.0.1:50798 - "POST /node/69b2a7354bce73bcf0234eb3 HTTP/1.0" 202 Accepted 2026-03-12 12:17:35,365 - lava_callback - INFO - Starting async job submit for node 69b2a7354bce73bcf0234eb3 2026-03-12 12:17:35,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:17:35,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2a7354bce73bcf0234eb3 HTTP/1.1" 200 None 2026-03-12 12:17:35,399 - lava_callback - DEBUG - Processing node 69b2a7354bce73bcf0234eb3: baseline-arm64-broonie 2026-03-12 12:17:35,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 12:17:35,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 12:17:35,765 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2a7354bce73bcf0234eb3/log.txt.gz 2026-03-12 12:17:35,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 12:17:36,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 12:17:36,162 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2a7354bce73bcf0234eb3/lava-logs.yaml 2026-03-12 12:17:36,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-12 12:17:36,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-12 12:17:36,310 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69b2a7354bce73bcf0234eb3/lava_callback.json.gz 2026-03-12 12:17:36,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:17:36,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b2a7354bce73bcf0234eb3 HTTP/1.1" 200 None 2026-03-12 12:17:36,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:17:36,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b276764bce73bcf0233d23 HTTP/1.1" 200 None 2026-03-12 12:17:36,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:17:36,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b2a7354bce73bcf0234eb3 HTTP/1.1" 200 None 2026-03-12 12:17:36,452 - lava_callback - INFO - Completed processing callback for node 69b2a7354bce73bcf0234eb3 INFO: 172.20.0.1:50800 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:17:44,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:17:44,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35438 - "GET /node/69ae1eec8ef4424e487b3e5d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44898 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:39:06,091 - lava_callback - INFO - Received callback for node 69b010d8ae557f2600f1072b INFO: 172.20.0.1:50500 - "POST /node/69b010d8ae557f2600f1072b HTTP/1.0" 202 Accepted 2026-03-12 12:39:06,097 - lava_callback - INFO - Starting async job submit for node 69b010d8ae557f2600f1072b 2026-03-12 12:39:06,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:39:06,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b010d8ae557f2600f1072b HTTP/1.1" 200 None 2026-03-12 12:39:06,112 - lava_callback - DEBUG - Processing node 69b010d8ae557f2600f1072b: baseline-arm-baylibre 2026-03-12 12:39:06,113 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b010d8ae557f2600f1072b 2026-03-12 12:39:06,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:39:06,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b010d8ae557f2600f1072b HTTP/1.1" 200 None 2026-03-12 12:39:06,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:39:06,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:39:06,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:39:06,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b010d8ae557f2600f1072b HTTP/1.1" 200 None 2026-03-12 12:39:06,171 - lava_callback - ERROR - Error processing callback for node 69b010d8ae557f2600f1072b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44336 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:39:46,084 - lava_callback - INFO - Received callback for node 69b010fdae557f2600f10739 INFO: 172.20.0.1:40990 - "POST /node/69b010fdae557f2600f10739 HTTP/1.0" 202 Accepted 2026-03-12 12:39:46,090 - lava_callback - INFO - Starting async job submit for node 69b010fdae557f2600f10739 2026-03-12 12:39:46,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:39:46,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b010fdae557f2600f10739 HTTP/1.1" 200 None 2026-03-12 12:39:46,106 - lava_callback - DEBUG - Processing node 69b010fdae557f2600f10739: baseline-arm-baylibre 2026-03-12 12:39:46,106 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b010fdae557f2600f10739 2026-03-12 12:39:46,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:39:46,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b010fdae557f2600f10739 HTTP/1.1" 200 None 2026-03-12 12:39:46,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:39:46,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:39:46,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:39:46,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b010fdae557f2600f10739 HTTP/1.1" 200 None 2026-03-12 12:39:46,166 - lava_callback - ERROR - Error processing callback for node 69b010fdae557f2600f10739: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37740 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:40:26,388 - lava_callback - INFO - Received callback for node 69b0112eae557f2600f10745 INFO: 172.20.0.1:54748 - "POST /node/69b0112eae557f2600f10745 HTTP/1.0" 202 Accepted 2026-03-12 12:40:26,393 - lava_callback - INFO - Starting async job submit for node 69b0112eae557f2600f10745 2026-03-12 12:40:26,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:40:26,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0112eae557f2600f10745 HTTP/1.1" 200 None 2026-03-12 12:40:26,409 - lava_callback - DEBUG - Processing node 69b0112eae557f2600f10745: baseline-arm-baylibre 2026-03-12 12:40:26,409 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0112eae557f2600f10745 2026-03-12 12:40:26,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:40:26,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0112eae557f2600f10745 HTTP/1.1" 200 None 2026-03-12 12:40:26,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:40:26,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:40:26,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:40:26,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0112eae557f2600f10745 HTTP/1.1" 200 None 2026-03-12 12:40:26,465 - lava_callback - ERROR - Error processing callback for node 69b0112eae557f2600f10745: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36740 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:41:06,112 - lava_callback - INFO - Received callback for node 69b01153ae557f2600f10750 INFO: 172.20.0.1:39208 - "POST /node/69b01153ae557f2600f10750 HTTP/1.0" 202 Accepted 2026-03-12 12:41:06,119 - lava_callback - INFO - Starting async job submit for node 69b01153ae557f2600f10750 2026-03-12 12:41:06,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:41:06,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01153ae557f2600f10750 HTTP/1.1" 200 None 2026-03-12 12:41:06,142 - lava_callback - DEBUG - Processing node 69b01153ae557f2600f10750: baseline-arm-baylibre 2026-03-12 12:41:06,143 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01153ae557f2600f10750 2026-03-12 12:41:06,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:41:06,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01153ae557f2600f10750 HTTP/1.1" 200 None 2026-03-12 12:41:06,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:41:06,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:41:06,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:41:06,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01153ae557f2600f10750 HTTP/1.1" 200 None 2026-03-12 12:41:06,212 - lava_callback - ERROR - Error processing callback for node 69b01153ae557f2600f10750: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:41:46,196 - lava_callback - INFO - Received callback for node 69b0117bae557f2600f1075d INFO: 172.20.0.1:38016 - "POST /node/69b0117bae557f2600f1075d HTTP/1.0" 202 Accepted 2026-03-12 12:41:46,201 - lava_callback - INFO - Starting async job submit for node 69b0117bae557f2600f1075d 2026-03-12 12:41:46,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:41:46,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0117bae557f2600f1075d HTTP/1.1" 200 None 2026-03-12 12:41:46,217 - lava_callback - DEBUG - Processing node 69b0117bae557f2600f1075d: baseline-arm-baylibre 2026-03-12 12:41:46,217 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0117bae557f2600f1075d 2026-03-12 12:41:46,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:41:46,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0117bae557f2600f1075d HTTP/1.1" 200 None 2026-03-12 12:41:46,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:41:46,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:41:46,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:41:46,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0117bae557f2600f1075d HTTP/1.1" 200 None 2026-03-12 12:41:46,281 - lava_callback - ERROR - Error processing callback for node 69b0117bae557f2600f1075d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40134 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:42:26,185 - lava_callback - INFO - Received callback for node 69b011a8ae557f2600f1076b INFO: 172.20.0.1:37254 - "POST /node/69b011a8ae557f2600f1076b HTTP/1.0" 202 Accepted 2026-03-12 12:42:26,191 - lava_callback - INFO - Starting async job submit for node 69b011a8ae557f2600f1076b 2026-03-12 12:42:26,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:42:26,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b011a8ae557f2600f1076b HTTP/1.1" 200 None 2026-03-12 12:42:26,209 - lava_callback - DEBUG - Processing node 69b011a8ae557f2600f1076b: baseline-arm-baylibre 2026-03-12 12:42:26,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b011a8ae557f2600f1076b 2026-03-12 12:42:26,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:42:26,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b011a8ae557f2600f1076b HTTP/1.1" 200 None 2026-03-12 12:42:26,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:42:26,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:42:26,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:42:26,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b011a8ae557f2600f1076b HTTP/1.1" 200 None 2026-03-12 12:42:26,303 - lava_callback - ERROR - Error processing callback for node 69b011a8ae557f2600f1076b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56650 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:44:13,368 - lava_callback - INFO - Received callback for node 69b0121aae557f2600f10793 INFO: 172.20.0.1:49798 - "POST /node/69b0121aae557f2600f10793 HTTP/1.0" 202 Accepted 2026-03-12 12:44:13,373 - lava_callback - INFO - Starting async job submit for node 69b0121aae557f2600f10793 2026-03-12 12:44:13,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:44:13,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0121aae557f2600f10793 HTTP/1.1" 200 None 2026-03-12 12:44:13,388 - lava_callback - DEBUG - Processing node 69b0121aae557f2600f10793: baseline-arm-clabbe 2026-03-12 12:44:13,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0121aae557f2600f10793 2026-03-12 12:44:13,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:44:13,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0121aae557f2600f10793 HTTP/1.1" 200 None 2026-03-12 12:44:13,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:44:13,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:44:13,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:44:13,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0121aae557f2600f10793 HTTP/1.1" 200 None 2026-03-12 12:44:13,443 - lava_callback - ERROR - Error processing callback for node 69b0121aae557f2600f10793: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60972 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:45:13,286 - lava_callback - INFO - Received callback for node 69b01243ae557f2600f1079f INFO: 172.20.0.1:47708 - "POST /node/69b01243ae557f2600f1079f HTTP/1.0" 202 Accepted 2026-03-12 12:45:13,292 - lava_callback - INFO - Starting async job submit for node 69b01243ae557f2600f1079f 2026-03-12 12:45:13,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:45:13,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01243ae557f2600f1079f HTTP/1.1" 200 None 2026-03-12 12:45:13,320 - lava_callback - DEBUG - Processing node 69b01243ae557f2600f1079f: baseline-arm-clabbe 2026-03-12 12:45:13,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01243ae557f2600f1079f 2026-03-12 12:45:13,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:45:13,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01243ae557f2600f1079f HTTP/1.1" 200 None 2026-03-12 12:45:13,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:45:13,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:45:13,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:45:13,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01243ae557f2600f1079f HTTP/1.1" 200 None 2026-03-12 12:45:13,385 - lava_callback - ERROR - Error processing callback for node 69b01243ae557f2600f1079f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36144 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:45:53,357 - lava_callback - INFO - Received callback for node 69b0126cae557f2600f107b1 INFO: 172.20.0.1:53958 - "POST /node/69b0126cae557f2600f107b1 HTTP/1.0" 202 Accepted 2026-03-12 12:45:53,363 - lava_callback - INFO - Starting async job submit for node 69b0126cae557f2600f107b1 2026-03-12 12:45:53,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:45:53,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0126cae557f2600f107b1 HTTP/1.1" 200 None 2026-03-12 12:45:53,379 - lava_callback - DEBUG - Processing node 69b0126cae557f2600f107b1: baseline-arm-clabbe 2026-03-12 12:45:53,379 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0126cae557f2600f107b1 2026-03-12 12:45:53,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:45:53,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0126cae557f2600f107b1 HTTP/1.1" 200 None 2026-03-12 12:45:53,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:45:53,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:45:53,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:45:53,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0126cae557f2600f107b1 HTTP/1.1" 200 None 2026-03-12 12:45:53,441 - lava_callback - ERROR - Error processing callback for node 69b0126cae557f2600f107b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47752 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:46:33,463 - lava_callback - INFO - Received callback for node 69b01295ae557f2600f107be INFO: 172.20.0.1:55262 - "POST /node/69b01295ae557f2600f107be HTTP/1.0" 202 Accepted 2026-03-12 12:46:33,468 - lava_callback - INFO - Starting async job submit for node 69b01295ae557f2600f107be 2026-03-12 12:46:33,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:46:33,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01295ae557f2600f107be HTTP/1.1" 200 None 2026-03-12 12:46:33,491 - lava_callback - DEBUG - Processing node 69b01295ae557f2600f107be: baseline-arm-clabbe 2026-03-12 12:46:33,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01295ae557f2600f107be 2026-03-12 12:46:33,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:46:33,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01295ae557f2600f107be HTTP/1.1" 200 None 2026-03-12 12:46:33,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:46:33,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:46:33,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:46:33,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01295ae557f2600f107be HTTP/1.1" 200 None 2026-03-12 12:46:33,545 - lava_callback - ERROR - Error processing callback for node 69b01295ae557f2600f107be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42286 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:47:53,507 - lava_callback - INFO - Received callback for node 69b012e3ae557f2600f107d5 INFO: 172.20.0.1:37600 - "POST /node/69b012e3ae557f2600f107d5 HTTP/1.0" 202 Accepted 2026-03-12 12:47:53,512 - lava_callback - INFO - Starting async job submit for node 69b012e3ae557f2600f107d5 2026-03-12 12:47:53,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:47:53,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b012e3ae557f2600f107d5 HTTP/1.1" 200 None 2026-03-12 12:47:53,528 - lava_callback - DEBUG - Processing node 69b012e3ae557f2600f107d5: baseline-arm-clabbe 2026-03-12 12:47:53,529 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b012e3ae557f2600f107d5 2026-03-12 12:47:53,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:47:53,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b012e3ae557f2600f107d5 HTTP/1.1" 200 None 2026-03-12 12:47:53,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:47:53,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af632c275c8d3b79399db4 HTTP/1.1" 200 None 2026-03-12 12:47:53,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:47:53,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b012e3ae557f2600f107d5 HTTP/1.1" 200 None 2026-03-12 12:47:53,585 - lava_callback - ERROR - Error processing callback for node 69b012e3ae557f2600f107d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52062 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:58:13,891 - lava_callback - INFO - Received callback for node 69b01561ae557f2600f1080c INFO: 172.20.0.1:43600 - "POST /node/69b01561ae557f2600f1080c HTTP/1.0" 202 Accepted 2026-03-12 12:58:13,898 - lava_callback - INFO - Starting async job submit for node 69b01561ae557f2600f1080c 2026-03-12 12:58:13,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:58:13,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01561ae557f2600f1080c HTTP/1.1" 200 None 2026-03-12 12:58:13,916 - lava_callback - DEBUG - Processing node 69b01561ae557f2600f1080c: baseline-arm-clabbe 2026-03-12 12:58:13,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01561ae557f2600f1080c 2026-03-12 12:58:13,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:58:13,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01561ae557f2600f1080c HTTP/1.1" 200 None 2026-03-12 12:58:13,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:58:13,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-12 12:58:13,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:58:13,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01561ae557f2600f1080c HTTP/1.1" 200 None 2026-03-12 12:58:13,993 - lava_callback - ERROR - Error processing callback for node 69b01561ae557f2600f1080c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56166 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:58:53,954 - lava_callback - INFO - Received callback for node 69b01587ae557f2600f10817 INFO: 172.20.0.1:37946 - "POST /node/69b01587ae557f2600f10817 HTTP/1.0" 202 Accepted 2026-03-12 12:58:53,959 - lava_callback - INFO - Starting async job submit for node 69b01587ae557f2600f10817 2026-03-12 12:58:53,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:58:53,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01587ae557f2600f10817 HTTP/1.1" 200 None 2026-03-12 12:58:53,976 - lava_callback - DEBUG - Processing node 69b01587ae557f2600f10817: baseline-arm-clabbe 2026-03-12 12:58:53,976 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01587ae557f2600f10817 2026-03-12 12:58:53,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:58:53,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01587ae557f2600f10817 HTTP/1.1" 200 None 2026-03-12 12:58:53,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:58:54,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-12 12:58:54,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:58:54,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01587ae557f2600f10817 HTTP/1.1" 200 None 2026-03-12 12:58:54,034 - lava_callback - ERROR - Error processing callback for node 69b01587ae557f2600f10817: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47012 - "GET / HTTP/1.0" 200 OK 2026-03-12 12:59:47,241 - lava_callback - INFO - Received callback for node 69b015b2ae557f2600f10825 INFO: 172.20.0.1:47016 - "POST /node/69b015b2ae557f2600f10825 HTTP/1.0" 202 Accepted 2026-03-12 12:59:47,251 - lava_callback - INFO - Starting async job submit for node 69b015b2ae557f2600f10825 2026-03-12 12:59:47,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:59:47,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b015b2ae557f2600f10825 HTTP/1.1" 200 None 2026-03-12 12:59:47,272 - lava_callback - DEBUG - Processing node 69b015b2ae557f2600f10825: baseline-arm64-baylibre 2026-03-12 12:59:47,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b015b2ae557f2600f10825 2026-03-12 12:59:47,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:59:47,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b015b2ae557f2600f10825 HTTP/1.1" 200 None 2026-03-12 12:59:47,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:59:47,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-12 12:59:47,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 12:59:47,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b015b2ae557f2600f10825 HTTP/1.1" 200 None 2026-03-12 12:59:47,350 - lava_callback - ERROR - Error processing callback for node 69b015b2ae557f2600f10825: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40024 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:00:54,088 - lava_callback - INFO - Received callback for node 69b015ffae557f2600f1082f INFO: 172.20.0.1:45112 - "POST /node/69b015ffae557f2600f1082f HTTP/1.0" 202 Accepted 2026-03-12 13:00:54,097 - lava_callback - INFO - Starting async job submit for node 69b015ffae557f2600f1082f 2026-03-12 13:00:54,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:00:54,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b015ffae557f2600f1082f HTTP/1.1" 200 None 2026-03-12 13:00:54,331 - lava_callback - DEBUG - Processing node 69b015ffae557f2600f1082f: baseline-arm-clabbe 2026-03-12 13:00:54,332 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b015ffae557f2600f1082f 2026-03-12 13:00:54,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:00:54,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b015ffae557f2600f1082f HTTP/1.1" 200 None 2026-03-12 13:00:54,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:00:54,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-12 13:00:54,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:00:54,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b015ffae557f2600f1082f HTTP/1.1" 200 None 2026-03-12 13:00:54,418 - lava_callback - ERROR - Error processing callback for node 69b015ffae557f2600f1082f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53746 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:04:27,470 - lava_callback - INFO - Received callback for node 69b016d2ae557f2600f10855 INFO: 172.20.0.1:53762 - "POST /node/69b016d2ae557f2600f10855 HTTP/1.0" 202 Accepted 2026-03-12 13:04:27,479 - lava_callback - INFO - Starting async job submit for node 69b016d2ae557f2600f10855 2026-03-12 13:04:27,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:04:27,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b016d2ae557f2600f10855 HTTP/1.1" 200 None 2026-03-12 13:04:27,533 - lava_callback - DEBUG - Processing node 69b016d2ae557f2600f10855: baseline-arm64-baylibre 2026-03-12 13:04:27,534 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b016d2ae557f2600f10855 2026-03-12 13:04:27,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:04:27,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b016d2ae557f2600f10855 HTTP/1.1" 200 None 2026-03-12 13:04:27,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:04:27,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-12 13:04:27,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:04:27,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b016d2ae557f2600f10855 HTTP/1.1" 200 None 2026-03-12 13:04:27,589 - lava_callback - ERROR - Error processing callback for node 69b016d2ae557f2600f10855: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41002 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:05:47,525 - lava_callback - INFO - Received callback for node 69b01726ae557f2600f10861 INFO: 172.20.0.1:41018 - "POST /node/69b01726ae557f2600f10861 HTTP/1.0" 202 Accepted 2026-03-12 13:05:47,531 - lava_callback - INFO - Starting async job submit for node 69b01726ae557f2600f10861 2026-03-12 13:05:47,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:05:47,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01726ae557f2600f10861 HTTP/1.1" 200 None 2026-03-12 13:05:47,561 - lava_callback - DEBUG - Processing node 69b01726ae557f2600f10861: baseline-arm64-baylibre 2026-03-12 13:05:47,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01726ae557f2600f10861 2026-03-12 13:05:47,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:05:47,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01726ae557f2600f10861 HTTP/1.1" 200 None 2026-03-12 13:05:47,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:05:47,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-12 13:05:47,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:05:47,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01726ae557f2600f10861 HTTP/1.1" 200 None 2026-03-12 13:05:47,618 - lava_callback - ERROR - Error processing callback for node 69b01726ae557f2600f10861: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33734 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:07:34,382 - lava_callback - INFO - Received callback for node 69b0177fae557f2600f10873 INFO: 172.20.0.1:52212 - "POST /node/69b0177fae557f2600f10873 HTTP/1.0" 202 Accepted 2026-03-12 13:07:34,387 - lava_callback - INFO - Starting async job submit for node 69b0177fae557f2600f10873 2026-03-12 13:07:34,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:07:34,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0177fae557f2600f10873 HTTP/1.1" 200 None 2026-03-12 13:07:34,403 - lava_callback - DEBUG - Processing node 69b0177fae557f2600f10873: baseline-arm64-clabbe 2026-03-12 13:07:34,404 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0177fae557f2600f10873 2026-03-12 13:07:34,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:07:34,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0177fae557f2600f10873 HTTP/1.1" 200 None 2026-03-12 13:07:34,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:07:34,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-12 13:07:34,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:07:34,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0177fae557f2600f10873 HTTP/1.1" 200 None 2026-03-12 13:07:34,460 - lava_callback - ERROR - Error processing callback for node 69b0177fae557f2600f10873: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44766 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:08:54,441 - lava_callback - INFO - Received callback for node 69b017d3ae557f2600f10887 INFO: 172.20.0.1:43670 - "POST /node/69b017d3ae557f2600f10887 HTTP/1.0" 202 Accepted 2026-03-12 13:08:54,446 - lava_callback - INFO - Starting async job submit for node 69b017d3ae557f2600f10887 2026-03-12 13:08:54,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:08:54,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b017d3ae557f2600f10887 HTTP/1.1" 200 None 2026-03-12 13:08:54,470 - lava_callback - DEBUG - Processing node 69b017d3ae557f2600f10887: baseline-arm64-clabbe 2026-03-12 13:08:54,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b017d3ae557f2600f10887 2026-03-12 13:08:54,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:08:54,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b017d3ae557f2600f10887 HTTP/1.1" 200 None 2026-03-12 13:08:54,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:08:54,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-12 13:08:54,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:08:54,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b017d3ae557f2600f10887 HTTP/1.1" 200 None 2026-03-12 13:08:54,537 - lava_callback - ERROR - Error processing callback for node 69b017d3ae557f2600f10887: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48038 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:10:14,459 - lava_callback - INFO - Received callback for node 69b01828ae557f2600f10897 INFO: 172.20.0.1:35280 - "POST /node/69b01828ae557f2600f10897 HTTP/1.0" 202 Accepted 2026-03-12 13:10:14,465 - lava_callback - INFO - Starting async job submit for node 69b01828ae557f2600f10897 2026-03-12 13:10:14,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:10:14,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01828ae557f2600f10897 HTTP/1.1" 200 None 2026-03-12 13:10:14,481 - lava_callback - DEBUG - Processing node 69b01828ae557f2600f10897: baseline-arm64-clabbe 2026-03-12 13:10:14,481 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01828ae557f2600f10897 2026-03-12 13:10:14,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:10:14,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01828ae557f2600f10897 HTTP/1.1" 200 None 2026-03-12 13:10:14,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:10:14,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-12 13:10:14,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:10:14,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01828ae557f2600f10897 HTTP/1.1" 200 None 2026-03-12 13:10:14,569 - lava_callback - ERROR - Error processing callback for node 69b01828ae557f2600f10897: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36252 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:11:34,529 - lava_callback - INFO - Received callback for node 69b0187cae557f2600f108a3 INFO: 172.20.0.1:39310 - "POST /node/69b0187cae557f2600f108a3 HTTP/1.0" 202 Accepted 2026-03-12 13:11:34,534 - lava_callback - INFO - Starting async job submit for node 69b0187cae557f2600f108a3 2026-03-12 13:11:34,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:11:34,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0187cae557f2600f108a3 HTTP/1.1" 200 None 2026-03-12 13:11:34,550 - lava_callback - DEBUG - Processing node 69b0187cae557f2600f108a3: baseline-arm64-clabbe 2026-03-12 13:11:34,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0187cae557f2600f108a3 2026-03-12 13:11:34,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:11:34,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0187cae557f2600f108a3 HTTP/1.1" 200 None 2026-03-12 13:11:34,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:11:34,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-12 13:11:34,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:11:34,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0187cae557f2600f108a3 HTTP/1.1" 200 None 2026-03-12 13:11:34,622 - lava_callback - ERROR - Error processing callback for node 69b0187cae557f2600f108a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56474 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:12:54,598 - lava_callback - INFO - Received callback for node 69b018ceae557f2600f108b4 INFO: 172.20.0.1:43830 - "POST /node/69b018ceae557f2600f108b4 HTTP/1.0" 202 Accepted 2026-03-12 13:12:54,603 - lava_callback - INFO - Starting async job submit for node 69b018ceae557f2600f108b4 2026-03-12 13:12:54,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:12:54,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b018ceae557f2600f108b4 HTTP/1.1" 200 None 2026-03-12 13:12:54,629 - lava_callback - DEBUG - Processing node 69b018ceae557f2600f108b4: baseline-arm64-clabbe 2026-03-12 13:12:54,629 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b018ceae557f2600f108b4 2026-03-12 13:12:54,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:12:54,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b018ceae557f2600f108b4 HTTP/1.1" 200 None 2026-03-12 13:12:54,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:12:54,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-12 13:12:54,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:12:54,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b018ceae557f2600f108b4 HTTP/1.1" 200 None 2026-03-12 13:12:54,701 - lava_callback - ERROR - Error processing callback for node 69b018ceae557f2600f108b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45538 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:14:14,649 - lava_callback - INFO - Received callback for node 69b01922ae557f2600f108c5 INFO: 172.20.0.1:50650 - "POST /node/69b01922ae557f2600f108c5 HTTP/1.0" 202 Accepted 2026-03-12 13:14:14,654 - lava_callback - INFO - Starting async job submit for node 69b01922ae557f2600f108c5 2026-03-12 13:14:14,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:14:14,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01922ae557f2600f108c5 HTTP/1.1" 200 None 2026-03-12 13:14:14,670 - lava_callback - DEBUG - Processing node 69b01922ae557f2600f108c5: baseline-arm64-clabbe 2026-03-12 13:14:14,671 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01922ae557f2600f108c5 2026-03-12 13:14:14,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:14:14,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01922ae557f2600f108c5 HTTP/1.1" 200 None 2026-03-12 13:14:14,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:14:14,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69af6340275c8d3b79399dc3 HTTP/1.1" 200 None 2026-03-12 13:14:14,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:14:14,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01922ae557f2600f108c5 HTTP/1.1" 200 None 2026-03-12 13:14:14,738 - lava_callback - ERROR - Error processing callback for node 69b01922ae557f2600f108c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34026 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:15:34,700 - lava_callback - INFO - Received callback for node 69b01967ae557f2600f108d7 INFO: 172.20.0.1:55760 - "POST /node/69b01967ae557f2600f108d7 HTTP/1.0" 202 Accepted 2026-03-12 13:15:34,707 - lava_callback - INFO - Starting async job submit for node 69b01967ae557f2600f108d7 2026-03-12 13:15:34,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:15:34,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01967ae557f2600f108d7 HTTP/1.1" 200 None 2026-03-12 13:15:34,732 - lava_callback - DEBUG - Processing node 69b01967ae557f2600f108d7: baseline-arm-clabbe 2026-03-12 13:15:34,733 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01967ae557f2600f108d7 2026-03-12 13:15:34,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:15:34,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01967ae557f2600f108d7 HTTP/1.1" 200 None 2026-03-12 13:15:34,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:15:34,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-12 13:15:34,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:15:34,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01967ae557f2600f108d7 HTTP/1.1" 200 None 2026-03-12 13:15:34,791 - lava_callback - ERROR - Error processing callback for node 69b01967ae557f2600f108d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48524 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:16:14,676 - lava_callback - INFO - Received callback for node 69b0198dae557f2600f108e3 INFO: 172.20.0.1:36304 - "POST /node/69b0198dae557f2600f108e3 HTTP/1.0" 202 Accepted 2026-03-12 13:16:14,681 - lava_callback - INFO - Starting async job submit for node 69b0198dae557f2600f108e3 2026-03-12 13:16:14,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:16:14,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0198dae557f2600f108e3 HTTP/1.1" 200 None 2026-03-12 13:16:14,720 - lava_callback - DEBUG - Processing node 69b0198dae557f2600f108e3: baseline-arm-clabbe 2026-03-12 13:16:14,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0198dae557f2600f108e3 2026-03-12 13:16:14,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:16:14,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0198dae557f2600f108e3 HTTP/1.1" 200 None 2026-03-12 13:16:14,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:16:14,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-12 13:16:14,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:16:14,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0198dae557f2600f108e3 HTTP/1.1" 200 None 2026-03-12 13:16:14,789 - lava_callback - ERROR - Error processing callback for node 69b0198dae557f2600f108e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40894 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:16:54,750 - lava_callback - INFO - Received callback for node 69b019b3ae557f2600f108f0 INFO: 172.20.0.1:55010 - "POST /node/69b019b3ae557f2600f108f0 HTTP/1.0" 202 Accepted 2026-03-12 13:16:54,756 - lava_callback - INFO - Starting async job submit for node 69b019b3ae557f2600f108f0 2026-03-12 13:16:54,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:16:54,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b019b3ae557f2600f108f0 HTTP/1.1" 200 None 2026-03-12 13:16:54,773 - lava_callback - DEBUG - Processing node 69b019b3ae557f2600f108f0: baseline-arm-clabbe 2026-03-12 13:16:54,774 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b019b3ae557f2600f108f0 2026-03-12 13:16:54,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:16:54,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b019b3ae557f2600f108f0 HTTP/1.1" 200 None 2026-03-12 13:16:54,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:16:54,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-12 13:16:54,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:16:54,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b019b3ae557f2600f108f0 HTTP/1.1" 200 None 2026-03-12 13:16:54,832 - lava_callback - ERROR - Error processing callback for node 69b019b3ae557f2600f108f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49658 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:17:34,740 - lava_callback - INFO - Received callback for node 69b019d9ae557f2600f108f9 INFO: 172.20.0.1:51894 - "POST /node/69b019d9ae557f2600f108f9 HTTP/1.0" 202 Accepted 2026-03-12 13:17:34,745 - lava_callback - INFO - Starting async job submit for node 69b019d9ae557f2600f108f9 2026-03-12 13:17:34,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:17:34,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b019d9ae557f2600f108f9 HTTP/1.1" 200 None 2026-03-12 13:17:34,761 - lava_callback - DEBUG - Processing node 69b019d9ae557f2600f108f9: baseline-arm-clabbe 2026-03-12 13:17:34,761 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b019d9ae557f2600f108f9 2026-03-12 13:17:34,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:17:34,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b019d9ae557f2600f108f9 HTTP/1.1" 200 None 2026-03-12 13:17:34,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:17:34,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-12 13:17:34,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:17:34,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b019d9ae557f2600f108f9 HTTP/1.1" 200 None 2026-03-12 13:17:34,815 - lava_callback - ERROR - Error processing callback for node 69b019d9ae557f2600f108f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39300 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:17:54,800 - lava_callback - INFO - Received callback for node 69b019ffae557f2600f10904 INFO: 172.20.0.1:54672 - "POST /node/69b019ffae557f2600f10904 HTTP/1.0" 202 Accepted 2026-03-12 13:17:54,805 - lava_callback - INFO - Starting async job submit for node 69b019ffae557f2600f10904 2026-03-12 13:17:54,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:17:54,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b019ffae557f2600f10904 HTTP/1.1" 200 None 2026-03-12 13:17:54,825 - lava_callback - DEBUG - Processing node 69b019ffae557f2600f10904: baseline-arm-clabbe 2026-03-12 13:17:54,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b019ffae557f2600f10904 2026-03-12 13:17:54,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:17:54,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b019ffae557f2600f10904 HTTP/1.1" 200 None 2026-03-12 13:17:54,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:17:54,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-12 13:17:54,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:17:54,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b019ffae557f2600f10904 HTTP/1.1" 200 None 2026-03-12 13:17:54,913 - lava_callback - ERROR - Error processing callback for node 69b019ffae557f2600f10904: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33180 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:18:34,805 - lava_callback - INFO - Received callback for node 69b01a25ae557f2600f10912 INFO: 172.20.0.1:45118 - "POST /node/69b01a25ae557f2600f10912 HTTP/1.0" 202 Accepted 2026-03-12 13:18:34,810 - lava_callback - INFO - Starting async job submit for node 69b01a25ae557f2600f10912 2026-03-12 13:18:34,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:18:34,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01a25ae557f2600f10912 HTTP/1.1" 200 None 2026-03-12 13:18:34,826 - lava_callback - DEBUG - Processing node 69b01a25ae557f2600f10912: baseline-arm-clabbe 2026-03-12 13:18:34,826 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01a25ae557f2600f10912 2026-03-12 13:18:34,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:18:34,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01a25ae557f2600f10912 HTTP/1.1" 200 None 2026-03-12 13:18:34,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:18:34,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc009bcd207e915d76b94 HTTP/1.1" 200 None 2026-03-12 13:18:34,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:18:34,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01a25ae557f2600f10912 HTTP/1.1" 200 None 2026-03-12 13:18:34,886 - lava_callback - ERROR - Error processing callback for node 69b01a25ae557f2600f10912: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36336 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:26:48,687 - lava_callback - INFO - Received callback for node 69b01c09ae557f2600f10935 INFO: 172.20.0.1:36344 - "POST /node/69b01c09ae557f2600f10935 HTTP/1.0" 202 Accepted 2026-03-12 13:26:48,693 - lava_callback - INFO - Starting async job submit for node 69b01c09ae557f2600f10935 2026-03-12 13:26:48,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:26:48,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01c09ae557f2600f10935 HTTP/1.1" 200 None 2026-03-12 13:26:48,710 - lava_callback - DEBUG - Processing node 69b01c09ae557f2600f10935: baseline-arm64-baylibre 2026-03-12 13:26:48,710 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01c09ae557f2600f10935 2026-03-12 13:26:48,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:26:48,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01c09ae557f2600f10935 HTTP/1.1" 200 None 2026-03-12 13:26:48,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:26:48,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 13:26:48,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:26:48,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01c09ae557f2600f10935 HTTP/1.1" 200 None 2026-03-12 13:26:48,773 - lava_callback - ERROR - Error processing callback for node 69b01c09ae557f2600f10935: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58892 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:28:48,943 - lava_callback - INFO - Received callback for node 69b01c7cae557f2600f10942 INFO: 172.20.0.1:58908 - "POST /node/69b01c7cae557f2600f10942 HTTP/1.0" 202 Accepted 2026-03-12 13:28:48,948 - lava_callback - INFO - Starting async job submit for node 69b01c7cae557f2600f10942 2026-03-12 13:28:48,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:28:48,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01c7cae557f2600f10942 HTTP/1.1" 200 None 2026-03-12 13:28:48,966 - lava_callback - DEBUG - Processing node 69b01c7cae557f2600f10942: baseline-arm64-baylibre 2026-03-12 13:28:48,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01c7cae557f2600f10942 2026-03-12 13:28:48,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:28:48,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01c7cae557f2600f10942 HTTP/1.1" 200 None 2026-03-12 13:28:48,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:28:48,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 13:28:48,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:28:49,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01c7cae557f2600f10942 HTTP/1.1" 200 None 2026-03-12 13:28:49,029 - lava_callback - ERROR - Error processing callback for node 69b01c7cae557f2600f10942: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42890 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:30:08,846 - lava_callback - INFO - Received callback for node 69b01cd6ae557f2600f1094f INFO: 172.20.0.1:42904 - "POST /node/69b01cd6ae557f2600f1094f HTTP/1.0" 202 Accepted 2026-03-12 13:30:08,851 - lava_callback - INFO - Starting async job submit for node 69b01cd6ae557f2600f1094f 2026-03-12 13:30:08,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:30:08,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01cd6ae557f2600f1094f HTTP/1.1" 200 None 2026-03-12 13:30:08,871 - lava_callback - DEBUG - Processing node 69b01cd6ae557f2600f1094f: baseline-arm64-baylibre 2026-03-12 13:30:08,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01cd6ae557f2600f1094f 2026-03-12 13:30:08,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:30:08,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01cd6ae557f2600f1094f HTTP/1.1" 200 None 2026-03-12 13:30:08,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:30:08,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 13:30:08,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:30:08,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01cd6ae557f2600f1094f HTTP/1.1" 200 None 2026-03-12 13:30:08,963 - lava_callback - ERROR - Error processing callback for node 69b01cd6ae557f2600f1094f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37458 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:33:09,048 - lava_callback - INFO - Received callback for node 69b01d80ae557f2600f1097d INFO: 172.20.0.1:37474 - "POST /node/69b01d80ae557f2600f1097d HTTP/1.0" 202 Accepted 2026-03-12 13:33:09,053 - lava_callback - INFO - Starting async job submit for node 69b01d80ae557f2600f1097d 2026-03-12 13:33:09,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:33:09,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01d80ae557f2600f1097d HTTP/1.1" 200 None 2026-03-12 13:33:09,070 - lava_callback - DEBUG - Processing node 69b01d80ae557f2600f1097d: baseline-arm64-baylibre 2026-03-12 13:33:09,070 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01d80ae557f2600f1097d 2026-03-12 13:33:09,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:33:09,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01d80ae557f2600f1097d HTTP/1.1" 200 None 2026-03-12 13:33:09,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:33:09,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 13:33:09,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:33:09,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01d80ae557f2600f1097d HTTP/1.1" 200 None 2026-03-12 13:33:09,137 - lava_callback - ERROR - Error processing callback for node 69b01d80ae557f2600f1097d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37018 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:37:35,670 - lava_callback - INFO - Received callback for node 69b01e91ae557f2600f109a5 INFO: 172.20.0.1:59596 - "POST /node/69b01e91ae557f2600f109a5 HTTP/1.0" 202 Accepted 2026-03-12 13:37:35,677 - lava_callback - INFO - Starting async job submit for node 69b01e91ae557f2600f109a5 2026-03-12 13:37:35,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:37:35,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01e91ae557f2600f109a5 HTTP/1.1" 200 None 2026-03-12 13:37:35,693 - lava_callback - DEBUG - Processing node 69b01e91ae557f2600f109a5: baseline-arm64-clabbe 2026-03-12 13:37:35,693 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01e91ae557f2600f109a5 2026-03-12 13:37:35,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:37:35,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01e91ae557f2600f109a5 HTTP/1.1" 200 None 2026-03-12 13:37:35,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:37:35,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 13:37:35,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:37:35,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01e91ae557f2600f109a5 HTTP/1.1" 200 None 2026-03-12 13:37:35,817 - lava_callback - ERROR - Error processing callback for node 69b01e91ae557f2600f109a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36958 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 13:40:15,776 - lava_callback - INFO - Received callback for node 69b01f3aae557f2600f109bb INFO: 172.20.0.1:36970 - "POST /node/69b01f3aae557f2600f109bb HTTP/1.0" 202 Accepted 2026-03-12 13:40:15,782 - lava_callback - INFO - Starting async job submit for node 69b01f3aae557f2600f109bb 2026-03-12 13:40:15,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:40:15,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01f3aae557f2600f109bb HTTP/1.1" 200 None 2026-03-12 13:40:15,801 - lava_callback - DEBUG - Processing node 69b01f3aae557f2600f109bb: baseline-arm64-clabbe 2026-03-12 13:40:15,801 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01f3aae557f2600f109bb 2026-03-12 13:40:15,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:40:15,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01f3aae557f2600f109bb HTTP/1.1" 200 None 2026-03-12 13:40:15,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:40:15,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 13:40:15,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:40:15,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01f3aae557f2600f109bb HTTP/1.1" 200 None 2026-03-12 13:40:15,860 - lava_callback - ERROR - Error processing callback for node 69b01f3aae557f2600f109bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52806 - "GET / HTTP/1.0" 200 OK 2026-03-12 13:41:55,878 - lava_callback - INFO - Received callback for node 69b01f91ae557f2600f109c8 INFO: 172.20.0.1:33574 - "POST /node/69b01f91ae557f2600f109c8 HTTP/1.0" 202 Accepted 2026-03-12 13:41:55,884 - lava_callback - INFO - Starting async job submit for node 69b01f91ae557f2600f109c8 2026-03-12 13:41:55,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:41:55,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01f91ae557f2600f109c8 HTTP/1.1" 200 None 2026-03-12 13:41:55,906 - lava_callback - DEBUG - Processing node 69b01f91ae557f2600f109c8: baseline-arm64-clabbe 2026-03-12 13:41:55,906 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01f91ae557f2600f109c8 2026-03-12 13:41:55,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:41:55,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01f91ae557f2600f109c8 HTTP/1.1" 200 None 2026-03-12 13:41:55,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:41:55,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 13:41:55,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:41:55,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01f91ae557f2600f109c8 HTTP/1.1" 200 None 2026-03-12 13:41:55,971 - lava_callback - ERROR - Error processing callback for node 69b01f91ae557f2600f109c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53262 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 13:43:15,944 - lava_callback - INFO - Received callback for node 69b01fe6ae557f2600f109d8 INFO: 172.20.0.1:53264 - "POST /node/69b01fe6ae557f2600f109d8 HTTP/1.0" 202 Accepted 2026-03-12 13:43:15,949 - lava_callback - INFO - Starting async job submit for node 69b01fe6ae557f2600f109d8 2026-03-12 13:43:15,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:43:15,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01fe6ae557f2600f109d8 HTTP/1.1" 200 None 2026-03-12 13:43:15,966 - lava_callback - DEBUG - Processing node 69b01fe6ae557f2600f109d8: baseline-arm64-clabbe 2026-03-12 13:43:15,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b01fe6ae557f2600f109d8 2026-03-12 13:43:15,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:43:15,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b01fe6ae557f2600f109d8 HTTP/1.1" 200 None 2026-03-12 13:43:15,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:43:15,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 13:43:15,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 13:43:16,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b01fe6ae557f2600f109d8 HTTP/1.1" 200 None 2026-03-12 13:43:16,020 - lava_callback - ERROR - Error processing callback for node 69b01fe6ae557f2600f109d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54286 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:30:38,008 - lava_callback - INFO - Received callback for node 69b02b04ae557f2600f10a41 INFO: 172.20.0.1:54292 - "POST /node/69b02b04ae557f2600f10a41 HTTP/1.0" 202 Accepted 2026-03-12 14:30:38,013 - lava_callback - INFO - Starting async job submit for node 69b02b04ae557f2600f10a41 2026-03-12 14:30:38,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:30:38,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02b04ae557f2600f10a41 HTTP/1.1" 200 None 2026-03-12 14:30:38,046 - lava_callback - DEBUG - Processing node 69b02b04ae557f2600f10a41: baseline-arm64-clabbe 2026-03-12 14:30:38,046 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02b04ae557f2600f10a41 2026-03-12 14:30:38,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:30:38,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02b04ae557f2600f10a41 HTTP/1.1" 200 None 2026-03-12 14:30:38,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:30:38,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 14:30:38,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:30:38,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02b04ae557f2600f10a41 HTTP/1.1" 200 None 2026-03-12 14:30:38,149 - lava_callback - ERROR - Error processing callback for node 69b02b04ae557f2600f10a41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39042 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:31:58,015 - lava_callback - INFO - Received callback for node 69b02b59ae557f2600f10a49 INFO: 172.20.0.1:39048 - "POST /node/69b02b59ae557f2600f10a49 HTTP/1.0" 202 Accepted 2026-03-12 14:31:58,020 - lava_callback - INFO - Starting async job submit for node 69b02b59ae557f2600f10a49 2026-03-12 14:31:58,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:31:58,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02b59ae557f2600f10a49 HTTP/1.1" 200 None 2026-03-12 14:31:58,036 - lava_callback - DEBUG - Processing node 69b02b59ae557f2600f10a49: baseline-arm64-clabbe 2026-03-12 14:31:58,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02b59ae557f2600f10a49 2026-03-12 14:31:58,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:31:58,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02b59ae557f2600f10a49 HTTP/1.1" 200 None 2026-03-12 14:31:58,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:31:58,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69acc01dbcd207e915d76ba7 HTTP/1.1" 200 None 2026-03-12 14:31:58,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:31:58,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02b59ae557f2600f10a49 HTTP/1.1" 200 None 2026-03-12 14:31:58,133 - lava_callback - ERROR - Error processing callback for node 69b02b59ae557f2600f10a49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51962 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:40:32,745 - lava_callback - INFO - Received callback for node 69b02d58ae557f2600f10a5d INFO: 172.20.0.1:54258 - "POST /node/69b02d58ae557f2600f10a5d HTTP/1.0" 202 Accepted 2026-03-12 14:40:32,750 - lava_callback - INFO - Starting async job submit for node 69b02d58ae557f2600f10a5d 2026-03-12 14:40:32,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:40:32,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02d58ae557f2600f10a5d HTTP/1.1" 200 None 2026-03-12 14:40:32,767 - lava_callback - DEBUG - Processing node 69b02d58ae557f2600f10a5d: baseline-arm-baylibre 2026-03-12 14:40:32,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02d58ae557f2600f10a5d 2026-03-12 14:40:32,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:40:32,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02d58ae557f2600f10a5d HTTP/1.1" 200 None 2026-03-12 14:40:32,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:40:32,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:40:32,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:40:32,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02d58ae557f2600f10a5d HTTP/1.1" 200 None 2026-03-12 14:40:32,863 - lava_callback - ERROR - Error processing callback for node 69b02d58ae557f2600f10a5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51940 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:41:32,776 - lava_callback - INFO - Received callback for node 69b02d8fae557f2600f10a69 INFO: 172.20.0.1:36016 - "POST /node/69b02d8fae557f2600f10a69 HTTP/1.0" 202 Accepted 2026-03-12 14:41:32,782 - lava_callback - INFO - Starting async job submit for node 69b02d8fae557f2600f10a69 2026-03-12 14:41:32,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:41:32,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02d8fae557f2600f10a69 HTTP/1.1" 200 None 2026-03-12 14:41:32,811 - lava_callback - DEBUG - Processing node 69b02d8fae557f2600f10a69: baseline-arm-baylibre 2026-03-12 14:41:32,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02d8fae557f2600f10a69 2026-03-12 14:41:32,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:41:32,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02d8fae557f2600f10a69 HTTP/1.1" 200 None 2026-03-12 14:41:32,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:41:32,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:41:32,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:41:32,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02d8fae557f2600f10a69 HTTP/1.1" 200 None 2026-03-12 14:41:32,896 - lava_callback - ERROR - Error processing callback for node 69b02d8fae557f2600f10a69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41580 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:42:12,808 - lava_callback - INFO - Received callback for node 69b02db5ae557f2600f10a79 INFO: 172.20.0.1:48018 - "POST /node/69b02db5ae557f2600f10a79 HTTP/1.0" 202 Accepted 2026-03-12 14:42:12,814 - lava_callback - INFO - Starting async job submit for node 69b02db5ae557f2600f10a79 2026-03-12 14:42:12,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:42:12,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02db5ae557f2600f10a79 HTTP/1.1" 200 None 2026-03-12 14:42:12,830 - lava_callback - DEBUG - Processing node 69b02db5ae557f2600f10a79: baseline-arm-baylibre 2026-03-12 14:42:12,830 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02db5ae557f2600f10a79 2026-03-12 14:42:12,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:42:12,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02db5ae557f2600f10a79 HTTP/1.1" 200 None 2026-03-12 14:42:12,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:42:12,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:42:12,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:42:12,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02db5ae557f2600f10a79 HTTP/1.1" 200 None 2026-03-12 14:42:12,905 - lava_callback - ERROR - Error processing callback for node 69b02db5ae557f2600f10a79: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57002 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:42:52,820 - lava_callback - INFO - Received callback for node 69b02ddcae557f2600f10a86 INFO: 172.20.0.1:60046 - "POST /node/69b02ddcae557f2600f10a86 HTTP/1.0" 202 Accepted 2026-03-12 14:42:52,825 - lava_callback - INFO - Starting async job submit for node 69b02ddcae557f2600f10a86 2026-03-12 14:42:52,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:42:52,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02ddcae557f2600f10a86 HTTP/1.1" 200 None 2026-03-12 14:42:52,842 - lava_callback - DEBUG - Processing node 69b02ddcae557f2600f10a86: baseline-arm-baylibre 2026-03-12 14:42:52,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02ddcae557f2600f10a86 2026-03-12 14:42:52,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:42:52,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02ddcae557f2600f10a86 HTTP/1.1" 200 None 2026-03-12 14:42:52,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:42:52,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:42:52,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:42:52,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02ddcae557f2600f10a86 HTTP/1.1" 200 None 2026-03-12 14:42:52,920 - lava_callback - ERROR - Error processing callback for node 69b02ddcae557f2600f10a86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33264 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:43:32,881 - lava_callback - INFO - Received callback for node 69b02e03ae557f2600f10a91 INFO: 172.20.0.1:57956 - "POST /node/69b02e03ae557f2600f10a91 HTTP/1.0" 202 Accepted 2026-03-12 14:43:32,887 - lava_callback - INFO - Starting async job submit for node 69b02e03ae557f2600f10a91 2026-03-12 14:43:32,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:43:32,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02e03ae557f2600f10a91 HTTP/1.1" 200 None 2026-03-12 14:43:32,904 - lava_callback - DEBUG - Processing node 69b02e03ae557f2600f10a91: baseline-arm-baylibre 2026-03-12 14:43:32,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02e03ae557f2600f10a91 2026-03-12 14:43:32,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:43:32,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02e03ae557f2600f10a91 HTTP/1.1" 200 None 2026-03-12 14:43:32,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:43:32,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:43:32,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:43:32,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02e03ae557f2600f10a91 HTTP/1.1" 200 None 2026-03-12 14:43:32,970 - lava_callback - ERROR - Error processing callback for node 69b02e03ae557f2600f10a91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40326 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:44:13,299 - lava_callback - INFO - Received callback for node 69b02e29ae557f2600f10aa1 INFO: 172.20.0.1:57244 - "POST /node/69b02e29ae557f2600f10aa1 HTTP/1.0" 202 Accepted 2026-03-12 14:44:13,305 - lava_callback - INFO - Starting async job submit for node 69b02e29ae557f2600f10aa1 2026-03-12 14:44:13,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:44:13,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02e29ae557f2600f10aa1 HTTP/1.1" 200 None 2026-03-12 14:44:13,347 - lava_callback - DEBUG - Processing node 69b02e29ae557f2600f10aa1: baseline-arm-baylibre 2026-03-12 14:44:13,347 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02e29ae557f2600f10aa1 2026-03-12 14:44:13,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:44:13,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02e29ae557f2600f10aa1 HTTP/1.1" 200 None 2026-03-12 14:44:13,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:44:13,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:44:13,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:44:13,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02e29ae557f2600f10aa1 HTTP/1.1" 200 None 2026-03-12 14:44:13,408 - lava_callback - ERROR - Error processing callback for node 69b02e29ae557f2600f10aa1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41416 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:44:52,917 - lava_callback - INFO - Received callback for node 69b02e50ae557f2600f10ab0 INFO: 172.20.0.1:60954 - "POST /node/69b02e50ae557f2600f10ab0 HTTP/1.0" 202 Accepted 2026-03-12 14:44:52,922 - lava_callback - INFO - Starting async job submit for node 69b02e50ae557f2600f10ab0 2026-03-12 14:44:52,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:44:52,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02e50ae557f2600f10ab0 HTTP/1.1" 200 None 2026-03-12 14:44:52,949 - lava_callback - DEBUG - Processing node 69b02e50ae557f2600f10ab0: baseline-arm-baylibre 2026-03-12 14:44:52,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02e50ae557f2600f10ab0 2026-03-12 14:44:52,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:44:52,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02e50ae557f2600f10ab0 HTTP/1.1" 200 None 2026-03-12 14:44:52,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:44:53,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:44:53,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:44:53,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02e50ae557f2600f10ab0 HTTP/1.1" 200 None 2026-03-12 14:44:53,041 - lava_callback - ERROR - Error processing callback for node 69b02e50ae557f2600f10ab0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55758 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:45:33,089 - lava_callback - INFO - Received callback for node 69b02e77ae557f2600f10abd INFO: 172.20.0.1:52144 - "POST /node/69b02e77ae557f2600f10abd HTTP/1.0" 202 Accepted 2026-03-12 14:45:33,094 - lava_callback - INFO - Starting async job submit for node 69b02e77ae557f2600f10abd 2026-03-12 14:45:33,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:45:33,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02e77ae557f2600f10abd HTTP/1.1" 200 None 2026-03-12 14:45:33,110 - lava_callback - DEBUG - Processing node 69b02e77ae557f2600f10abd: baseline-arm-baylibre 2026-03-12 14:45:33,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02e77ae557f2600f10abd 2026-03-12 14:45:33,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:45:33,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02e77ae557f2600f10abd HTTP/1.1" 200 None 2026-03-12 14:45:33,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:45:33,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:45:33,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:45:33,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02e77ae557f2600f10abd HTTP/1.1" 200 None 2026-03-12 14:45:33,386 - lava_callback - ERROR - Error processing callback for node 69b02e77ae557f2600f10abd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56242 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:46:13,013 - lava_callback - INFO - Received callback for node 69b02ea0ae557f2600f10acb INFO: 172.20.0.1:37452 - "POST /node/69b02ea0ae557f2600f10acb HTTP/1.0" 202 Accepted 2026-03-12 14:46:13,018 - lava_callback - INFO - Starting async job submit for node 69b02ea0ae557f2600f10acb 2026-03-12 14:46:13,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:46:13,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02ea0ae557f2600f10acb HTTP/1.1" 200 None 2026-03-12 14:46:13,034 - lava_callback - DEBUG - Processing node 69b02ea0ae557f2600f10acb: baseline-arm-baylibre 2026-03-12 14:46:13,034 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02ea0ae557f2600f10acb 2026-03-12 14:46:13,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:46:13,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02ea0ae557f2600f10acb HTTP/1.1" 200 None 2026-03-12 14:46:13,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:46:13,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:46:13,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:46:13,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02ea0ae557f2600f10acb HTTP/1.1" 200 None 2026-03-12 14:46:13,091 - lava_callback - ERROR - Error processing callback for node 69b02ea0ae557f2600f10acb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40396 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:46:53,075 - lava_callback - INFO - Received callback for node 69b02ec8ae557f2600f10ad7 INFO: 172.20.0.1:33842 - "POST /node/69b02ec8ae557f2600f10ad7 HTTP/1.0" 202 Accepted 2026-03-12 14:46:53,081 - lava_callback - INFO - Starting async job submit for node 69b02ec8ae557f2600f10ad7 2026-03-12 14:46:53,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:46:53,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02ec8ae557f2600f10ad7 HTTP/1.1" 200 None 2026-03-12 14:46:53,096 - lava_callback - DEBUG - Processing node 69b02ec8ae557f2600f10ad7: baseline-arm-baylibre 2026-03-12 14:46:53,096 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02ec8ae557f2600f10ad7 2026-03-12 14:46:53,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:46:53,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02ec8ae557f2600f10ad7 HTTP/1.1" 200 None 2026-03-12 14:46:53,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:46:53,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:46:53,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:46:53,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02ec8ae557f2600f10ad7 HTTP/1.1" 200 None 2026-03-12 14:46:53,185 - lava_callback - ERROR - Error processing callback for node 69b02ec8ae557f2600f10ad7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55244 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:47:18,750 - lava_callback - INFO - Received callback for node 69b02ef1ae557f2600f10ae1 INFO: 172.20.0.1:55246 - "POST /node/69b02ef1ae557f2600f10ae1 HTTP/1.0" 202 Accepted 2026-03-12 14:47:18,755 - lava_callback - INFO - Starting async job submit for node 69b02ef1ae557f2600f10ae1 2026-03-12 14:47:18,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:47:18,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02ef1ae557f2600f10ae1 HTTP/1.1" 200 None 2026-03-12 14:47:18,771 - lava_callback - DEBUG - Processing node 69b02ef1ae557f2600f10ae1: baseline-arm-clabbe 2026-03-12 14:47:18,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02ef1ae557f2600f10ae1 2026-03-12 14:47:18,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:47:18,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02ef1ae557f2600f10ae1 HTTP/1.1" 200 None 2026-03-12 14:47:18,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:47:18,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:47:18,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:47:18,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02ef1ae557f2600f10ae1 HTTP/1.1" 200 None 2026-03-12 14:47:18,833 - lava_callback - ERROR - Error processing callback for node 69b02ef1ae557f2600f10ae1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50912 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:47:58,728 - lava_callback - INFO - Received callback for node 69b02f1aae557f2600f10af6 INFO: 172.20.0.1:50914 - "POST /node/69b02f1aae557f2600f10af6 HTTP/1.0" 202 Accepted 2026-03-12 14:47:58,733 - lava_callback - INFO - Starting async job submit for node 69b02f1aae557f2600f10af6 2026-03-12 14:47:58,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:47:58,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02f1aae557f2600f10af6 HTTP/1.1" 200 None 2026-03-12 14:47:58,752 - lava_callback - DEBUG - Processing node 69b02f1aae557f2600f10af6: baseline-arm-clabbe 2026-03-12 14:47:58,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02f1aae557f2600f10af6 2026-03-12 14:47:58,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:47:58,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02f1aae557f2600f10af6 HTTP/1.1" 200 None 2026-03-12 14:47:58,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:47:58,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:47:58,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:47:58,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02f1aae557f2600f10af6 HTTP/1.1" 200 None 2026-03-12 14:47:58,820 - lava_callback - ERROR - Error processing callback for node 69b02f1aae557f2600f10af6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51532 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:48:38,817 - lava_callback - INFO - Received callback for node 69b02f42ae557f2600f10b19 INFO: 172.20.0.1:51534 - "POST /node/69b02f42ae557f2600f10b19 HTTP/1.0" 202 Accepted 2026-03-12 14:48:38,822 - lava_callback - INFO - Starting async job submit for node 69b02f42ae557f2600f10b19 2026-03-12 14:48:38,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:48:38,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02f42ae557f2600f10b19 HTTP/1.1" 200 None 2026-03-12 14:48:38,838 - lava_callback - DEBUG - Processing node 69b02f42ae557f2600f10b19: baseline-arm-clabbe 2026-03-12 14:48:38,839 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02f42ae557f2600f10b19 2026-03-12 14:48:38,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:48:38,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02f42ae557f2600f10b19 HTTP/1.1" 200 None 2026-03-12 14:48:38,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:48:38,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:48:38,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:48:38,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02f42ae557f2600f10b19 HTTP/1.1" 200 None 2026-03-12 14:48:38,893 - lava_callback - ERROR - Error processing callback for node 69b02f42ae557f2600f10b19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38720 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:49:18,792 - lava_callback - INFO - Received callback for node 69b02f6aae557f2600f10b2e INFO: 172.20.0.1:38726 - "POST /node/69b02f6aae557f2600f10b2e HTTP/1.0" 202 Accepted 2026-03-12 14:49:18,799 - lava_callback - INFO - Starting async job submit for node 69b02f6aae557f2600f10b2e 2026-03-12 14:49:18,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:49:18,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02f6aae557f2600f10b2e HTTP/1.1" 200 None 2026-03-12 14:49:18,821 - lava_callback - DEBUG - Processing node 69b02f6aae557f2600f10b2e: baseline-arm-clabbe 2026-03-12 14:49:18,821 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02f6aae557f2600f10b2e 2026-03-12 14:49:18,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:49:18,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02f6aae557f2600f10b2e HTTP/1.1" 200 None 2026-03-12 14:49:18,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:49:18,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:49:18,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:49:18,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02f6aae557f2600f10b2e HTTP/1.1" 200 None 2026-03-12 14:49:18,892 - lava_callback - ERROR - Error processing callback for node 69b02f6aae557f2600f10b2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57188 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:49:58,898 - lava_callback - INFO - Received callback for node 69b02f93ae557f2600f10b3a INFO: 172.20.0.1:57204 - "POST /node/69b02f93ae557f2600f10b3a HTTP/1.0" 202 Accepted 2026-03-12 14:49:58,903 - lava_callback - INFO - Starting async job submit for node 69b02f93ae557f2600f10b3a 2026-03-12 14:49:58,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:49:58,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02f93ae557f2600f10b3a HTTP/1.1" 200 None 2026-03-12 14:49:58,919 - lava_callback - DEBUG - Processing node 69b02f93ae557f2600f10b3a: baseline-arm-clabbe 2026-03-12 14:49:58,919 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02f93ae557f2600f10b3a 2026-03-12 14:49:58,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:49:58,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02f93ae557f2600f10b3a HTTP/1.1" 200 None 2026-03-12 14:49:58,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:49:58,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:49:58,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:49:58,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02f93ae557f2600f10b3a HTTP/1.1" 200 None 2026-03-12 14:49:58,980 - lava_callback - ERROR - Error processing callback for node 69b02f93ae557f2600f10b3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33220 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:50:38,874 - lava_callback - INFO - Received callback for node 69b02fbbae557f2600f10b5a INFO: 172.20.0.1:33226 - "POST /node/69b02fbbae557f2600f10b5a HTTP/1.0" 202 Accepted 2026-03-12 14:50:38,883 - lava_callback - INFO - Starting async job submit for node 69b02fbbae557f2600f10b5a 2026-03-12 14:50:38,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:50:38,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02fbbae557f2600f10b5a HTTP/1.1" 200 None 2026-03-12 14:50:38,911 - lava_callback - DEBUG - Processing node 69b02fbbae557f2600f10b5a: baseline-arm-clabbe 2026-03-12 14:50:38,911 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02fbbae557f2600f10b5a 2026-03-12 14:50:38,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:50:38,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02fbbae557f2600f10b5a HTTP/1.1" 200 None 2026-03-12 14:50:38,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:50:39,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:50:39,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:50:39,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02fbbae557f2600f10b5a HTTP/1.1" 200 None 2026-03-12 14:50:39,087 - lava_callback - ERROR - Error processing callback for node 69b02fbbae557f2600f10b5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34964 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:51:18,934 - lava_callback - INFO - Received callback for node 69b02fe1ae557f2600f10b6d INFO: 172.20.0.1:34968 - "POST /node/69b02fe1ae557f2600f10b6d HTTP/1.0" 202 Accepted 2026-03-12 14:51:18,944 - lava_callback - INFO - Starting async job submit for node 69b02fe1ae557f2600f10b6d 2026-03-12 14:51:18,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:51:18,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02fe1ae557f2600f10b6d HTTP/1.1" 200 None 2026-03-12 14:51:18,971 - lava_callback - DEBUG - Processing node 69b02fe1ae557f2600f10b6d: baseline-arm-clabbe 2026-03-12 14:51:18,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b02fe1ae557f2600f10b6d 2026-03-12 14:51:18,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:51:18,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b02fe1ae557f2600f10b6d HTTP/1.1" 200 None 2026-03-12 14:51:18,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:51:19,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:51:19,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:51:19,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b02fe1ae557f2600f10b6d HTTP/1.1" 200 None 2026-03-12 14:51:19,075 - lava_callback - ERROR - Error processing callback for node 69b02fe1ae557f2600f10b6d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47604 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:51:58,895 - lava_callback - INFO - Received callback for node 69b03009ae557f2600f10b85 INFO: 172.20.0.1:47612 - "POST /node/69b03009ae557f2600f10b85 HTTP/1.0" 202 Accepted 2026-03-12 14:51:58,901 - lava_callback - INFO - Starting async job submit for node 69b03009ae557f2600f10b85 2026-03-12 14:51:58,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:51:58,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03009ae557f2600f10b85 HTTP/1.1" 200 None 2026-03-12 14:51:58,917 - lava_callback - DEBUG - Processing node 69b03009ae557f2600f10b85: baseline-arm-clabbe 2026-03-12 14:51:58,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03009ae557f2600f10b85 2026-03-12 14:51:58,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:51:58,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03009ae557f2600f10b85 HTTP/1.1" 200 None 2026-03-12 14:51:58,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:51:58,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 14:51:58,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:51:58,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03009ae557f2600f10b85 HTTP/1.1" 200 None 2026-03-12 14:51:58,983 - lava_callback - ERROR - Error processing callback for node 69b03009ae557f2600f10b85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50148 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:52:53,401 - lava_callback - INFO - Received callback for node 69b03037ae557f2600f10b93 INFO: 172.20.0.1:43162 - "POST /node/69b03037ae557f2600f10b93 HTTP/1.0" 202 Accepted 2026-03-12 14:52:53,407 - lava_callback - INFO - Starting async job submit for node 69b03037ae557f2600f10b93 2026-03-12 14:52:53,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:52:53,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03037ae557f2600f10b93 HTTP/1.1" 200 None 2026-03-12 14:52:53,441 - lava_callback - DEBUG - Processing node 69b03037ae557f2600f10b93: baseline-arm64-baylibre 2026-03-12 14:52:53,441 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03037ae557f2600f10b93 2026-03-12 14:52:53,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:52:53,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03037ae557f2600f10b93 HTTP/1.1" 200 None 2026-03-12 14:52:53,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:52:53,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 14:52:53,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:52:53,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03037ae557f2600f10b93 HTTP/1.1" 200 None 2026-03-12 14:52:53,502 - lava_callback - ERROR - Error processing callback for node 69b03037ae557f2600f10b93: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38360 - "GET /node/69a767105a0c0e10c4822228%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52462 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:58:33,709 - lava_callback - INFO - Received callback for node 69b03189ae557f2600f10bce INFO: 172.20.0.1:57830 - "POST /node/69b03189ae557f2600f10bce HTTP/1.0" 202 Accepted 2026-03-12 14:58:33,715 - lava_callback - INFO - Starting async job submit for node 69b03189ae557f2600f10bce 2026-03-12 14:58:33,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:58:33,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03189ae557f2600f10bce HTTP/1.1" 200 None 2026-03-12 14:58:33,761 - lava_callback - DEBUG - Processing node 69b03189ae557f2600f10bce: baseline-arm64-baylibre 2026-03-12 14:58:33,761 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03189ae557f2600f10bce 2026-03-12 14:58:33,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:58:33,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03189ae557f2600f10bce HTTP/1.1" 200 None 2026-03-12 14:58:33,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:58:33,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 14:58:33,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:58:33,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03189ae557f2600f10bce HTTP/1.1" 200 None 2026-03-12 14:58:33,824 - lava_callback - ERROR - Error processing callback for node 69b03189ae557f2600f10bce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60324 - "GET / HTTP/1.0" 200 OK 2026-03-12 14:59:53,909 - lava_callback - INFO - Received callback for node 69b031deae557f2600f10be6 INFO: 172.20.0.1:56374 - "POST /node/69b031deae557f2600f10be6 HTTP/1.0" 202 Accepted 2026-03-12 14:59:53,915 - lava_callback - INFO - Starting async job submit for node 69b031deae557f2600f10be6 2026-03-12 14:59:53,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:59:53,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b031deae557f2600f10be6 HTTP/1.1" 200 None 2026-03-12 14:59:53,931 - lava_callback - DEBUG - Processing node 69b031deae557f2600f10be6: baseline-arm64-baylibre 2026-03-12 14:59:53,931 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b031deae557f2600f10be6 2026-03-12 14:59:53,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:59:53,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b031deae557f2600f10be6 HTTP/1.1" 200 None 2026-03-12 14:59:53,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:59:53,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 14:59:53,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 14:59:53,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b031deae557f2600f10be6 HTTP/1.1" 200 None 2026-03-12 14:59:53,985 - lava_callback - ERROR - Error processing callback for node 69b031deae557f2600f10be6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36822 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:01:13,926 - lava_callback - INFO - Received callback for node 69b03234ae557f2600f10bfa INFO: 172.20.0.1:45990 - "POST /node/69b03234ae557f2600f10bfa HTTP/1.0" 202 Accepted 2026-03-12 15:01:13,932 - lava_callback - INFO - Starting async job submit for node 69b03234ae557f2600f10bfa 2026-03-12 15:01:13,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:01:13,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03234ae557f2600f10bfa HTTP/1.1" 200 None 2026-03-12 15:01:13,948 - lava_callback - DEBUG - Processing node 69b03234ae557f2600f10bfa: baseline-arm64-baylibre 2026-03-12 15:01:13,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03234ae557f2600f10bfa 2026-03-12 15:01:13,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:01:13,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03234ae557f2600f10bfa HTTP/1.1" 200 None 2026-03-12 15:01:13,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:01:13,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 15:01:13,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:01:14,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03234ae557f2600f10bfa HTTP/1.1" 200 None 2026-03-12 15:01:14,009 - lava_callback - ERROR - Error processing callback for node 69b03234ae557f2600f10bfa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48160 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:02:53,976 - lava_callback - INFO - Received callback for node 69b0328bae557f2600f10c0d INFO: 172.20.0.1:41784 - "POST /node/69b0328bae557f2600f10c0d HTTP/1.0" 202 Accepted 2026-03-12 15:02:53,982 - lava_callback - INFO - Starting async job submit for node 69b0328bae557f2600f10c0d 2026-03-12 15:02:53,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:02:53,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0328bae557f2600f10c0d HTTP/1.1" 200 None 2026-03-12 15:02:53,998 - lava_callback - DEBUG - Processing node 69b0328bae557f2600f10c0d: baseline-arm64-baylibre 2026-03-12 15:02:53,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0328bae557f2600f10c0d 2026-03-12 15:02:53,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:02:54,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0328bae557f2600f10c0d HTTP/1.1" 200 None 2026-03-12 15:02:54,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:02:54,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 15:02:54,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:02:54,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0328bae557f2600f10c0d HTTP/1.1" 200 None 2026-03-12 15:02:54,061 - lava_callback - ERROR - Error processing callback for node 69b0328bae557f2600f10c0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45746 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:04:14,296 - lava_callback - INFO - Received callback for node 69b032dfae557f2600f10c1f INFO: 172.20.0.1:36204 - "POST /node/69b032dfae557f2600f10c1f HTTP/1.0" 202 Accepted 2026-03-12 15:04:14,301 - lava_callback - INFO - Starting async job submit for node 69b032dfae557f2600f10c1f 2026-03-12 15:04:14,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:04:14,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b032dfae557f2600f10c1f HTTP/1.1" 200 None 2026-03-12 15:04:14,329 - lava_callback - DEBUG - Processing node 69b032dfae557f2600f10c1f: baseline-arm64-baylibre 2026-03-12 15:04:14,329 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b032dfae557f2600f10c1f 2026-03-12 15:04:14,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:04:14,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b032dfae557f2600f10c1f HTTP/1.1" 200 None 2026-03-12 15:04:14,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:04:14,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 15:04:14,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:04:14,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b032dfae557f2600f10c1f HTTP/1.1" 200 None 2026-03-12 15:04:14,389 - lava_callback - ERROR - Error processing callback for node 69b032dfae557f2600f10c1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53358 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:05:34,136 - lava_callback - INFO - Received callback for node 69b03334ae557f2600f10c27 INFO: 172.20.0.1:38404 - "POST /node/69b03334ae557f2600f10c27 HTTP/1.0" 202 Accepted 2026-03-12 15:05:34,141 - lava_callback - INFO - Starting async job submit for node 69b03334ae557f2600f10c27 2026-03-12 15:05:34,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:05:34,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03334ae557f2600f10c27 HTTP/1.1" 200 None 2026-03-12 15:05:34,157 - lava_callback - DEBUG - Processing node 69b03334ae557f2600f10c27: baseline-arm64-baylibre 2026-03-12 15:05:34,158 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03334ae557f2600f10c27 2026-03-12 15:05:34,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:05:34,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03334ae557f2600f10c27 HTTP/1.1" 200 None 2026-03-12 15:05:34,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:05:34,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 15:05:34,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:05:34,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03334ae557f2600f10c27 HTTP/1.1" 200 None 2026-03-12 15:05:34,210 - lava_callback - ERROR - Error processing callback for node 69b03334ae557f2600f10c27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57666 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:06:59,673 - lava_callback - INFO - Received callback for node 69b0338fae557f2600f10c38 INFO: 172.20.0.1:57674 - "POST /node/69b0338fae557f2600f10c38 HTTP/1.0" 202 Accepted 2026-03-12 15:06:59,678 - lava_callback - INFO - Starting async job submit for node 69b0338fae557f2600f10c38 2026-03-12 15:06:59,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:06:59,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0338fae557f2600f10c38 HTTP/1.1" 200 None 2026-03-12 15:06:59,695 - lava_callback - DEBUG - Processing node 69b0338fae557f2600f10c38: baseline-arm64-clabbe 2026-03-12 15:06:59,695 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0338fae557f2600f10c38 2026-03-12 15:06:59,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:06:59,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0338fae557f2600f10c38 HTTP/1.1" 200 None 2026-03-12 15:06:59,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:06:59,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 15:06:59,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:06:59,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0338fae557f2600f10c38 HTTP/1.1" 200 None 2026-03-12 15:06:59,760 - lava_callback - ERROR - Error processing callback for node 69b0338fae557f2600f10c38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35156 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:08:39,702 - lava_callback - INFO - Received callback for node 69b033e7ae557f2600f10c46 INFO: 172.20.0.1:57632 - "POST /node/69b033e7ae557f2600f10c46 HTTP/1.0" 202 Accepted 2026-03-12 15:08:39,707 - lava_callback - INFO - Starting async job submit for node 69b033e7ae557f2600f10c46 2026-03-12 15:08:39,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:08:39,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b033e7ae557f2600f10c46 HTTP/1.1" 200 None 2026-03-12 15:08:39,723 - lava_callback - DEBUG - Processing node 69b033e7ae557f2600f10c46: baseline-arm64-clabbe 2026-03-12 15:08:39,723 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b033e7ae557f2600f10c46 2026-03-12 15:08:39,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:08:39,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b033e7ae557f2600f10c46 HTTP/1.1" 200 None 2026-03-12 15:08:39,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:08:39,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 15:08:39,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:08:39,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b033e7ae557f2600f10c46 HTTP/1.1" 200 None 2026-03-12 15:08:39,783 - lava_callback - ERROR - Error processing callback for node 69b033e7ae557f2600f10c46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46032 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:09:59,759 - lava_callback - INFO - Received callback for node 69b0343aae557f2600f10c56 INFO: 172.20.0.1:46936 - "POST /node/69b0343aae557f2600f10c56 HTTP/1.0" 202 Accepted 2026-03-12 15:09:59,765 - lava_callback - INFO - Starting async job submit for node 69b0343aae557f2600f10c56 2026-03-12 15:09:59,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:09:59,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0343aae557f2600f10c56 HTTP/1.1" 200 None 2026-03-12 15:09:59,812 - lava_callback - DEBUG - Processing node 69b0343aae557f2600f10c56: baseline-arm64-clabbe 2026-03-12 15:09:59,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0343aae557f2600f10c56 2026-03-12 15:09:59,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:09:59,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0343aae557f2600f10c56 HTTP/1.1" 200 None 2026-03-12 15:09:59,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:09:59,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 15:09:59,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:09:59,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0343aae557f2600f10c56 HTTP/1.1" 200 None 2026-03-12 15:09:59,871 - lava_callback - ERROR - Error processing callback for node 69b0343aae557f2600f10c56: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:11:19,806 - lava_callback - INFO - Received callback for node 69b03490ae557f2600f10c67 INFO: 172.20.0.1:49222 - "POST /node/69b03490ae557f2600f10c67 HTTP/1.0" 202 Accepted 2026-03-12 15:11:19,811 - lava_callback - INFO - Starting async job submit for node 69b03490ae557f2600f10c67 2026-03-12 15:11:19,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:11:19,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03490ae557f2600f10c67 HTTP/1.1" 200 None 2026-03-12 15:11:19,830 - lava_callback - DEBUG - Processing node 69b03490ae557f2600f10c67: baseline-arm64-clabbe 2026-03-12 15:11:19,830 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03490ae557f2600f10c67 2026-03-12 15:11:19,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:11:19,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03490ae557f2600f10c67 HTTP/1.1" 200 None 2026-03-12 15:11:19,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:11:19,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 15:11:19,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:11:19,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03490ae557f2600f10c67 HTTP/1.1" 200 None 2026-03-12 15:11:19,915 - lava_callback - ERROR - Error processing callback for node 69b03490ae557f2600f10c67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51352 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:13:01,323 - lava_callback - INFO - Received callback for node 69b034e7ae557f2600f10c84 INFO: 172.20.0.1:55984 - "POST /node/69b034e7ae557f2600f10c84 HTTP/1.0" 202 Accepted 2026-03-12 15:13:01,328 - lava_callback - INFO - Starting async job submit for node 69b034e7ae557f2600f10c84 2026-03-12 15:13:01,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:13:01,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b034e7ae557f2600f10c84 HTTP/1.1" 200 None 2026-03-12 15:13:01,343 - lava_callback - DEBUG - Processing node 69b034e7ae557f2600f10c84: baseline-arm64-clabbe 2026-03-12 15:13:01,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b034e7ae557f2600f10c84 2026-03-12 15:13:01,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:13:01,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b034e7ae557f2600f10c84 HTTP/1.1" 200 None 2026-03-12 15:13:01,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:13:01,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 15:13:01,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:13:01,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b034e7ae557f2600f10c84 HTTP/1.1" 200 None 2026-03-12 15:13:01,401 - lava_callback - ERROR - Error processing callback for node 69b034e7ae557f2600f10c84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58866 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:14:19,928 - lava_callback - INFO - Received callback for node 69b0353dae557f2600f10cb3 INFO: 172.20.0.1:33530 - "POST /node/69b0353dae557f2600f10cb3 HTTP/1.0" 202 Accepted 2026-03-12 15:14:19,934 - lava_callback - INFO - Starting async job submit for node 69b0353dae557f2600f10cb3 2026-03-12 15:14:19,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:14:19,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0353dae557f2600f10cb3 HTTP/1.1" 200 None 2026-03-12 15:14:19,949 - lava_callback - DEBUG - Processing node 69b0353dae557f2600f10cb3: baseline-arm64-clabbe 2026-03-12 15:14:19,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0353dae557f2600f10cb3 2026-03-12 15:14:19,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:14:19,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0353dae557f2600f10cb3 HTTP/1.1" 200 None 2026-03-12 15:14:19,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:14:19,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3b0c9209b146d8e1d77 HTTP/1.1" 200 None 2026-03-12 15:14:19,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:14:20,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0353dae557f2600f10cb3 HTTP/1.1" 200 None 2026-03-12 15:14:20,002 - lava_callback - ERROR - Error processing callback for node 69b0353dae557f2600f10cb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32858 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:15:19,948 - lava_callback - INFO - Received callback for node 69b03585ae557f2600f10cbd INFO: 172.20.0.1:40604 - "POST /node/69b03585ae557f2600f10cbd HTTP/1.0" 202 Accepted 2026-03-12 15:15:19,954 - lava_callback - INFO - Starting async job submit for node 69b03585ae557f2600f10cbd 2026-03-12 15:15:19,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:15:19,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03585ae557f2600f10cbd HTTP/1.1" 200 None 2026-03-12 15:15:19,985 - lava_callback - DEBUG - Processing node 69b03585ae557f2600f10cbd: baseline-arm-clabbe 2026-03-12 15:15:19,985 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03585ae557f2600f10cbd 2026-03-12 15:15:19,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:15:19,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03585ae557f2600f10cbd HTTP/1.1" 200 None 2026-03-12 15:15:20,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:15:20,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8de776ddd5d9e606d488 HTTP/1.1" 200 None 2026-03-12 15:15:20,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:15:20,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03585ae557f2600f10cbd HTTP/1.1" 200 None 2026-03-12 15:15:20,037 - lava_callback - ERROR - Error processing callback for node 69b03585ae557f2600f10cbd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41710 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:16:54,780 - lava_callback - INFO - Received callback for node 69b035d9ae557f2600f10cd0 INFO: 172.20.0.1:56024 - "POST /node/69b035d9ae557f2600f10cd0 HTTP/1.0" 202 Accepted 2026-03-12 15:16:54,785 - lava_callback - INFO - Starting async job submit for node 69b035d9ae557f2600f10cd0 2026-03-12 15:16:54,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:16:54,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b035d9ae557f2600f10cd0 HTTP/1.1" 200 None 2026-03-12 15:16:54,802 - lava_callback - DEBUG - Processing node 69b035d9ae557f2600f10cd0: baseline-arm64-baylibre 2026-03-12 15:16:54,802 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b035d9ae557f2600f10cd0 2026-03-12 15:16:54,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:16:54,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b035d9ae557f2600f10cd0 HTTP/1.1" 200 None 2026-03-12 15:16:54,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:16:54,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:16:54,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:16:54,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b035d9ae557f2600f10cd0 HTTP/1.1" 200 None 2026-03-12 15:16:54,860 - lava_callback - ERROR - Error processing callback for node 69b035d9ae557f2600f10cd0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41144 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:18:54,889 - lava_callback - INFO - Received callback for node 69b03649ae557f2600f10cf8 INFO: 172.20.0.1:34460 - "POST /node/69b03649ae557f2600f10cf8 HTTP/1.0" 202 Accepted 2026-03-12 15:18:54,894 - lava_callback - INFO - Starting async job submit for node 69b03649ae557f2600f10cf8 2026-03-12 15:18:54,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:18:54,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03649ae557f2600f10cf8 HTTP/1.1" 200 None 2026-03-12 15:18:54,910 - lava_callback - DEBUG - Processing node 69b03649ae557f2600f10cf8: baseline-arm-baylibre 2026-03-12 15:18:54,910 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03649ae557f2600f10cf8 2026-03-12 15:18:54,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:18:54,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03649ae557f2600f10cf8 HTTP/1.1" 200 None 2026-03-12 15:18:54,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:18:54,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:18:54,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:18:54,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03649ae557f2600f10cf8 HTTP/1.1" 200 None 2026-03-12 15:18:54,998 - lava_callback - ERROR - Error processing callback for node 69b03649ae557f2600f10cf8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38038 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:19:14,880 - lava_callback - INFO - Received callback for node 69b03671ae557f2600f10d0b INFO: 172.20.0.1:48604 - "POST /node/69b03671ae557f2600f10d0b HTTP/1.0" 202 Accepted 2026-03-12 15:19:14,886 - lava_callback - INFO - Starting async job submit for node 69b03671ae557f2600f10d0b 2026-03-12 15:19:14,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:19:14,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03671ae557f2600f10d0b HTTP/1.1" 200 None 2026-03-12 15:19:14,905 - lava_callback - DEBUG - Processing node 69b03671ae557f2600f10d0b: baseline-arm-baylibre 2026-03-12 15:19:14,905 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03671ae557f2600f10d0b 2026-03-12 15:19:14,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:19:14,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03671ae557f2600f10d0b HTTP/1.1" 200 None 2026-03-12 15:19:14,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:19:14,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:19:14,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:19:14,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03671ae557f2600f10d0b HTTP/1.1" 200 None 2026-03-12 15:19:14,977 - lava_callback - ERROR - Error processing callback for node 69b03671ae557f2600f10d0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38308 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:20:14,980 - lava_callback - INFO - Received callback for node 69b0369bae557f2600f10d26 INFO: 172.20.0.1:46344 - "POST /node/69b0369bae557f2600f10d26 HTTP/1.0" 202 Accepted 2026-03-12 15:20:14,986 - lava_callback - INFO - Starting async job submit for node 69b0369bae557f2600f10d26 2026-03-12 15:20:14,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:20:15,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0369bae557f2600f10d26 HTTP/1.1" 200 None 2026-03-12 15:20:15,000 - lava_callback - DEBUG - Processing node 69b0369bae557f2600f10d26: baseline-arm-baylibre 2026-03-12 15:20:15,001 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0369bae557f2600f10d26 2026-03-12 15:20:15,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:20:15,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0369bae557f2600f10d26 HTTP/1.1" 200 None 2026-03-12 15:20:15,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:20:15,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:20:15,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:20:15,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0369bae557f2600f10d26 HTTP/1.1" 200 None 2026-03-12 15:20:15,064 - lava_callback - ERROR - Error processing callback for node 69b0369bae557f2600f10d26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33398 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:20:54,965 - lava_callback - INFO - Received callback for node 69b036c2ae557f2600f10d32 INFO: 172.20.0.1:39334 - "POST /node/69b036c2ae557f2600f10d32 HTTP/1.0" 202 Accepted 2026-03-12 15:20:54,970 - lava_callback - INFO - Starting async job submit for node 69b036c2ae557f2600f10d32 2026-03-12 15:20:54,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:20:54,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b036c2ae557f2600f10d32 HTTP/1.1" 200 None 2026-03-12 15:20:54,986 - lava_callback - DEBUG - Processing node 69b036c2ae557f2600f10d32: baseline-arm-baylibre 2026-03-12 15:20:54,987 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b036c2ae557f2600f10d32 2026-03-12 15:20:54,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:20:55,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b036c2ae557f2600f10d32 HTTP/1.1" 200 None 2026-03-12 15:20:55,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:20:55,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:20:55,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:20:55,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b036c2ae557f2600f10d32 HTTP/1.1" 200 None 2026-03-12 15:20:55,049 - lava_callback - ERROR - Error processing callback for node 69b036c2ae557f2600f10d32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35118 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:21:35,051 - lava_callback - INFO - Received callback for node 69b036ebae557f2600f10d3b INFO: 172.20.0.1:53182 - "POST /node/69b036ebae557f2600f10d3b HTTP/1.0" 202 Accepted 2026-03-12 15:21:35,056 - lava_callback - INFO - Starting async job submit for node 69b036ebae557f2600f10d3b 2026-03-12 15:21:35,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:21:35,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b036ebae557f2600f10d3b HTTP/1.1" 200 None 2026-03-12 15:21:35,099 - lava_callback - DEBUG - Processing node 69b036ebae557f2600f10d3b: baseline-arm-baylibre 2026-03-12 15:21:35,100 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b036ebae557f2600f10d3b 2026-03-12 15:21:35,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:21:35,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b036ebae557f2600f10d3b HTTP/1.1" 200 None 2026-03-12 15:21:35,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:21:35,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:21:35,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:21:35,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b036ebae557f2600f10d3b HTTP/1.1" 200 None 2026-03-12 15:21:35,160 - lava_callback - ERROR - Error processing callback for node 69b036ebae557f2600f10d3b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35756 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:22:15,101 - lava_callback - INFO - Received callback for node 69b03713ae557f2600f10d46 INFO: 172.20.0.1:43884 - "POST /node/69b03713ae557f2600f10d46 HTTP/1.0" 202 Accepted 2026-03-12 15:22:15,106 - lava_callback - INFO - Starting async job submit for node 69b03713ae557f2600f10d46 2026-03-12 15:22:15,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:22:15,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03713ae557f2600f10d46 HTTP/1.1" 200 None 2026-03-12 15:22:15,123 - lava_callback - DEBUG - Processing node 69b03713ae557f2600f10d46: baseline-arm-baylibre 2026-03-12 15:22:15,123 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03713ae557f2600f10d46 2026-03-12 15:22:15,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:22:15,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03713ae557f2600f10d46 HTTP/1.1" 200 None 2026-03-12 15:22:15,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:22:15,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:22:15,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:43900 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 15:22:15,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03713ae557f2600f10d46 HTTP/1.1" 200 None 2026-03-12 15:22:15,177 - lava_callback - ERROR - Error processing callback for node 69b03713ae557f2600f10d46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37966 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:24:00,369 - lava_callback - INFO - Received callback for node 69b0378bae557f2600f10d63 INFO: 172.20.0.1:50056 - "POST /node/69b0378bae557f2600f10d63 HTTP/1.0" 202 Accepted 2026-03-12 15:24:00,375 - lava_callback - INFO - Starting async job submit for node 69b0378bae557f2600f10d63 2026-03-12 15:24:00,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:24:00,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0378bae557f2600f10d63 HTTP/1.1" 200 None 2026-03-12 15:24:00,392 - lava_callback - DEBUG - Processing node 69b0378bae557f2600f10d63: baseline-arm-clabbe 2026-03-12 15:24:00,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0378bae557f2600f10d63 2026-03-12 15:24:00,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:24:00,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0378bae557f2600f10d63 HTTP/1.1" 200 None 2026-03-12 15:24:00,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:24:00,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:24:00,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:24:00,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0378bae557f2600f10d63 HTTP/1.1" 200 None 2026-03-12 15:24:00,465 - lava_callback - ERROR - Error processing callback for node 69b0378bae557f2600f10d63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37724 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:24:40,368 - lava_callback - INFO - Received callback for node 69b037b3ae557f2600f10d70 INFO: 172.20.0.1:47674 - "POST /node/69b037b3ae557f2600f10d70 HTTP/1.0" 202 Accepted 2026-03-12 15:24:40,373 - lava_callback - INFO - Starting async job submit for node 69b037b3ae557f2600f10d70 2026-03-12 15:24:40,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:24:40,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b037b3ae557f2600f10d70 HTTP/1.1" 200 None 2026-03-12 15:24:40,388 - lava_callback - DEBUG - Processing node 69b037b3ae557f2600f10d70: baseline-arm-clabbe 2026-03-12 15:24:40,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b037b3ae557f2600f10d70 2026-03-12 15:24:40,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:24:40,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b037b3ae557f2600f10d70 HTTP/1.1" 200 None 2026-03-12 15:24:40,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:24:40,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:24:40,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:24:40,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b037b3ae557f2600f10d70 HTTP/1.1" 200 None 2026-03-12 15:24:40,460 - lava_callback - ERROR - Error processing callback for node 69b037b3ae557f2600f10d70: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47000 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:25:35,198 - lava_callback - INFO - Received callback for node 69b037e6ae557f2600f10d7a INFO: 172.20.0.1:48420 - "POST /node/69b037e6ae557f2600f10d7a HTTP/1.0" 202 Accepted 2026-03-12 15:25:35,203 - lava_callback - INFO - Starting async job submit for node 69b037e6ae557f2600f10d7a 2026-03-12 15:25:35,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:25:35,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b037e6ae557f2600f10d7a HTTP/1.1" 200 None 2026-03-12 15:25:35,228 - lava_callback - DEBUG - Processing node 69b037e6ae557f2600f10d7a: baseline-arm64-baylibre 2026-03-12 15:25:35,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b037e6ae557f2600f10d7a 2026-03-12 15:25:35,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:25:35,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b037e6ae557f2600f10d7a HTTP/1.1" 200 None 2026-03-12 15:25:35,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:25:35,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:25:35,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:25:35,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b037e6ae557f2600f10d7a HTTP/1.1" 200 None 2026-03-12 15:25:35,296 - lava_callback - ERROR - Error processing callback for node 69b037e6ae557f2600f10d7a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37928 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:27:00,495 - lava_callback - INFO - Received callback for node 69b03836ae557f2600f10d86 INFO: 172.20.0.1:37936 - "POST /node/69b03836ae557f2600f10d86 HTTP/1.0" 202 Accepted 2026-03-12 15:27:00,501 - lava_callback - INFO - Starting async job submit for node 69b03836ae557f2600f10d86 2026-03-12 15:27:00,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:27:00,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03836ae557f2600f10d86 HTTP/1.1" 200 None 2026-03-12 15:27:00,517 - lava_callback - DEBUG - Processing node 69b03836ae557f2600f10d86: baseline-arm-clabbe 2026-03-12 15:27:00,517 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03836ae557f2600f10d86 2026-03-12 15:27:00,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:27:00,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03836ae557f2600f10d86 HTTP/1.1" 200 None 2026-03-12 15:27:00,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:27:00,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:27:00,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:27:00,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03836ae557f2600f10d86 HTTP/1.1" 200 None 2026-03-12 15:27:00,571 - lava_callback - ERROR - Error processing callback for node 69b03836ae557f2600f10d86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37362 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:27:40,499 - lava_callback - INFO - Received callback for node 69b0385eae557f2600f10d90 INFO: 172.20.0.1:43284 - "POST /node/69b0385eae557f2600f10d90 HTTP/1.0" 202 Accepted 2026-03-12 15:27:40,504 - lava_callback - INFO - Starting async job submit for node 69b0385eae557f2600f10d90 2026-03-12 15:27:40,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:27:40,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0385eae557f2600f10d90 HTTP/1.1" 200 None 2026-03-12 15:27:40,527 - lava_callback - DEBUG - Processing node 69b0385eae557f2600f10d90: baseline-arm-clabbe 2026-03-12 15:27:40,527 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0385eae557f2600f10d90 2026-03-12 15:27:40,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:27:40,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0385eae557f2600f10d90 HTTP/1.1" 200 None 2026-03-12 15:27:40,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:27:40,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:27:40,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:27:40,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0385eae557f2600f10d90 HTTP/1.1" 200 None 2026-03-12 15:27:40,583 - lava_callback - ERROR - Error processing callback for node 69b0385eae557f2600f10d90: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40000 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:28:20,572 - lava_callback - INFO - Received callback for node 69b03888ae557f2600f10d9d INFO: 172.20.0.1:35152 - "POST /node/69b03888ae557f2600f10d9d HTTP/1.0" 202 Accepted 2026-03-12 15:28:20,577 - lava_callback - INFO - Starting async job submit for node 69b03888ae557f2600f10d9d 2026-03-12 15:28:20,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:28:20,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03888ae557f2600f10d9d HTTP/1.1" 200 None 2026-03-12 15:28:20,601 - lava_callback - DEBUG - Processing node 69b03888ae557f2600f10d9d: baseline-arm-clabbe 2026-03-12 15:28:20,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03888ae557f2600f10d9d 2026-03-12 15:28:20,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:28:20,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03888ae557f2600f10d9d HTTP/1.1" 200 None 2026-03-12 15:28:20,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:28:20,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd3a3c9209b146d8e1d68 HTTP/1.1" 200 None 2026-03-12 15:28:20,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:28:20,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03888ae557f2600f10d9d HTTP/1.1" 200 None 2026-03-12 15:28:20,660 - lava_callback - ERROR - Error processing callback for node 69b03888ae557f2600f10d9d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55084 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:28:55,527 - lava_callback - INFO - Received callback for node 69b038acae557f2600f10da7 INFO: 172.20.0.1:33684 - "POST /node/69b038acae557f2600f10da7 HTTP/1.0" 202 Accepted 2026-03-12 15:28:55,532 - lava_callback - INFO - Starting async job submit for node 69b038acae557f2600f10da7 2026-03-12 15:28:55,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:28:55,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b038acae557f2600f10da7 HTTP/1.1" 200 None 2026-03-12 15:28:55,549 - lava_callback - DEBUG - Processing node 69b038acae557f2600f10da7: baseline-arm-baylibre 2026-03-12 15:28:55,549 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b038acae557f2600f10da7 2026-03-12 15:28:55,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:28:55,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b038acae557f2600f10da7 HTTP/1.1" 200 None 2026-03-12 15:28:55,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:28:55,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:28:55,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:28:55,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b038acae557f2600f10da7 HTTP/1.1" 200 None 2026-03-12 15:28:55,605 - lava_callback - ERROR - Error processing callback for node 69b038acae557f2600f10da7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37344 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:29:55,456 - lava_callback - INFO - Received callback for node 69b038dfae557f2600f10db3 INFO: 172.20.0.1:34034 - "POST /node/69b038dfae557f2600f10db3 HTTP/1.0" 202 Accepted 2026-03-12 15:29:55,461 - lava_callback - INFO - Starting async job submit for node 69b038dfae557f2600f10db3 2026-03-12 15:29:55,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:29:55,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b038dfae557f2600f10db3 HTTP/1.1" 200 None 2026-03-12 15:29:55,478 - lava_callback - DEBUG - Processing node 69b038dfae557f2600f10db3: baseline-arm64-baylibre 2026-03-12 15:29:55,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b038dfae557f2600f10db3 2026-03-12 15:29:55,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:29:55,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b038dfae557f2600f10db3 HTTP/1.1" 200 None 2026-03-12 15:29:55,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:29:55,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:29:55,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:29:55,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b038dfae557f2600f10db3 HTTP/1.1" 200 None 2026-03-12 15:29:55,531 - lava_callback - ERROR - Error processing callback for node 69b038dfae557f2600f10db3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45478 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:45484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35382 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 15:33:15,727 - lava_callback - INFO - Received callback for node 69b039b1ae557f2600f10dcd INFO: 172.20.0.1:35384 - "POST /node/69b039b1ae557f2600f10dcd HTTP/1.0" 202 Accepted 2026-03-12 15:33:15,732 - lava_callback - INFO - Starting async job submit for node 69b039b1ae557f2600f10dcd 2026-03-12 15:33:15,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:33:15,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b039b1ae557f2600f10dcd HTTP/1.1" 200 None 2026-03-12 15:33:15,748 - lava_callback - DEBUG - Processing node 69b039b1ae557f2600f10dcd: baseline-arm-baylibre 2026-03-12 15:33:15,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b039b1ae557f2600f10dcd 2026-03-12 15:33:15,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:33:15,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b039b1ae557f2600f10dcd HTTP/1.1" 200 None 2026-03-12 15:33:15,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:33:15,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:33:15,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:33:15,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b039b1ae557f2600f10dcd HTTP/1.1" 200 None 2026-03-12 15:33:15,824 - lava_callback - ERROR - Error processing callback for node 69b039b1ae557f2600f10dcd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56546 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:33:55,707 - lava_callback - INFO - Received callback for node 69b039d8ae557f2600f10dd9 INFO: 172.20.0.1:56338 - "POST /node/69b039d8ae557f2600f10dd9 HTTP/1.0" 202 Accepted 2026-03-12 15:33:55,712 - lava_callback - INFO - Starting async job submit for node 69b039d8ae557f2600f10dd9 2026-03-12 15:33:55,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:33:55,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b039d8ae557f2600f10dd9 HTTP/1.1" 200 None 2026-03-12 15:33:55,729 - lava_callback - DEBUG - Processing node 69b039d8ae557f2600f10dd9: baseline-arm-baylibre 2026-03-12 15:33:55,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b039d8ae557f2600f10dd9 2026-03-12 15:33:55,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:33:55,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b039d8ae557f2600f10dd9 HTTP/1.1" 200 None 2026-03-12 15:33:55,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:33:55,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:33:55,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:33:55,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b039d8ae557f2600f10dd9 HTTP/1.1" 200 None 2026-03-12 15:33:55,831 - lava_callback - ERROR - Error processing callback for node 69b039d8ae557f2600f10dd9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48012 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:34:35,749 - lava_callback - INFO - Received callback for node 69b03a01ae557f2600f10de2 INFO: 172.20.0.1:48126 - "POST /node/69b03a01ae557f2600f10de2 HTTP/1.0" 202 Accepted 2026-03-12 15:34:35,757 - lava_callback - INFO - Starting async job submit for node 69b03a01ae557f2600f10de2 2026-03-12 15:34:35,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:34:35,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03a01ae557f2600f10de2 HTTP/1.1" 200 None 2026-03-12 15:34:35,772 - lava_callback - DEBUG - Processing node 69b03a01ae557f2600f10de2: baseline-arm-baylibre 2026-03-12 15:34:35,773 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03a01ae557f2600f10de2 2026-03-12 15:34:35,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:34:35,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03a01ae557f2600f10de2 HTTP/1.1" 200 None 2026-03-12 15:34:35,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:34:35,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:34:35,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:34:35,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03a01ae557f2600f10de2 HTTP/1.1" 200 None 2026-03-12 15:34:35,841 - lava_callback - ERROR - Error processing callback for node 69b03a01ae557f2600f10de2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39098 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 15:35:15,780 - lava_callback - INFO - Received callback for node 69b03a2bae557f2600f10def INFO: 172.20.0.1:39106 - "POST /node/69b03a2bae557f2600f10def HTTP/1.0" 202 Accepted 2026-03-12 15:35:15,786 - lava_callback - INFO - Starting async job submit for node 69b03a2bae557f2600f10def 2026-03-12 15:35:15,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:35:15,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03a2bae557f2600f10def HTTP/1.1" 200 None 2026-03-12 15:35:15,803 - lava_callback - DEBUG - Processing node 69b03a2bae557f2600f10def: baseline-arm-baylibre 2026-03-12 15:35:15,803 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03a2bae557f2600f10def 2026-03-12 15:35:15,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:35:15,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03a2bae557f2600f10def HTTP/1.1" 200 None 2026-03-12 15:35:15,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:35:15,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:35:15,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:35:15,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03a2bae557f2600f10def HTTP/1.1" 200 None 2026-03-12 15:35:15,863 - lava_callback - ERROR - Error processing callback for node 69b03a2bae557f2600f10def: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49360 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:36:00,940 - lava_callback - INFO - Received callback for node 69b03a58ae557f2600f10dfa INFO: 172.20.0.1:49376 - "POST /node/69b03a58ae557f2600f10dfa HTTP/1.0" 202 Accepted 2026-03-12 15:36:00,945 - lava_callback - INFO - Starting async job submit for node 69b03a58ae557f2600f10dfa 2026-03-12 15:36:00,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:36:00,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03a58ae557f2600f10dfa HTTP/1.1" 200 None 2026-03-12 15:36:00,961 - lava_callback - DEBUG - Processing node 69b03a58ae557f2600f10dfa: baseline-arm-clabbe 2026-03-12 15:36:00,962 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03a58ae557f2600f10dfa 2026-03-12 15:36:00,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:36:00,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03a58ae557f2600f10dfa HTTP/1.1" 200 None 2026-03-12 15:36:00,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:36:00,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:36:00,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:36:01,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03a58ae557f2600f10dfa HTTP/1.1" 200 None 2026-03-12 15:36:01,024 - lava_callback - ERROR - Error processing callback for node 69b03a58ae557f2600f10dfa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49756 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:36:40,939 - lava_callback - INFO - Received callback for node 69b03a81ae557f2600f10e0c INFO: 172.20.0.1:48224 - "POST /node/69b03a81ae557f2600f10e0c HTTP/1.0" 202 Accepted 2026-03-12 15:36:40,944 - lava_callback - INFO - Starting async job submit for node 69b03a81ae557f2600f10e0c 2026-03-12 15:36:40,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:36:40,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03a81ae557f2600f10e0c HTTP/1.1" 200 None 2026-03-12 15:36:40,960 - lava_callback - DEBUG - Processing node 69b03a81ae557f2600f10e0c: baseline-arm-clabbe 2026-03-12 15:36:40,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03a81ae557f2600f10e0c 2026-03-12 15:36:40,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:36:40,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03a81ae557f2600f10e0c HTTP/1.1" 200 None 2026-03-12 15:36:40,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:36:40,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:36:40,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:36:41,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03a81ae557f2600f10e0c HTTP/1.1" 200 None 2026-03-12 15:36:41,017 - lava_callback - ERROR - Error processing callback for node 69b03a81ae557f2600f10e0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39224 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 15:38:15,978 - lava_callback - INFO - Received callback for node 69b03ad8ae557f2600f10e1a INFO: 172.20.0.1:39232 - "POST /node/69b03ad8ae557f2600f10e1a HTTP/1.0" 202 Accepted 2026-03-12 15:38:15,985 - lava_callback - INFO - Starting async job submit for node 69b03ad8ae557f2600f10e1a 2026-03-12 15:38:15,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:38:16,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03ad8ae557f2600f10e1a HTTP/1.1" 200 None 2026-03-12 15:38:16,001 - lava_callback - DEBUG - Processing node 69b03ad8ae557f2600f10e1a: baseline-arm-baylibre 2026-03-12 15:38:16,001 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03ad8ae557f2600f10e1a 2026-03-12 15:38:16,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:38:16,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03ad8ae557f2600f10e1a HTTP/1.1" 200 None 2026-03-12 15:38:16,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:38:16,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:38:16,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:38:16,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03ad8ae557f2600f10e1a HTTP/1.1" 200 None 2026-03-12 15:38:16,063 - lava_callback - ERROR - Error processing callback for node 69b03ad8ae557f2600f10e1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48910 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:38:55,968 - lava_callback - INFO - Received callback for node 69b03b01ae557f2600f10e24 INFO: 172.20.0.1:37090 - "POST /node/69b03b01ae557f2600f10e24 HTTP/1.0" 202 Accepted 2026-03-12 15:38:55,974 - lava_callback - INFO - Starting async job submit for node 69b03b01ae557f2600f10e24 2026-03-12 15:38:55,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:38:55,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03b01ae557f2600f10e24 HTTP/1.1" 200 None 2026-03-12 15:38:55,990 - lava_callback - DEBUG - Processing node 69b03b01ae557f2600f10e24: baseline-arm-baylibre 2026-03-12 15:38:55,990 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03b01ae557f2600f10e24 2026-03-12 15:38:55,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:38:56,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03b01ae557f2600f10e24 HTTP/1.1" 200 None 2026-03-12 15:38:56,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:38:56,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:38:56,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:38:56,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03b01ae557f2600f10e24 HTTP/1.1" 200 None 2026-03-12 15:38:56,080 - lava_callback - ERROR - Error processing callback for node 69b03b01ae557f2600f10e24: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37002 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:39:41,098 - lava_callback - INFO - Received callback for node 69b03b2cae557f2600f10e31 INFO: 172.20.0.1:34534 - "POST /node/69b03b2cae557f2600f10e31 HTTP/1.0" 202 Accepted 2026-03-12 15:39:41,103 - lava_callback - INFO - Starting async job submit for node 69b03b2cae557f2600f10e31 2026-03-12 15:39:41,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:39:41,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03b2cae557f2600f10e31 HTTP/1.1" 200 None 2026-03-12 15:39:41,119 - lava_callback - DEBUG - Processing node 69b03b2cae557f2600f10e31: baseline-arm-clabbe 2026-03-12 15:39:41,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03b2cae557f2600f10e31 2026-03-12 15:39:41,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:39:41,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03b2cae557f2600f10e31 HTTP/1.1" 200 None 2026-03-12 15:39:41,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:39:41,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:39:41,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:39:41,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03b2cae557f2600f10e31 HTTP/1.1" 200 None 2026-03-12 15:39:41,222 - lava_callback - ERROR - Error processing callback for node 69b03b2cae557f2600f10e31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45494 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 15:40:16,346 - lava_callback - INFO - Received callback for node 69b03b5bae557f2600f10e42 INFO: 172.20.0.1:45504 - "POST /node/69b03b5bae557f2600f10e42 HTTP/1.0" 202 Accepted 2026-03-12 15:40:16,351 - lava_callback - INFO - Starting async job submit for node 69b03b5bae557f2600f10e42 2026-03-12 15:40:16,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:40:16,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03b5bae557f2600f10e42 HTTP/1.1" 200 None 2026-03-12 15:40:16,368 - lava_callback - DEBUG - Processing node 69b03b5bae557f2600f10e42: baseline-arm64-baylibre 2026-03-12 15:40:16,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03b5bae557f2600f10e42 2026-03-12 15:40:16,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:40:16,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03b5bae557f2600f10e42 HTTP/1.1" 200 None 2026-03-12 15:40:16,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:40:16,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:40:16,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:40:16,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03b5bae557f2600f10e42 HTTP/1.1" 200 None 2026-03-12 15:40:16,424 - lava_callback - ERROR - Error processing callback for node 69b03b5bae557f2600f10e42: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52528 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:41:41,198 - lava_callback - INFO - Received callback for node 69b03baaae557f2600f10e53 INFO: 172.20.0.1:37320 - "POST /node/69b03baaae557f2600f10e53 HTTP/1.0" 202 Accepted 2026-03-12 15:41:41,203 - lava_callback - INFO - Starting async job submit for node 69b03baaae557f2600f10e53 2026-03-12 15:41:41,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:41:41,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03baaae557f2600f10e53 HTTP/1.1" 200 None 2026-03-12 15:41:41,219 - lava_callback - DEBUG - Processing node 69b03baaae557f2600f10e53: baseline-arm-clabbe 2026-03-12 15:41:41,219 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03baaae557f2600f10e53 2026-03-12 15:41:41,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:41:41,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03baaae557f2600f10e53 HTTP/1.1" 200 None 2026-03-12 15:41:41,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:41:41,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:41:41,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:41:41,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03baaae557f2600f10e53 HTTP/1.1" 200 None 2026-03-12 15:41:41,287 - lava_callback - ERROR - Error processing callback for node 69b03baaae557f2600f10e53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34664 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:42:21,181 - lava_callback - INFO - Received callback for node 69b03bd4ae557f2600f10e5b INFO: 172.20.0.1:52648 - "POST /node/69b03bd4ae557f2600f10e5b HTTP/1.0" 202 Accepted 2026-03-12 15:42:21,186 - lava_callback - INFO - Starting async job submit for node 69b03bd4ae557f2600f10e5b 2026-03-12 15:42:21,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:42:21,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03bd4ae557f2600f10e5b HTTP/1.1" 200 None 2026-03-12 15:42:21,203 - lava_callback - DEBUG - Processing node 69b03bd4ae557f2600f10e5b: baseline-arm-clabbe 2026-03-12 15:42:21,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03bd4ae557f2600f10e5b 2026-03-12 15:42:21,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:42:21,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03bd4ae557f2600f10e5b HTTP/1.1" 200 None 2026-03-12 15:42:21,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:42:21,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:42:21,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:42:21,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03bd4ae557f2600f10e5b HTTP/1.1" 200 None 2026-03-12 15:42:21,264 - lava_callback - ERROR - Error processing callback for node 69b03bd4ae557f2600f10e5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36710 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:43:01,248 - lava_callback - INFO - Received callback for node 69b03bfeae557f2600f10e6a INFO: 172.20.0.1:36716 - "POST /node/69b03bfeae557f2600f10e6a HTTP/1.0" 202 Accepted 2026-03-12 15:43:01,254 - lava_callback - INFO - Starting async job submit for node 69b03bfeae557f2600f10e6a 2026-03-12 15:43:01,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:43:01,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03bfeae557f2600f10e6a HTTP/1.1" 200 None 2026-03-12 15:43:01,270 - lava_callback - DEBUG - Processing node 69b03bfeae557f2600f10e6a: baseline-arm-clabbe 2026-03-12 15:43:01,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03bfeae557f2600f10e6a 2026-03-12 15:43:01,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:43:01,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03bfeae557f2600f10e6a HTTP/1.1" 200 None 2026-03-12 15:43:01,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:43:01,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ad30bced37758c525e15d2 HTTP/1.1" 200 None 2026-03-12 15:43:01,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:43:01,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03bfeae557f2600f10e6a HTTP/1.1" 200 None 2026-03-12 15:43:01,349 - lava_callback - ERROR - Error processing callback for node 69b03bfeae557f2600f10e6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52670 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:43:56,319 - lava_callback - INFO - Received callback for node 69b03c2eae557f2600f10e7a INFO: 172.20.0.1:39616 - "POST /node/69b03c2eae557f2600f10e7a HTTP/1.0" 202 Accepted 2026-03-12 15:43:56,325 - lava_callback - INFO - Starting async job submit for node 69b03c2eae557f2600f10e7a 2026-03-12 15:43:56,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:43:56,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03c2eae557f2600f10e7a HTTP/1.1" 200 None 2026-03-12 15:43:56,341 - lava_callback - DEBUG - Processing node 69b03c2eae557f2600f10e7a: baseline-arm64-baylibre 2026-03-12 15:43:56,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03c2eae557f2600f10e7a 2026-03-12 15:43:56,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:43:56,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03c2eae557f2600f10e7a HTTP/1.1" 200 None 2026-03-12 15:43:56,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:43:56,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:43:56,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:43:56,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03c2eae557f2600f10e7a HTTP/1.1" 200 None 2026-03-12 15:43:56,401 - lava_callback - ERROR - Error processing callback for node 69b03c2eae557f2600f10e7a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51284 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-12 15:45:16,272 - lava_callback - INFO - Received callback for node 69b03c83ae557f2600f10e86 INFO: 172.20.0.1:51288 - "POST /node/69b03c83ae557f2600f10e86 HTTP/1.0" 202 Accepted 2026-03-12 15:45:16,278 - lava_callback - INFO - Starting async job submit for node 69b03c83ae557f2600f10e86 2026-03-12 15:45:16,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:45:16,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03c83ae557f2600f10e86 HTTP/1.1" 200 None 2026-03-12 15:45:16,294 - lava_callback - DEBUG - Processing node 69b03c83ae557f2600f10e86: baseline-arm64-baylibre 2026-03-12 15:45:16,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03c83ae557f2600f10e86 2026-03-12 15:45:16,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:45:16,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03c83ae557f2600f10e86 HTTP/1.1" 200 None 2026-03-12 15:45:16,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:45:16,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:45:16,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:45:16,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03c83ae557f2600f10e86 HTTP/1.1" 200 None 2026-03-12 15:45:16,352 - lava_callback - ERROR - Error processing callback for node 69b03c83ae557f2600f10e86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41160 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:46:36,407 - lava_callback - INFO - Received callback for node 69b03cd6ae557f2600f10e97 INFO: 172.20.0.1:46056 - "POST /node/69b03cd6ae557f2600f10e97 HTTP/1.0" 202 Accepted 2026-03-12 15:46:36,413 - lava_callback - INFO - Starting async job submit for node 69b03cd6ae557f2600f10e97 2026-03-12 15:46:36,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:46:36,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03cd6ae557f2600f10e97 HTTP/1.1" 200 None 2026-03-12 15:46:36,430 - lava_callback - DEBUG - Processing node 69b03cd6ae557f2600f10e97: baseline-arm64-baylibre 2026-03-12 15:46:36,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03cd6ae557f2600f10e97 2026-03-12 15:46:36,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:46:36,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03cd6ae557f2600f10e97 HTTP/1.1" 200 None 2026-03-12 15:46:36,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:46:36,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:46:36,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:46:36,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03cd6ae557f2600f10e97 HTTP/1.1" 200 None 2026-03-12 15:46:36,511 - lava_callback - ERROR - Error processing callback for node 69b03cd6ae557f2600f10e97: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:49:41,517 - lava_callback - INFO - Received callback for node 69b03d84ae557f2600f10ea4 INFO: 172.20.0.1:39682 - "POST /node/69b03d84ae557f2600f10ea4 HTTP/1.0" 202 Accepted 2026-03-12 15:49:41,522 - lava_callback - INFO - Starting async job submit for node 69b03d84ae557f2600f10ea4 2026-03-12 15:49:41,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:49:41,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03d84ae557f2600f10ea4 HTTP/1.1" 200 None 2026-03-12 15:49:41,538 - lava_callback - DEBUG - Processing node 69b03d84ae557f2600f10ea4: baseline-arm64-clabbe 2026-03-12 15:49:41,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03d84ae557f2600f10ea4 2026-03-12 15:49:41,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:49:41,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03d84ae557f2600f10ea4 HTTP/1.1" 200 None 2026-03-12 15:49:41,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:49:41,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:49:41,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:49:41,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03d84ae557f2600f10ea4 HTTP/1.1" 200 None 2026-03-12 15:49:41,600 - lava_callback - ERROR - Error processing callback for node 69b03d84ae557f2600f10ea4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43728 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:51:01,591 - lava_callback - INFO - Received callback for node 69b03dd9ae557f2600f10eb2 INFO: 172.20.0.1:43730 - "POST /node/69b03dd9ae557f2600f10eb2 HTTP/1.0" 202 Accepted 2026-03-12 15:51:01,597 - lava_callback - INFO - Starting async job submit for node 69b03dd9ae557f2600f10eb2 2026-03-12 15:51:01,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:51:01,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03dd9ae557f2600f10eb2 HTTP/1.1" 200 None 2026-03-12 15:51:01,613 - lava_callback - DEBUG - Processing node 69b03dd9ae557f2600f10eb2: baseline-arm64-clabbe 2026-03-12 15:51:01,613 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03dd9ae557f2600f10eb2 2026-03-12 15:51:01,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:51:01,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03dd9ae557f2600f10eb2 HTTP/1.1" 200 None 2026-03-12 15:51:01,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:51:01,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:51:01,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:51:01,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03dd9ae557f2600f10eb2 HTTP/1.1" 200 None 2026-03-12 15:51:01,665 - lava_callback - ERROR - Error processing callback for node 69b03dd9ae557f2600f10eb2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52632 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:52:16,689 - lava_callback - INFO - Received callback for node 69b03e29ae557f2600f10ebe INFO: 172.20.0.1:52644 - "POST /node/69b03e29ae557f2600f10ebe HTTP/1.0" 202 Accepted 2026-03-12 15:52:16,694 - lava_callback - INFO - Starting async job submit for node 69b03e29ae557f2600f10ebe 2026-03-12 15:52:16,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:52:16,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03e29ae557f2600f10ebe HTTP/1.1" 200 None 2026-03-12 15:52:16,710 - lava_callback - DEBUG - Processing node 69b03e29ae557f2600f10ebe: baseline-arm64-baylibre 2026-03-12 15:52:16,710 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03e29ae557f2600f10ebe 2026-03-12 15:52:16,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:52:16,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03e29ae557f2600f10ebe HTTP/1.1" 200 None 2026-03-12 15:52:16,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:52:16,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:52:16,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:52:16,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03e29ae557f2600f10ebe HTTP/1.1" 200 None 2026-03-12 15:52:16,770 - lava_callback - ERROR - Error processing callback for node 69b03e29ae557f2600f10ebe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60560 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52404 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:53:41,672 - lava_callback - INFO - Received callback for node 69b03e83ae557f2600f10ecc INFO: 172.20.0.1:49400 - "POST /node/69b03e83ae557f2600f10ecc HTTP/1.0" 202 Accepted 2026-03-12 15:53:41,677 - lava_callback - INFO - Starting async job submit for node 69b03e83ae557f2600f10ecc 2026-03-12 15:53:41,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:53:41,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03e83ae557f2600f10ecc HTTP/1.1" 200 None 2026-03-12 15:53:41,695 - lava_callback - DEBUG - Processing node 69b03e83ae557f2600f10ecc: baseline-arm64-clabbe 2026-03-12 15:53:41,695 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03e83ae557f2600f10ecc 2026-03-12 15:53:41,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:53:41,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03e83ae557f2600f10ecc HTTP/1.1" 200 None 2026-03-12 15:53:41,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:53:41,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:53:41,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:53:41,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03e83ae557f2600f10ecc HTTP/1.1" 200 None 2026-03-12 15:53:41,757 - lava_callback - ERROR - Error processing callback for node 69b03e83ae557f2600f10ecc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46804 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:56:41,795 - lava_callback - INFO - Received callback for node 69b03f2cae557f2600f10edb INFO: 172.20.0.1:44718 - "POST /node/69b03f2cae557f2600f10edb HTTP/1.0" 202 Accepted 2026-03-12 15:56:41,800 - lava_callback - INFO - Starting async job submit for node 69b03f2cae557f2600f10edb 2026-03-12 15:56:41,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:56:41,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03f2cae557f2600f10edb HTTP/1.1" 200 None 2026-03-12 15:56:41,817 - lava_callback - DEBUG - Processing node 69b03f2cae557f2600f10edb: baseline-arm64-clabbe 2026-03-12 15:56:41,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03f2cae557f2600f10edb 2026-03-12 15:56:41,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:56:41,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03f2cae557f2600f10edb HTTP/1.1" 200 None 2026-03-12 15:56:41,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:56:41,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 15:56:41,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:56:41,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03f2cae557f2600f10edb HTTP/1.1" 200 None 2026-03-12 15:56:41,874 - lava_callback - ERROR - Error processing callback for node 69b03f2cae557f2600f10edb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41182 - "GET / HTTP/1.0" 200 OK 2026-03-12 15:58:57,111 - lava_callback - INFO - Received callback for node 69b03fbeae557f2600f10f00 INFO: 172.20.0.1:41186 - "POST /node/69b03fbeae557f2600f10f00 HTTP/1.0" 202 Accepted 2026-03-12 15:58:57,117 - lava_callback - INFO - Starting async job submit for node 69b03fbeae557f2600f10f00 2026-03-12 15:58:57,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:58:57,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03fbeae557f2600f10f00 HTTP/1.1" 200 None 2026-03-12 15:58:57,138 - lava_callback - DEBUG - Processing node 69b03fbeae557f2600f10f00: baseline-arm-baylibre 2026-03-12 15:58:57,138 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b03fbeae557f2600f10f00 2026-03-12 15:58:57,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:58:57,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b03fbeae557f2600f10f00 HTTP/1.1" 200 None 2026-03-12 15:58:57,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:58:57,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 15:58:57,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 15:58:57,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b03fbeae557f2600f10f00 HTTP/1.1" 200 None 2026-03-12 15:58:57,228 - lava_callback - ERROR - Error processing callback for node 69b03fbeae557f2600f10f00: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55202 - "GET / HTTP/1.0" 200 OK 2026-03-12 16:01:37,223 - lava_callback - INFO - Received callback for node 69b0405dae557f2600f10f2a INFO: 172.20.0.1:55204 - "POST /node/69b0405dae557f2600f10f2a HTTP/1.0" 202 Accepted 2026-03-12 16:01:37,228 - lava_callback - INFO - Starting async job submit for node 69b0405dae557f2600f10f2a 2026-03-12 16:01:37,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:01:37,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0405dae557f2600f10f2a HTTP/1.1" 200 None 2026-03-12 16:01:37,243 - lava_callback - DEBUG - Processing node 69b0405dae557f2600f10f2a: baseline-arm-baylibre 2026-03-12 16:01:37,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0405dae557f2600f10f2a 2026-03-12 16:01:37,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:01:37,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0405dae557f2600f10f2a HTTP/1.1" 200 None 2026-03-12 16:01:37,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:01:37,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 16:01:37,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:01:37,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0405dae557f2600f10f2a HTTP/1.1" 200 None 2026-03-12 16:01:37,321 - lava_callback - ERROR - Error processing callback for node 69b0405dae557f2600f10f2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56122 - "GET / HTTP/1.0" 200 OK 2026-03-12 16:04:22,155 - lava_callback - INFO - Received callback for node 69b04101ae557f2600f10f5f INFO: 172.20.0.1:34692 - "POST /node/69b04101ae557f2600f10f5f HTTP/1.0" 202 Accepted 2026-03-12 16:04:22,160 - lava_callback - INFO - Starting async job submit for node 69b04101ae557f2600f10f5f 2026-03-12 16:04:22,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:04:22,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b04101ae557f2600f10f5f HTTP/1.1" 200 None 2026-03-12 16:04:22,180 - lava_callback - DEBUG - Processing node 69b04101ae557f2600f10f5f: baseline-arm-clabbe 2026-03-12 16:04:22,181 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b04101ae557f2600f10f5f 2026-03-12 16:04:22,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:04:22,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b04101ae557f2600f10f5f HTTP/1.1" 200 None 2026-03-12 16:04:22,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:04:22,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 16:04:22,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:04:22,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b04101ae557f2600f10f5f HTTP/1.1" 200 None 2026-03-12 16:04:22,240 - lava_callback - ERROR - Error processing callback for node 69b04101ae557f2600f10f5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43578 - "GET / HTTP/1.0" 200 OK 2026-03-12 16:05:02,131 - lava_callback - INFO - Received callback for node 69b0412bae557f2600f10f71 INFO: 172.20.0.1:43584 - "POST /node/69b0412bae557f2600f10f71 HTTP/1.0" 202 Accepted 2026-03-12 16:05:02,137 - lava_callback - INFO - Starting async job submit for node 69b0412bae557f2600f10f71 2026-03-12 16:05:02,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:05:02,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0412bae557f2600f10f71 HTTP/1.1" 200 None 2026-03-12 16:05:02,153 - lava_callback - DEBUG - Processing node 69b0412bae557f2600f10f71: baseline-arm-clabbe 2026-03-12 16:05:02,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0412bae557f2600f10f71 2026-03-12 16:05:02,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:05:02,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0412bae557f2600f10f71 HTTP/1.1" 200 None 2026-03-12 16:05:02,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:05:02,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 16:05:02,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:05:02,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0412bae557f2600f10f71 HTTP/1.1" 200 None 2026-03-12 16:05:02,212 - lava_callback - ERROR - Error processing callback for node 69b0412bae557f2600f10f71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58462 - "GET / HTTP/1.0" 200 OK 2026-03-12 16:06:02,265 - lava_callback - INFO - Received callback for node 69b04155ae557f2600f10f80 INFO: 172.20.0.1:58476 - "POST /node/69b04155ae557f2600f10f80 HTTP/1.0" 202 Accepted 2026-03-12 16:06:02,270 - lava_callback - INFO - Starting async job submit for node 69b04155ae557f2600f10f80 2026-03-12 16:06:02,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:06:02,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b04155ae557f2600f10f80 HTTP/1.1" 200 None 2026-03-12 16:06:02,286 - lava_callback - DEBUG - Processing node 69b04155ae557f2600f10f80: baseline-arm-clabbe 2026-03-12 16:06:02,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b04155ae557f2600f10f80 2026-03-12 16:06:02,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:06:02,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b04155ae557f2600f10f80 HTTP/1.1" 200 None 2026-03-12 16:06:02,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:06:02,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 16:06:02,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:06:02,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b04155ae557f2600f10f80 HTTP/1.1" 200 None 2026-03-12 16:06:02,342 - lava_callback - ERROR - Error processing callback for node 69b04155ae557f2600f10f80: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37946 - "GET / HTTP/1.0" 200 OK 2026-03-12 16:06:42,236 - lava_callback - INFO - Received callback for node 69b0417eae557f2600f10f8f INFO: 172.20.0.1:50654 - "POST /node/69b0417eae557f2600f10f8f HTTP/1.0" 202 Accepted 2026-03-12 16:06:42,241 - lava_callback - INFO - Starting async job submit for node 69b0417eae557f2600f10f8f 2026-03-12 16:06:42,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:06:42,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0417eae557f2600f10f8f HTTP/1.1" 200 None 2026-03-12 16:06:42,258 - lava_callback - DEBUG - Processing node 69b0417eae557f2600f10f8f: baseline-arm-clabbe 2026-03-12 16:06:42,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0417eae557f2600f10f8f 2026-03-12 16:06:42,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:06:42,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0417eae557f2600f10f8f HTTP/1.1" 200 None 2026-03-12 16:06:42,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:06:42,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 16:06:42,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:06:42,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0417eae557f2600f10f8f HTTP/1.1" 200 None 2026-03-12 16:06:42,327 - lava_callback - ERROR - Error processing callback for node 69b0417eae557f2600f10f8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42132 - "GET / HTTP/1.0" 200 OK 2026-03-12 16:07:22,310 - lava_callback - INFO - Received callback for node 69b041a7ae557f2600f10fa3 INFO: 172.20.0.1:40134 - "POST /node/69b041a7ae557f2600f10fa3 HTTP/1.0" 202 Accepted 2026-03-12 16:07:22,315 - lava_callback - INFO - Starting async job submit for node 69b041a7ae557f2600f10fa3 2026-03-12 16:07:22,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:07:22,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b041a7ae557f2600f10fa3 HTTP/1.1" 200 None 2026-03-12 16:07:22,332 - lava_callback - DEBUG - Processing node 69b041a7ae557f2600f10fa3: baseline-arm-clabbe 2026-03-12 16:07:22,332 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b041a7ae557f2600f10fa3 2026-03-12 16:07:22,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:07:22,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b041a7ae557f2600f10fa3 HTTP/1.1" 200 None 2026-03-12 16:07:22,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:07:22,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69afd826ae557f2600f0f036 HTTP/1.1" 200 None 2026-03-12 16:07:22,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:07:22,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b041a7ae557f2600f10fa3 HTTP/1.1" 200 None 2026-03-12 16:07:22,404 - lava_callback - ERROR - Error processing callback for node 69b041a7ae557f2600f10fa3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35628 - "GET / HTTP/1.0" 200 OK 2026-03-12 16:07:57,585 - lava_callback - INFO - Received callback for node 69b041d8ae557f2600f10fab INFO: 172.20.0.1:35644 - "POST /node/69b041d8ae557f2600f10fab HTTP/1.0" 202 Accepted 2026-03-12 16:07:57,590 - lava_callback - INFO - Starting async job submit for node 69b041d8ae557f2600f10fab 2026-03-12 16:07:57,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:07:57,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b041d8ae557f2600f10fab HTTP/1.1" 200 None 2026-03-12 16:07:57,606 - lava_callback - DEBUG - Processing node 69b041d8ae557f2600f10fab: baseline-arm64-baylibre 2026-03-12 16:07:57,607 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b041d8ae557f2600f10fab 2026-03-12 16:07:57,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:07:57,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b041d8ae557f2600f10fab HTTP/1.1" 200 None 2026-03-12 16:07:57,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:07:57,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 16:07:57,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:07:57,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b041d8ae557f2600f10fab HTTP/1.1" 200 None 2026-03-12 16:07:57,664 - lava_callback - ERROR - Error processing callback for node 69b041d8ae557f2600f10fab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59980 - "GET / HTTP/1.0" 200 OK 2026-03-12 16:09:37,650 - lava_callback - INFO - Received callback for node 69b0422cae557f2600f10fb7 INFO: 172.20.0.1:59994 - "POST /node/69b0422cae557f2600f10fb7 HTTP/1.0" 202 Accepted 2026-03-12 16:09:37,655 - lava_callback - INFO - Starting async job submit for node 69b0422cae557f2600f10fb7 2026-03-12 16:09:37,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:09:37,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0422cae557f2600f10fb7 HTTP/1.1" 200 None 2026-03-12 16:09:37,672 - lava_callback - DEBUG - Processing node 69b0422cae557f2600f10fb7: baseline-arm64-baylibre 2026-03-12 16:09:37,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69b0422cae557f2600f10fb7 2026-03-12 16:09:37,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:09:37,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b0422cae557f2600f10fb7 HTTP/1.1" 200 None 2026-03-12 16:09:37,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:09:37,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69aa8df576ddd5d9e606d49a HTTP/1.1" 200 None 2026-03-12 16:09:37,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-12 16:09:37,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69b0422cae557f2600f10fb7 HTTP/1.1" 200 None 2026-03-12 16:09:37,735 - lava_callback - ERROR - Error processing callback for node 69b0422cae557f2600f10fb7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]