INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:60242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50344 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:14:50,881 - lava_callback - INFO - Received callback for node 69ccd3f574bda3ec93a41230 2026-04-03 08:14:50,888 - lava_callback - INFO - Starting async job submit for node 69ccd3f574bda3ec93a41230 INFO: 172.21.0.1:48342 - "POST /node/69ccd3f574bda3ec93a41230 HTTP/1.0" 202 Accepted 2026-04-03 08:14:50,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:14:50,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd3f574bda3ec93a41230 HTTP/1.1" 200 None 2026-04-03 08:14:50,909 - lava_callback - DEBUG - Processing node 69ccd3f574bda3ec93a41230: baseline-arm64-clabbe 2026-04-03 08:14:50,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd3f574bda3ec93a41230 2026-04-03 08:14:50,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:14:50,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd3f574bda3ec93a41230 HTTP/1.1" 200 None 2026-04-03 08:14:50,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:14:50,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 08:14:50,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:14:51,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd3f574bda3ec93a41230 HTTP/1.1" 200 None 2026-04-03 08:14:51,011 - lava_callback - ERROR - Error processing callback for node 69ccd3f574bda3ec93a41230: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43540 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:16:30,972 - lava_callback - INFO - Received callback for node 69ccd44c74bda3ec93a4124d INFO: 172.21.0.1:54746 - "POST /node/69ccd44c74bda3ec93a4124d HTTP/1.0" 202 Accepted 2026-04-03 08:16:30,978 - lava_callback - INFO - Starting async job submit for node 69ccd44c74bda3ec93a4124d 2026-04-03 08:16:30,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:16:31,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd44c74bda3ec93a4124d HTTP/1.1" 200 None 2026-04-03 08:16:31,001 - lava_callback - DEBUG - Processing node 69ccd44c74bda3ec93a4124d: baseline-arm64-clabbe 2026-04-03 08:16:31,001 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd44c74bda3ec93a4124d 2026-04-03 08:16:31,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:16:31,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd44c74bda3ec93a4124d HTTP/1.1" 200 None 2026-04-03 08:16:31,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:16:31,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 08:16:31,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:16:31,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd44c74bda3ec93a4124d HTTP/1.1" 200 None 2026-04-03 08:16:31,079 - lava_callback - ERROR - Error processing callback for node 69ccd44c74bda3ec93a4124d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57514 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:17:51,052 - lava_callback - INFO - Received callback for node 69ccd4a874bda3ec93a4129e INFO: 172.21.0.1:36398 - "POST /node/69ccd4a874bda3ec93a4129e HTTP/1.0" 202 Accepted 2026-04-03 08:17:51,058 - lava_callback - INFO - Starting async job submit for node 69ccd4a874bda3ec93a4129e 2026-04-03 08:17:51,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:17:51,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd4a874bda3ec93a4129e HTTP/1.1" 200 None 2026-04-03 08:17:51,082 - lava_callback - DEBUG - Processing node 69ccd4a874bda3ec93a4129e: baseline-arm64-clabbe 2026-04-03 08:17:51,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd4a874bda3ec93a4129e 2026-04-03 08:17:51,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:17:51,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd4a874bda3ec93a4129e HTTP/1.1" 200 None 2026-04-03 08:17:51,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:17:51,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 08:17:51,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:17:51,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd4a874bda3ec93a4129e HTTP/1.1" 200 None 2026-04-03 08:17:51,142 - lava_callback - ERROR - Error processing callback for node 69ccd4a874bda3ec93a4129e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35950 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:19:31,297 - lava_callback - INFO - Received callback for node 69ccd4fd74bda3ec93a412cb INFO: 172.21.0.1:56906 - "POST /node/69ccd4fd74bda3ec93a412cb HTTP/1.0" 202 Accepted 2026-04-03 08:19:31,304 - lava_callback - INFO - Starting async job submit for node 69ccd4fd74bda3ec93a412cb 2026-04-03 08:19:31,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:19:31,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd4fd74bda3ec93a412cb HTTP/1.1" 200 None 2026-04-03 08:19:31,320 - lava_callback - DEBUG - Processing node 69ccd4fd74bda3ec93a412cb: baseline-arm64-clabbe 2026-04-03 08:19:31,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd4fd74bda3ec93a412cb 2026-04-03 08:19:31,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:19:31,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd4fd74bda3ec93a412cb HTTP/1.1" 200 None 2026-04-03 08:19:31,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:19:31,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 08:19:31,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:19:31,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd4fd74bda3ec93a412cb HTTP/1.1" 200 None 2026-04-03 08:19:31,383 - lava_callback - ERROR - Error processing callback for node 69ccd4fd74bda3ec93a412cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40832 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:20:51,300 - lava_callback - INFO - Received callback for node 69ccd55d74bda3ec93a412e3 INFO: 172.21.0.1:40836 - "POST /node/69ccd55d74bda3ec93a412e3 HTTP/1.0" 202 Accepted 2026-04-03 08:20:51,306 - lava_callback - INFO - Starting async job submit for node 69ccd55d74bda3ec93a412e3 2026-04-03 08:20:51,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:20:51,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd55d74bda3ec93a412e3 HTTP/1.1" 200 None 2026-04-03 08:20:51,327 - lava_callback - DEBUG - Processing node 69ccd55d74bda3ec93a412e3: baseline-arm64-clabbe 2026-04-03 08:20:51,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd55d74bda3ec93a412e3 2026-04-03 08:20:51,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:20:51,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd55d74bda3ec93a412e3 HTTP/1.1" 200 None 2026-04-03 08:20:51,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:20:51,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 08:20:51,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:20:51,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd55d74bda3ec93a412e3 HTTP/1.1" 200 None 2026-04-03 08:20:51,384 - lava_callback - ERROR - Error processing callback for node 69ccd55d74bda3ec93a412e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36760 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:37:59,676 - lava_callback - INFO - Received callback for node 69ccd95674bda3ec93a41502 INFO: 172.21.0.1:36770 - "POST /node/69ccd95674bda3ec93a41502 HTTP/1.0" 202 Accepted 2026-04-03 08:37:59,682 - lava_callback - INFO - Starting async job submit for node 69ccd95674bda3ec93a41502 2026-04-03 08:37:59,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:37:59,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd95674bda3ec93a41502 HTTP/1.1" 200 None 2026-04-03 08:37:59,747 - lava_callback - DEBUG - Processing node 69ccd95674bda3ec93a41502: baseline-arm-baylibre 2026-04-03 08:37:59,747 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd95674bda3ec93a41502 2026-04-03 08:37:59,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:37:59,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd95674bda3ec93a41502 HTTP/1.1" 200 None 2026-04-03 08:37:59,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:37:59,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 08:37:59,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:37:59,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd95674bda3ec93a41502 HTTP/1.1" 200 None 2026-04-03 08:37:59,852 - lava_callback - ERROR - Error processing callback for node 69ccd95674bda3ec93a41502: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 08:38:00,322 - lava_callback - INFO - Received callback for node 69ccd95774bda3ec93a41505 INFO: 172.21.0.1:45206 - "POST /node/69ccd95774bda3ec93a41505 HTTP/1.0" 202 Accepted 2026-04-03 08:38:00,327 - lava_callback - INFO - Starting async job submit for node 69ccd95774bda3ec93a41505 2026-04-03 08:38:00,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:00,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd95774bda3ec93a41505 HTTP/1.1" 200 None 2026-04-03 08:38:00,344 - lava_callback - DEBUG - Processing node 69ccd95774bda3ec93a41505: baseline-arm-baylibre 2026-04-03 08:38:00,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd95774bda3ec93a41505 2026-04-03 08:38:00,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:00,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd95774bda3ec93a41505 HTTP/1.1" 200 None 2026-04-03 08:38:00,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:00,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 08:38:00,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:00,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd95774bda3ec93a41505 HTTP/1.1" 200 None 2026-04-03 08:38:00,398 - lava_callback - ERROR - Error processing callback for node 69ccd95774bda3ec93a41505: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 08:38:00,999 - lava_callback - INFO - Received callback for node 69ccd95874bda3ec93a41508 INFO: 172.21.0.1:45218 - "POST /node/69ccd95874bda3ec93a41508 HTTP/1.0" 202 Accepted 2026-04-03 08:38:01,004 - lava_callback - INFO - Starting async job submit for node 69ccd95874bda3ec93a41508 2026-04-03 08:38:01,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:01,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd95874bda3ec93a41508 HTTP/1.1" 200 None 2026-04-03 08:38:01,021 - lava_callback - DEBUG - Processing node 69ccd95874bda3ec93a41508: baseline-arm-baylibre 2026-04-03 08:38:01,021 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd95874bda3ec93a41508 2026-04-03 08:38:01,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:01,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd95874bda3ec93a41508 HTTP/1.1" 200 None 2026-04-03 08:38:01,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:01,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 08:38:01,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:01,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd95874bda3ec93a41508 HTTP/1.1" 200 None 2026-04-03 08:38:01,084 - lava_callback - ERROR - Error processing callback for node 69ccd95874bda3ec93a41508: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45234 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:38:12,453 - lava_callback - INFO - Received callback for node 69ccd96374bda3ec93a41548 INFO: 172.21.0.1:54522 - "POST /node/69ccd96374bda3ec93a41548 HTTP/1.0" 202 Accepted 2026-04-03 08:38:12,459 - lava_callback - INFO - Starting async job submit for node 69ccd96374bda3ec93a41548 2026-04-03 08:38:12,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:12,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96374bda3ec93a41548 HTTP/1.1" 200 None 2026-04-03 08:38:12,475 - lava_callback - DEBUG - Processing node 69ccd96374bda3ec93a41548: baseline-arm-clabbe 2026-04-03 08:38:12,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd96374bda3ec93a41548 2026-04-03 08:38:12,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:12,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96374bda3ec93a41548 HTTP/1.1" 200 None 2026-04-03 08:38:12,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:12,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 08:38:12,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:12,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd96374bda3ec93a41548 HTTP/1.1" 200 None 2026-04-03 08:38:12,530 - lava_callback - ERROR - Error processing callback for node 69ccd96374bda3ec93a41548: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 08:38:13,174 - lava_callback - INFO - Received callback for node 69ccd96574bda3ec93a4154e INFO: 172.21.0.1:54532 - "POST /node/69ccd96574bda3ec93a4154e HTTP/1.0" 202 Accepted 2026-04-03 08:38:13,180 - lava_callback - INFO - Starting async job submit for node 69ccd96574bda3ec93a4154e 2026-04-03 08:38:13,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:13,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96574bda3ec93a4154e HTTP/1.1" 200 None 2026-04-03 08:38:13,196 - lava_callback - DEBUG - Processing node 69ccd96574bda3ec93a4154e: baseline-arm-clabbe 2026-04-03 08:38:13,196 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd96574bda3ec93a4154e 2026-04-03 08:38:13,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:13,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96574bda3ec93a4154e HTTP/1.1" 200 None 2026-04-03 08:38:13,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:13,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 08:38:13,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:13,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd96574bda3ec93a4154e HTTP/1.1" 200 None 2026-04-03 08:38:13,251 - lava_callback - ERROR - Error processing callback for node 69ccd96574bda3ec93a4154e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 08:38:13,866 - lava_callback - INFO - Received callback for node 69ccd96774bda3ec93a41555 INFO: 172.21.0.1:54544 - "POST /node/69ccd96774bda3ec93a41555 HTTP/1.0" 202 Accepted 2026-04-03 08:38:13,872 - lava_callback - INFO - Starting async job submit for node 69ccd96774bda3ec93a41555 2026-04-03 08:38:13,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:13,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96774bda3ec93a41555 HTTP/1.1" 200 None 2026-04-03 08:38:13,888 - lava_callback - DEBUG - Processing node 69ccd96774bda3ec93a41555: baseline-arm-clabbe 2026-04-03 08:38:13,888 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd96774bda3ec93a41555 2026-04-03 08:38:13,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:13,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96774bda3ec93a41555 HTTP/1.1" 200 None 2026-04-03 08:38:13,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:13,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 08:38:13,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:13,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd96774bda3ec93a41555 HTTP/1.1" 200 None 2026-04-03 08:38:13,949 - lava_callback - ERROR - Error processing callback for node 69ccd96774bda3ec93a41555: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 08:38:14,554 - lava_callback - INFO - Received callback for node 69ccd96774bda3ec93a41558 INFO: 172.21.0.1:54556 - "POST /node/69ccd96774bda3ec93a41558 HTTP/1.0" 202 Accepted 2026-04-03 08:38:14,559 - lava_callback - INFO - Starting async job submit for node 69ccd96774bda3ec93a41558 2026-04-03 08:38:14,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:14,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96774bda3ec93a41558 HTTP/1.1" 200 None 2026-04-03 08:38:14,580 - lava_callback - DEBUG - Processing node 69ccd96774bda3ec93a41558: baseline-arm-clabbe 2026-04-03 08:38:14,580 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd96774bda3ec93a41558 2026-04-03 08:38:14,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:14,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96774bda3ec93a41558 HTTP/1.1" 200 None 2026-04-03 08:38:14,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:14,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 08:38:14,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:14,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd96774bda3ec93a41558 HTTP/1.1" 200 None 2026-04-03 08:38:14,632 - lava_callback - ERROR - Error processing callback for node 69ccd96774bda3ec93a41558: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54562 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 08:38:15,257 - lava_callback - INFO - Received callback for node 69ccd96874bda3ec93a4155b INFO: 172.21.0.1:54572 - "POST /node/69ccd96874bda3ec93a4155b HTTP/1.0" 202 Accepted 2026-04-03 08:38:15,262 - lava_callback - INFO - Starting async job submit for node 69ccd96874bda3ec93a4155b 2026-04-03 08:38:15,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:15,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96874bda3ec93a4155b HTTP/1.1" 200 None 2026-04-03 08:38:15,278 - lava_callback - DEBUG - Processing node 69ccd96874bda3ec93a4155b: baseline-arm-clabbe 2026-04-03 08:38:15,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd96874bda3ec93a4155b 2026-04-03 08:38:15,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:15,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96874bda3ec93a4155b HTTP/1.1" 200 None 2026-04-03 08:38:15,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:15,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 08:38:15,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:15,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd96874bda3ec93a4155b HTTP/1.1" 200 None 2026-04-03 08:38:15,332 - lava_callback - ERROR - Error processing callback for node 69ccd96874bda3ec93a4155b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 08:38:15,919 - lava_callback - INFO - Received callback for node 69ccd96b74bda3ec93a41564 INFO: 172.21.0.1:54584 - "POST /node/69ccd96b74bda3ec93a41564 HTTP/1.0" 202 Accepted 2026-04-03 08:38:15,925 - lava_callback - INFO - Starting async job submit for node 69ccd96b74bda3ec93a41564 2026-04-03 08:38:15,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:15,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96b74bda3ec93a41564 HTTP/1.1" 200 None 2026-04-03 08:38:15,940 - lava_callback - DEBUG - Processing node 69ccd96b74bda3ec93a41564: baseline-arm-clabbe 2026-04-03 08:38:15,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd96b74bda3ec93a41564 2026-04-03 08:38:15,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:15,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd96b74bda3ec93a41564 HTTP/1.1" 200 None 2026-04-03 08:38:15,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:15,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 08:38:15,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:38:15,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd96b74bda3ec93a41564 HTTP/1.1" 200 None 2026-04-03 08:38:15,997 - lava_callback - ERROR - Error processing callback for node 69ccd96b74bda3ec93a41564: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46372 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:43:53,238 - lava_callback - INFO - Received callback for node 69cf7da3000c489abc2bfe71 INFO: 172.21.0.1:46384 - "POST /node/69cf7da3000c489abc2bfe71 HTTP/1.0" 202 Accepted 2026-04-03 08:43:53,245 - lava_callback - INFO - Starting async job submit for node 69cf7da3000c489abc2bfe71 2026-04-03 08:43:53,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:43:53,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7da3000c489abc2bfe71 HTTP/1.1" 200 None 2026-04-03 08:43:53,280 - lava_callback - DEBUG - Processing node 69cf7da3000c489abc2bfe71: baseline-x86 2026-04-03 08:43:53,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:43:53,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:43:53,626 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da3000c489abc2bfe71/log.txt.gz 2026-04-03 08:43:53,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:43:53,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:43:53,939 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da3000c489abc2bfe71/lava-logs.yaml 2026-04-03 08:43:53,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:43:54,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:43:54,082 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da3000c489abc2bfe71/lava_callback.json.gz 2026-04-03 08:43:54,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:43:54,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7da3000c489abc2bfe71 HTTP/1.1" 200 None 2026-04-03 08:43:54,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:43:54,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77d1000c489abc2bf955 HTTP/1.1" 200 None 2026-04-03 08:43:54,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:43:54,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7da3000c489abc2bfe71 HTTP/1.1" 200 None 2026-04-03 08:43:54,167 - lava_callback - INFO - Completed processing callback for node 69cf7da3000c489abc2bfe71 INFO: 172.21.0.1:46390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54812 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:44:12,033 - lava_callback - INFO - Received callback for node 69cf7da6000c489abc2bfe7a INFO: 172.21.0.1:45778 - "POST /node/69cf7da6000c489abc2bfe7a HTTP/1.0" 202 Accepted 2026-04-03 08:44:12,039 - lava_callback - INFO - Starting async job submit for node 69cf7da6000c489abc2bfe7a 2026-04-03 08:44:12,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:44:12,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7da6000c489abc2bfe7a HTTP/1.1" 200 None 2026-04-03 08:44:12,073 - lava_callback - DEBUG - Processing node 69cf7da6000c489abc2bfe7a: baseline-x86 2026-04-03 08:44:12,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:44:12,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:44:12,373 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da6000c489abc2bfe7a/log.txt.gz 2026-04-03 08:44:12,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:44:12,682 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:44:12,683 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da6000c489abc2bfe7a/lava-logs.yaml 2026-04-03 08:44:12,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:44:12,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:44:12,823 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da6000c489abc2bfe7a/lava_callback.json.gz 2026-04-03 08:44:12,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:44:12,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7da6000c489abc2bfe7a HTTP/1.1" 200 None 2026-04-03 08:44:12,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:44:12,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77d1000c489abc2bf955 HTTP/1.1" 200 None 2026-04-03 08:44:12,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:44:12,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7da6000c489abc2bfe7a HTTP/1.1" 200 None 2026-04-03 08:44:12,956 - lava_callback - INFO - Completed processing callback for node 69cf7da6000c489abc2bfe7a INFO: 172.21.0.1:45792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45796 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:44:21,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:44:21,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36784 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:44:34,907 - lava_callback - INFO - Received callback for node 69cf7da4000c489abc2bfe74 INFO: 172.21.0.1:37284 - "POST /node/69cf7da4000c489abc2bfe74 HTTP/1.0" 202 Accepted 2026-04-03 08:44:34,917 - lava_callback - INFO - Starting async job submit for node 69cf7da4000c489abc2bfe74 2026-04-03 08:44:34,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:44:34,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7da4000c489abc2bfe74 HTTP/1.1" 200 None 2026-04-03 08:44:34,978 - lava_callback - DEBUG - Processing node 69cf7da4000c489abc2bfe74: baseline-x86 2026-04-03 08:44:35,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:44:35,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:44:35,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da4000c489abc2bfe74/log.txt.gz 2026-04-03 08:44:35,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:44:35,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:44:35,882 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da4000c489abc2bfe74/lava-logs.yaml 2026-04-03 08:44:35,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:44:36,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:44:36,040 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da4000c489abc2bfe74/lava_callback.json.gz 2026-04-03 08:44:36,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:44:36,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7da4000c489abc2bfe74 HTTP/1.1" 200 None 2026-04-03 08:44:36,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:44:36,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77d1000c489abc2bf955 HTTP/1.1" 200 None 2026-04-03 08:44:36,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:44:36,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7da4000c489abc2bfe74 HTTP/1.1" 200 None 2026-04-03 08:44:36,143 - lava_callback - INFO - Completed processing callback for node 69cf7da4000c489abc2bfe74 INFO: 172.21.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37356 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:44:51,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:44:51,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33068 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:46:12,475 - lava_callback - INFO - Received callback for node 69ccdb5174bda3ec93a4175e INFO: 172.21.0.1:47660 - "POST /node/69ccdb5174bda3ec93a4175e HTTP/1.0" 202 Accepted 2026-04-03 08:46:12,482 - lava_callback - INFO - Starting async job submit for node 69ccdb5174bda3ec93a4175e 2026-04-03 08:46:12,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:46:12,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdb5174bda3ec93a4175e HTTP/1.1" 200 None 2026-04-03 08:46:12,507 - lava_callback - DEBUG - Processing node 69ccdb5174bda3ec93a4175e: baseline-arm-clabbe 2026-04-03 08:46:12,507 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdb5174bda3ec93a4175e 2026-04-03 08:46:12,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:46:12,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdb5174bda3ec93a4175e HTTP/1.1" 200 None 2026-04-03 08:46:12,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:46:12,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 08:46:12,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:46:12,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdb5174bda3ec93a4175e HTTP/1.1" 200 None 2026-04-03 08:46:12,605 - lava_callback - ERROR - Error processing callback for node 69ccdb5174bda3ec93a4175e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:47:32,607 - lava_callback - INFO - Received callback for node 69ccdb9e74bda3ec93a417cb INFO: 172.21.0.1:49672 - "POST /node/69ccdb9e74bda3ec93a417cb HTTP/1.0" 202 Accepted 2026-04-03 08:47:32,614 - lava_callback - INFO - Starting async job submit for node 69ccdb9e74bda3ec93a417cb 2026-04-03 08:47:32,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:47:32,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdb9e74bda3ec93a417cb HTTP/1.1" 200 None 2026-04-03 08:47:32,656 - lava_callback - DEBUG - Processing node 69ccdb9e74bda3ec93a417cb: baseline-arm-clabbe 2026-04-03 08:47:32,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdb9e74bda3ec93a417cb 2026-04-03 08:47:32,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:47:32,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdb9e74bda3ec93a417cb HTTP/1.1" 200 None 2026-04-03 08:47:32,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:47:32,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 08:47:32,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:47:32,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdb9e74bda3ec93a417cb HTTP/1.1" 200 None 2026-04-03 08:47:32,764 - lava_callback - ERROR - Error processing callback for node 69ccdb9e74bda3ec93a417cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44180 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:48:01,810 - lava_callback - INFO - Received callback for node 69cf7a9d000c489abc2bfd40 INFO: 172.21.0.1:57420 - "POST /node/69cf7a9d000c489abc2bfd40 HTTP/1.0" 202 Accepted 2026-04-03 08:48:01,822 - lava_callback - INFO - Starting async job submit for node 69cf7a9d000c489abc2bfd40 2026-04-03 08:48:02,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:02,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7a9d000c489abc2bfd40 HTTP/1.1" 200 None 2026-04-03 08:48:02,089 - lava_callback - DEBUG - Processing node 69cf7a9d000c489abc2bfd40: ltp-smoke 2026-04-03 08:48:02,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:02,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:02,986 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf7a9d000c489abc2bfd40/log.txt.gz 2026-04-03 08:48:03,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:03,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:03,604 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf7a9d000c489abc2bfd40/lava-logs.yaml 2026-04-03 08:48:03,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:03,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:03,865 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf7a9d000c489abc2bfd40/lava_callback.json.gz 2026-04-03 08:48:04,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:04,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7a9d000c489abc2bfd40 HTTP/1.1" 200 None 2026-04-03 08:48:04,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:04,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b7000c489abc2bf92f HTTP/1.1" 200 None 2026-04-03 08:48:04,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:04,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7a9d000c489abc2bfd40 HTTP/1.1" 200 None 2026-04-03 08:48:04,240 - lava_callback - INFO - Completed processing callback for node 69cf7a9d000c489abc2bfd40 INFO: 172.21.0.1:57432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34288 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:48:21,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:21,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 08:48:24,560 - lava_callback - INFO - Received callback for node 69cf7ec3000c489abc2bff0c INFO: 172.21.0.1:34896 - "POST /node/69cf7ec3000c489abc2bff0c HTTP/1.0" 202 Accepted 2026-04-03 08:48:24,567 - lava_callback - INFO - Starting async job submit for node 69cf7ec3000c489abc2bff0c 2026-04-03 08:48:24,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:24,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec3000c489abc2bff0c HTTP/1.1" 200 None 2026-04-03 08:48:24,649 - lava_callback - DEBUG - Processing node 69cf7ec3000c489abc2bff0c: baseline-arm64 2026-04-03 08:48:24,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:25,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:25,082 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec3000c489abc2bff0c/log.txt.gz 2026-04-03 08:48:25,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:25,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:25,495 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec3000c489abc2bff0c/lava-logs.yaml 2026-04-03 08:48:25,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:25,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:25,756 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec3000c489abc2bff0c/lava_callback.json.gz 2026-04-03 08:48:25,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:25,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec3000c489abc2bff0c HTTP/1.1" 200 None 2026-04-03 08:48:25,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:25,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:48:25,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:25,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ec3000c489abc2bff0c HTTP/1.1" 200 None 2026-04-03 08:48:25,847 - lava_callback - INFO - Completed processing callback for node 69cf7ec3000c489abc2bff0c INFO: 172.21.0.1:34906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53246 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:48:51,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:51,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33366 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:48:52,622 - lava_callback - INFO - Received callback for node 69ccdbf074bda3ec93a417fa INFO: 172.21.0.1:33376 - "POST /node/69ccdbf074bda3ec93a417fa HTTP/1.0" 202 Accepted 2026-04-03 08:48:52,628 - lava_callback - INFO - Starting async job submit for node 69ccdbf074bda3ec93a417fa 2026-04-03 08:48:52,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:52,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdbf074bda3ec93a417fa HTTP/1.1" 200 None 2026-04-03 08:48:52,646 - lava_callback - DEBUG - Processing node 69ccdbf074bda3ec93a417fa: baseline-arm-clabbe 2026-04-03 08:48:52,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdbf074bda3ec93a417fa 2026-04-03 08:48:52,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:52,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdbf074bda3ec93a417fa HTTP/1.1" 200 None 2026-04-03 08:48:52,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:52,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 08:48:52,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:52,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdbf074bda3ec93a417fa HTTP/1.1" 200 None 2026-04-03 08:48:52,704 - lava_callback - ERROR - Error processing callback for node 69ccdbf074bda3ec93a417fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33390 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:48:57,499 - lava_callback - INFO - Received callback for node 69cf7ed0000c489abc2bff69 INFO: 172.21.0.1:33402 - "POST /node/69cf7ed0000c489abc2bff69 HTTP/1.0" 202 Accepted 2026-04-03 08:48:57,507 - lava_callback - INFO - Starting async job submit for node 69cf7ed0000c489abc2bff69 2026-04-03 08:48:57,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:57,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed0000c489abc2bff69 HTTP/1.1" 200 None 2026-04-03 08:48:57,550 - lava_callback - DEBUG - Processing node 69cf7ed0000c489abc2bff69: baseline-arm64 2026-04-03 08:48:57,604 - lava_callback - INFO - Received callback for node 69cf7ee0000c489abc2bff9c 2026-04-03 08:48:57,637 - lava_callback - INFO - Starting async job submit for node 69cf7ee0000c489abc2bff9c INFO: 172.21.0.1:33414 - "POST /node/69cf7ee0000c489abc2bff9c HTTP/1.0" 202 Accepted 2026-04-03 08:48:57,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:57,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ee0000c489abc2bff9c HTTP/1.1" 200 None 2026-04-03 08:48:57,736 - lava_callback - DEBUG - Processing node 69cf7ee0000c489abc2bff9c: baseline-nfs-arm64 2026-04-03 08:48:57,809 - lava_callback - WARNING - Failed to upload log 2026-04-03 08:48:57,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:57,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:57,987 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:57,989 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee0000c489abc2bff9c/lava-logs.yaml 2026-04-03 08:48:57,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:58,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:58,007 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed0000c489abc2bff69/log.txt.gz 2026-04-03 08:48:58,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:58,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:58,177 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee0000c489abc2bff9c/lava_callback.json.gz 2026-04-03 08:48:58,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:58,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ee0000c489abc2bff9c HTTP/1.1" 200 None 2026-04-03 08:48:58,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:58,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:48:58,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:58,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ee0000c489abc2bff9c HTTP/1.1" 200 None 2026-04-03 08:48:58,248 - lava_callback - INFO - Completed processing callback for node 69cf7ee0000c489abc2bff9c 2026-04-03 08:48:58,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:58,430 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed0000c489abc2bff69/lava-logs.yaml 2026-04-03 08:48:58,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:48:58,644 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:48:58,646 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed0000c489abc2bff69/lava_callback.json.gz 2026-04-03 08:48:58,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:58,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed0000c489abc2bff69 HTTP/1.1" 200 None 2026-04-03 08:48:58,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:58,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:48:58,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:48:58,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ed0000c489abc2bff69 HTTP/1.1" 200 None 2026-04-03 08:48:58,735 - lava_callback - INFO - Completed processing callback for node 69cf7ed0000c489abc2bff69 2026-04-03 08:49:05,270 - lava_callback - INFO - Received callback for node 69cf7ec8000c489abc2bff1f INFO: 172.21.0.1:41746 - "POST /node/69cf7ec8000c489abc2bff1f HTTP/1.0" 202 Accepted 2026-04-03 08:49:05,276 - lava_callback - INFO - Starting async job submit for node 69cf7ec8000c489abc2bff1f 2026-04-03 08:49:05,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:05,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec8000c489abc2bff1f HTTP/1.1" 200 None 2026-04-03 08:49:05,324 - lava_callback - DEBUG - Processing node 69cf7ec8000c489abc2bff1f: baseline-arm64 2026-04-03 08:49:05,457 - lava_callback - WARNING - Failed to upload log 2026-04-03 08:49:05,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:05,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:05,772 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec8000c489abc2bff1f/lava-logs.yaml 2026-04-03 08:49:05,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:05,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:05,985 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec8000c489abc2bff1f/lava_callback.json.gz 2026-04-03 08:49:06,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:06,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec8000c489abc2bff1f HTTP/1.1" 200 None 2026-04-03 08:49:06,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:06,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:06,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:06,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ec8000c489abc2bff1f HTTP/1.1" 200 None 2026-04-03 08:49:06,122 - lava_callback - INFO - Completed processing callback for node 69cf7ec8000c489abc2bff1f INFO: 172.21.0.1:41760 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:49:06,945 - lava_callback - INFO - Received callback for node 69cf7ec9000c489abc2bff24 INFO: 172.21.0.1:41764 - "POST /node/69cf7ec9000c489abc2bff24 HTTP/1.0" 202 Accepted 2026-04-03 08:49:06,954 - lava_callback - INFO - Starting async job submit for node 69cf7ec9000c489abc2bff24 2026-04-03 08:49:06,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:07,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec9000c489abc2bff24 HTTP/1.1" 200 None 2026-04-03 08:49:07,015 - lava_callback - DEBUG - Processing node 69cf7ec9000c489abc2bff24: baseline-arm64 2026-04-03 08:49:07,177 - lava_callback - WARNING - Failed to upload log 2026-04-03 08:49:07,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:07,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:07,502 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec9000c489abc2bff24/lava-logs.yaml 2026-04-03 08:49:07,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:07,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:07,623 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec9000c489abc2bff24/lava_callback.json.gz 2026-04-03 08:49:07,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:07,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec9000c489abc2bff24 HTTP/1.1" 200 None 2026-04-03 08:49:07,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:07,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:07,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:07,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ec9000c489abc2bff24 HTTP/1.1" 200 None 2026-04-03 08:49:07,783 - lava_callback - INFO - Completed processing callback for node 69cf7ec9000c489abc2bff24 2026-04-03 08:49:13,416 - lava_callback - INFO - Received callback for node 69cf7ec5000c489abc2bff17 INFO: 172.21.0.1:54540 - "POST /node/69cf7ec5000c489abc2bff17 HTTP/1.0" 202 Accepted 2026-04-03 08:49:13,422 - lava_callback - INFO - Starting async job submit for node 69cf7ec5000c489abc2bff17 2026-04-03 08:49:13,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:13,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec5000c489abc2bff17 HTTP/1.1" 200 None 2026-04-03 08:49:13,480 - lava_callback - DEBUG - Processing node 69cf7ec5000c489abc2bff17: baseline-arm64 2026-04-03 08:49:13,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:13,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:13,895 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec5000c489abc2bff17/log.txt.gz 2026-04-03 08:49:14,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:14,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:14,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec5000c489abc2bff17/lava-logs.yaml 2026-04-03 08:49:14,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:14,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:14,426 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec5000c489abc2bff17/lava_callback.json.gz 2026-04-03 08:49:14,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:14,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec5000c489abc2bff17 HTTP/1.1" 200 None 2026-04-03 08:49:14,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:14,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:14,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:14,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ec5000c489abc2bff17 HTTP/1.1" 200 None 2026-04-03 08:49:14,514 - lava_callback - INFO - Completed processing callback for node 69cf7ec5000c489abc2bff17 INFO: 172.21.0.1:54550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54556 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:49:19,118 - lava_callback - INFO - Received callback for node 69cf7ecc000c489abc2bff31 INFO: 172.21.0.1:54570 - "POST /node/69cf7ecc000c489abc2bff31 HTTP/1.0" 202 Accepted 2026-04-03 08:49:19,128 - lava_callback - INFO - Starting async job submit for node 69cf7ecc000c489abc2bff31 2026-04-03 08:49:19,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:19,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ecc000c489abc2bff31 HTTP/1.1" 200 None 2026-04-03 08:49:19,166 - lava_callback - DEBUG - Processing node 69cf7ecc000c489abc2bff31: baseline-arm64 2026-04-03 08:49:19,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:20,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:20,418 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ecc000c489abc2bff31/log.txt.gz 2026-04-03 08:49:20,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:20,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:20,856 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ecc000c489abc2bff31/lava-logs.yaml 2026-04-03 08:49:20,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:20,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:21,000 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ecc000c489abc2bff31/lava_callback.json.gz 2026-04-03 08:49:21,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:21,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ecc000c489abc2bff31 HTTP/1.1" 200 None 2026-04-03 08:49:21,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:21,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:21,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:21,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:21,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 08:49:21,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ecc000c489abc2bff31 HTTP/1.1" 200 None 2026-04-03 08:49:21,138 - lava_callback - INFO - Completed processing callback for node 69cf7ecc000c489abc2bff31 INFO: 172.21.0.1:48302 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:49:27,196 - lava_callback - INFO - Received callback for node 69cf7ecf000c489abc2bff66 INFO: 172.21.0.1:48310 - "POST /node/69cf7ecf000c489abc2bff66 HTTP/1.0" 202 Accepted 2026-04-03 08:49:27,205 - lava_callback - INFO - Starting async job submit for node 69cf7ecf000c489abc2bff66 2026-04-03 08:49:27,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:27,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ecf000c489abc2bff66 HTTP/1.1" 200 None 2026-04-03 08:49:27,257 - lava_callback - DEBUG - Processing node 69cf7ecf000c489abc2bff66: baseline-arm64 2026-04-03 08:49:27,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:27,894 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:27,896 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ecf000c489abc2bff66/log.txt.gz 2026-04-03 08:49:28,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:28,471 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:28,474 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ecf000c489abc2bff66/lava-logs.yaml 2026-04-03 08:49:28,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:28,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:28,697 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ecf000c489abc2bff66/lava_callback.json.gz 2026-04-03 08:49:28,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:28,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ecf000c489abc2bff66 HTTP/1.1" 200 None 2026-04-03 08:49:28,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:28,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:28,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:28,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ecf000c489abc2bff66 HTTP/1.1" 200 None 2026-04-03 08:49:28,966 - lava_callback - INFO - Completed processing callback for node 69cf7ecf000c489abc2bff66 2026-04-03 08:49:34,098 - lava_callback - INFO - Received callback for node 69cf7ece000c489abc2bff35 INFO: 172.21.0.1:36128 - "POST /node/69cf7ece000c489abc2bff35 HTTP/1.0" 202 Accepted 2026-04-03 08:49:34,105 - lava_callback - INFO - Starting async job submit for node 69cf7ece000c489abc2bff35 2026-04-03 08:49:34,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:34,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ece000c489abc2bff35 HTTP/1.1" 200 None 2026-04-03 08:49:34,174 - lava_callback - DEBUG - Processing node 69cf7ece000c489abc2bff35: baseline-arm64 2026-04-03 08:49:34,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:34,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:34,833 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ece000c489abc2bff35/log.txt.gz 2026-04-03 08:49:35,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:35,341 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:35,343 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ece000c489abc2bff35/lava-logs.yaml 2026-04-03 08:49:35,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:35,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:35,499 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ece000c489abc2bff35/lava_callback.json.gz 2026-04-03 08:49:35,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:35,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ece000c489abc2bff35 HTTP/1.1" 200 None 2026-04-03 08:49:35,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:35,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:35,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:35,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ece000c489abc2bff35 HTTP/1.1" 200 None 2026-04-03 08:49:35,698 - lava_callback - INFO - Completed processing callback for node 69cf7ece000c489abc2bff35 INFO: 172.21.0.1:36142 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:49:40,043 - lava_callback - INFO - Received callback for node 69ccdc1174bda3ec93a41806 INFO: 172.21.0.1:39544 - "POST /node/69ccdc1174bda3ec93a41806 HTTP/1.0" 202 Accepted 2026-04-03 08:49:40,050 - lava_callback - INFO - Starting async job submit for node 69ccdc1174bda3ec93a41806 2026-04-03 08:49:40,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:40,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdc1174bda3ec93a41806 HTTP/1.1" 200 None 2026-04-03 08:49:40,066 - lava_callback - DEBUG - Processing node 69ccdc1174bda3ec93a41806: baseline-arm-baylibre 2026-04-03 08:49:40,066 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdc1174bda3ec93a41806 2026-04-03 08:49:40,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:40,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdc1174bda3ec93a41806 HTTP/1.1" 200 None 2026-04-03 08:49:40,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:40,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 08:49:40,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:40,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdc1174bda3ec93a41806 HTTP/1.1" 200 None 2026-04-03 08:49:40,174 - lava_callback - ERROR - Error processing callback for node 69ccdc1174bda3ec93a41806: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 08:49:44,397 - lava_callback - INFO - Received callback for node 69cf7edd000c489abc2bff96 INFO: 172.21.0.1:39556 - "POST /node/69cf7edd000c489abc2bff96 HTTP/1.0" 202 Accepted 2026-04-03 08:49:44,404 - lava_callback - INFO - Starting async job submit for node 69cf7edd000c489abc2bff96 2026-04-03 08:49:44,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:44,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7edd000c489abc2bff96 HTTP/1.1" 200 None 2026-04-03 08:49:44,452 - lava_callback - DEBUG - Processing node 69cf7edd000c489abc2bff96: baseline-nfs-arm64 2026-04-03 08:49:44,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:39564 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:49:48,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:48,560 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edd000c489abc2bff96/log.txt.gz 2026-04-03 08:49:48,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:49,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:49,143 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edd000c489abc2bff96/lava-logs.yaml 2026-04-03 08:49:49,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:49,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:49,350 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edd000c489abc2bff96/lava_callback.json.gz 2026-04-03 08:49:49,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:49,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7edd000c489abc2bff96 HTTP/1.1" 200 None 2026-04-03 08:49:49,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:49,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:49,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:49,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7edd000c489abc2bff96 HTTP/1.1" 200 None 2026-04-03 08:49:49,435 - lava_callback - INFO - Completed processing callback for node 69cf7edd000c489abc2bff96 2026-04-03 08:49:51,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:51,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:36004 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:49:53,208 - lava_callback - INFO - Received callback for node 69cf7ed5000c489abc2bff7c INFO: 172.21.0.1:36008 - "POST /node/69cf7ed5000c489abc2bff7c HTTP/1.0" 202 Accepted 2026-04-03 08:49:53,218 - lava_callback - INFO - Starting async job submit for node 69cf7ed5000c489abc2bff7c 2026-04-03 08:49:53,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:53,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed5000c489abc2bff7c HTTP/1.1" 200 None 2026-04-03 08:49:53,277 - lava_callback - DEBUG - Processing node 69cf7ed5000c489abc2bff7c: baseline-arm64 2026-04-03 08:49:53,516 - lava_callback - INFO - Received callback for node 69cf7ec4000c489abc2bff14 INFO: 172.21.0.1:36014 - "POST /node/69cf7ec4000c489abc2bff14 HTTP/1.0" 202 Accepted 2026-04-03 08:49:53,529 - lava_callback - INFO - Starting async job submit for node 69cf7ec4000c489abc2bff14 2026-04-03 08:49:53,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:53,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec4000c489abc2bff14 HTTP/1.1" 200 None 2026-04-03 08:49:53,687 - lava_callback - DEBUG - Processing node 69cf7ec4000c489abc2bff14: baseline-arm64 2026-04-03 08:49:53,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:54,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:54,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:54,033 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed5000c489abc2bff7c/log.txt.gz 2026-04-03 08:49:54,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:54,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:54,404 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec4000c489abc2bff14/log.txt.gz 2026-04-03 08:49:54,511 - lava_callback - INFO - Received callback for node 69cf7edb000c489abc2bff8c 2026-04-03 08:49:54,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:54,535 - lava_callback - INFO - Starting async job submit for node 69cf7edb000c489abc2bff8c INFO: 172.21.0.1:36024 - "POST /node/69cf7edb000c489abc2bff8c HTTP/1.0" 202 Accepted 2026-04-03 08:49:54,587 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed5000c489abc2bff7c/lava-logs.yaml 2026-04-03 08:49:54,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:54,595 - lava_callback - INFO - Received callback for node 69cf7edf000c489abc2bff99 2026-04-03 08:49:54,609 - lava_callback - INFO - Starting async job submit for node 69cf7edf000c489abc2bff99 INFO: 172.21.0.1:36034 - "POST /node/69cf7edf000c489abc2bff99 HTTP/1.0" 202 Accepted 2026-04-03 08:49:54,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:54,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:54,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:54,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7edb000c489abc2bff8c HTTP/1.1" 200 None 2026-04-03 08:49:54,702 - lava_callback - DEBUG - Processing node 69cf7edb000c489abc2bff8c: baseline-nfs-arm64 2026-04-03 08:49:54,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7edf000c489abc2bff99 HTTP/1.1" 200 None 2026-04-03 08:49:54,816 - lava_callback - DEBUG - Processing node 69cf7edf000c489abc2bff99: baseline-nfs-arm64 2026-04-03 08:49:54,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:54,982 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed5000c489abc2bff7c/lava_callback.json.gz 2026-04-03 08:49:55,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:55,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:55,328 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec4000c489abc2bff14/lava-logs.yaml 2026-04-03 08:49:55,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:55,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed5000c489abc2bff7c HTTP/1.1" 200 None 2026-04-03 08:49:55,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:55,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:55,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:55,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:55,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ed5000c489abc2bff7c HTTP/1.1" 200 None 2026-04-03 08:49:55,822 - lava_callback - INFO - Completed processing callback for node 69cf7ed5000c489abc2bff7c 2026-04-03 08:49:55,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:55,912 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec4000c489abc2bff14/lava_callback.json.gz 2026-04-03 08:49:55,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:56,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec4000c489abc2bff14 HTTP/1.1" 200 None 2026-04-03 08:49:56,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:56,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:56,163 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edb000c489abc2bff8c/log.txt.gz 2026-04-03 08:49:56,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:56,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:56,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:56,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ec4000c489abc2bff14 HTTP/1.1" 200 None 2026-04-03 08:49:56,291 - lava_callback - INFO - Completed processing callback for node 69cf7ec4000c489abc2bff14 INFO: 172.21.0.1:36044 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:49:56,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:56,678 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:56,680 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edf000c489abc2bff99/log.txt.gz 2026-04-03 08:49:56,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:56,918 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edb000c489abc2bff8c/lava-logs.yaml 2026-04-03 08:49:56,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:57,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:57,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:57,201 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edb000c489abc2bff8c/lava_callback.json.gz 2026-04-03 08:49:57,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:57,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7edb000c489abc2bff8c HTTP/1.1" 200 None 2026-04-03 08:49:57,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:57,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:57,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:57,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7edb000c489abc2bff8c HTTP/1.1" 200 None 2026-04-03 08:49:57,298 - lava_callback - INFO - Completed processing callback for node 69cf7edb000c489abc2bff8c 2026-04-03 08:49:57,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:57,325 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edf000c489abc2bff99/lava-logs.yaml 2026-04-03 08:49:57,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:49:57,646 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:49:57,649 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edf000c489abc2bff99/lava_callback.json.gz 2026-04-03 08:49:57,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:57,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7edf000c489abc2bff99 HTTP/1.1" 200 None 2026-04-03 08:49:57,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:57,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:49:57,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:49:57,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7edf000c489abc2bff99 HTTP/1.1" 200 None 2026-04-03 08:49:57,748 - lava_callback - INFO - Completed processing callback for node 69cf7edf000c489abc2bff99 2026-04-03 08:50:00,288 - lava_callback - INFO - Received callback for node 69cf7f04000c489abc2c005f INFO: 172.21.0.1:40230 - "POST /node/69cf7f04000c489abc2c005f HTTP/1.0" 202 Accepted 2026-04-03 08:50:00,295 - lava_callback - INFO - Starting async job submit for node 69cf7f04000c489abc2c005f 2026-04-03 08:50:00,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:00,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f04000c489abc2c005f HTTP/1.1" 200 None 2026-04-03 08:50:00,343 - lava_callback - DEBUG - Processing node 69cf7f04000c489abc2c005f: baseline-arm64-kontron 2026-04-03 08:50:00,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:00,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:00,881 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf7f04000c489abc2c005f/log.txt.gz 2026-04-03 08:50:01,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:01,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:01,517 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf7f04000c489abc2c005f/lava-logs.yaml 2026-04-03 08:50:01,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:01,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:01,746 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf7f04000c489abc2c005f/lava_callback.json.gz 2026-04-03 08:50:01,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:01,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f04000c489abc2c005f HTTP/1.1" 200 None 2026-04-03 08:50:01,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:01,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:01,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:01,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f04000c489abc2c005f HTTP/1.1" 200 None 2026-04-03 08:50:01,882 - lava_callback - INFO - Completed processing callback for node 69cf7f04000c489abc2c005f 2026-04-03 08:50:02,980 - lava_callback - INFO - Received callback for node 69cf7ed4000c489abc2bff72 INFO: 172.21.0.1:40242 - "POST /node/69cf7ed4000c489abc2bff72 HTTP/1.0" 202 Accepted 2026-04-03 08:50:02,992 - lava_callback - INFO - Starting async job submit for node 69cf7ed4000c489abc2bff72 2026-04-03 08:50:03,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:03,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed4000c489abc2bff72 HTTP/1.1" 200 None 2026-04-03 08:50:03,061 - lava_callback - DEBUG - Processing node 69cf7ed4000c489abc2bff72: baseline-arm64 2026-04-03 08:50:03,465 - lava_callback - INFO - Received callback for node 69cf7f06000c489abc2c0062 INFO: 172.21.0.1:40248 - "POST /node/69cf7f06000c489abc2c0062 HTTP/1.0" 202 Accepted 2026-04-03 08:50:03,481 - lava_callback - INFO - Starting async job submit for node 69cf7f06000c489abc2c0062 2026-04-03 08:50:03,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:03,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:03,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f06000c489abc2c0062 HTTP/1.1" 200 None 2026-04-03 08:50:03,666 - lava_callback - DEBUG - Processing node 69cf7f06000c489abc2c0062: baseline-arm64-pengutronix 2026-04-03 08:50:03,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:03,992 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed4000c489abc2bff72/log.txt.gz 2026-04-03 08:50:04,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:04,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:04,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:04,336 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf7f06000c489abc2c0062/log.txt.gz 2026-04-03 08:50:04,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:04,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:04,659 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed4000c489abc2bff72/lava-logs.yaml 2026-04-03 08:50:04,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:04,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:04,812 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf7f06000c489abc2c0062/lava-logs.yaml 2026-04-03 08:50:04,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:04,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:04,885 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed4000c489abc2bff72/lava_callback.json.gz 2026-04-03 08:50:04,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:04,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed4000c489abc2bff72 HTTP/1.1" 200 None 2026-04-03 08:50:04,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:04,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:05,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:05,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ed4000c489abc2bff72 HTTP/1.1" 200 None 2026-04-03 08:50:05,045 - lava_callback - INFO - Completed processing callback for node 69cf7ed4000c489abc2bff72 2026-04-03 08:50:05,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:05,131 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf7f06000c489abc2c0062/lava_callback.json.gz 2026-04-03 08:50:05,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:05,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f06000c489abc2c0062 HTTP/1.1" 200 None 2026-04-03 08:50:05,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:05,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:05,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:05,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f06000c489abc2c0062 HTTP/1.1" 200 None 2026-04-03 08:50:05,304 - lava_callback - INFO - Completed processing callback for node 69cf7f06000c489abc2c0062 2026-04-03 08:50:05,784 - lava_callback - INFO - Received callback for node 69cf7f09000c489abc2c006b INFO: 172.21.0.1:40254 - "POST /node/69cf7f09000c489abc2c006b HTTP/1.0" 202 Accepted 2026-04-03 08:50:05,792 - lava_callback - INFO - Starting async job submit for node 69cf7f09000c489abc2c006b 2026-04-03 08:50:05,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:05,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f09000c489abc2c006b HTTP/1.1" 200 None 2026-04-03 08:50:05,864 - lava_callback - DEBUG - Processing node 69cf7f09000c489abc2c006b: baseline-arm64-foundriesio 2026-04-03 08:50:06,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40266 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:50:06,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:06,572 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf7f09000c489abc2c006b/log.txt.gz 2026-04-03 08:50:06,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:07,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:07,187 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf7f09000c489abc2c006b/lava-logs.yaml 2026-04-03 08:50:07,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:07,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:07,433 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf7f09000c489abc2c006b/lava_callback.json.gz 2026-04-03 08:50:07,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:07,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f09000c489abc2c006b HTTP/1.1" 200 None 2026-04-03 08:50:07,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:07,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:07,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:07,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f09000c489abc2c006b HTTP/1.1" 200 None 2026-04-03 08:50:07,581 - lava_callback - INFO - Completed processing callback for node 69cf7f09000c489abc2c006b 2026-04-03 08:50:09,795 - lava_callback - INFO - Received callback for node 69cf7f08000c489abc2c0068 INFO: 172.21.0.1:38898 - "POST /node/69cf7f08000c489abc2c0068 HTTP/1.0" 202 Accepted 2026-04-03 08:50:09,805 - lava_callback - INFO - Starting async job submit for node 69cf7f08000c489abc2c0068 2026-04-03 08:50:09,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:09,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f08000c489abc2c0068 HTTP/1.1" 200 None 2026-04-03 08:50:09,856 - lava_callback - DEBUG - Processing node 69cf7f08000c489abc2c0068: baseline-arm64-foundriesio 2026-04-03 08:50:10,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:10,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:10,275 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf7f08000c489abc2c0068/log.txt.gz 2026-04-03 08:50:10,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:10,693 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:10,696 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf7f08000c489abc2c0068/lava-logs.yaml 2026-04-03 08:50:10,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:10,967 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:10,969 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf7f08000c489abc2c0068/lava_callback.json.gz 2026-04-03 08:50:11,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:11,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f08000c489abc2c0068 HTTP/1.1" 200 None 2026-04-03 08:50:11,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:11,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:11,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:11,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f08000c489abc2c0068 HTTP/1.1" 200 None 2026-04-03 08:50:11,072 - lava_callback - INFO - Completed processing callback for node 69cf7f08000c489abc2c0068 2026-04-03 08:50:11,146 - lava_callback - INFO - Received callback for node 69cf7f3b000c489abc2c00fc INFO: 172.21.0.1:38906 - "POST /node/69cf7f3b000c489abc2c00fc HTTP/1.0" 202 Accepted 2026-04-03 08:50:11,160 - lava_callback - INFO - Starting async job submit for node 69cf7f3b000c489abc2c00fc 2026-04-03 08:50:11,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:11,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3b000c489abc2c00fc HTTP/1.1" 200 None 2026-04-03 08:50:11,230 - lava_callback - DEBUG - Processing node 69cf7f3b000c489abc2c00fc: baseline-arm 2026-04-03 08:50:11,240 - lava_callback - WARNING - Failed to upload log 2026-04-03 08:50:11,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:11,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:11,437 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3b000c489abc2c00fc/lava-logs.yaml 2026-04-03 08:50:11,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:11,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:11,626 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3b000c489abc2c00fc/lava_callback.json.gz 2026-04-03 08:50:11,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:11,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3b000c489abc2c00fc HTTP/1.1" 200 None 2026-04-03 08:50:11,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:11,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:50:11,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:11,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f3b000c489abc2c00fc HTTP/1.1" 200 None 2026-04-03 08:50:11,731 - lava_callback - INFO - Completed processing callback for node 69cf7f3b000c489abc2c00fc 2026-04-03 08:50:14,096 - lava_callback - INFO - Received callback for node 69cf7f07000c489abc2c0065 INFO: 172.21.0.1:38912 - "POST /node/69cf7f07000c489abc2c0065 HTTP/1.0" 202 Accepted 2026-04-03 08:50:14,105 - lava_callback - INFO - Starting async job submit for node 69cf7f07000c489abc2c0065 2026-04-03 08:50:14,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:14,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f07000c489abc2c0065 HTTP/1.1" 200 None 2026-04-03 08:50:14,145 - lava_callback - DEBUG - Processing node 69cf7f07000c489abc2c0065: baseline-arm64-pengutronix 2026-04-03 08:50:14,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:14,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:14,592 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf7f07000c489abc2c0065/log.txt.gz 2026-04-03 08:50:14,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:15,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:15,007 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf7f07000c489abc2c0065/lava-logs.yaml 2026-04-03 08:50:15,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:38914 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 08:50:15,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:15,175 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf7f07000c489abc2c0065/lava_callback.json.gz 2026-04-03 08:50:15,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:15,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f07000c489abc2c0065 HTTP/1.1" 200 None 2026-04-03 08:50:15,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:15,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:15,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:15,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f07000c489abc2c0065 HTTP/1.1" 200 None 2026-04-03 08:50:15,357 - lava_callback - INFO - Completed processing callback for node 69cf7f07000c489abc2c0065 INFO: 172.21.0.1:38924 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:50:17,241 - lava_callback - INFO - Received callback for node 69cf7ee6000c489abc2bffac INFO: 172.21.0.1:38928 - "POST /node/69cf7ee6000c489abc2bffac HTTP/1.0" 202 Accepted 2026-04-03 08:50:17,251 - lava_callback - INFO - Starting async job submit for node 69cf7ee6000c489abc2bffac 2026-04-03 08:50:17,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:17,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ee6000c489abc2bffac HTTP/1.1" 200 None 2026-04-03 08:50:17,316 - lava_callback - DEBUG - Processing node 69cf7ee6000c489abc2bffac: baseline-nfs-arm64 2026-04-03 08:50:17,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:18,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:18,057 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee6000c489abc2bffac/log.txt.gz 2026-04-03 08:50:18,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:18,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:18,657 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee6000c489abc2bffac/lava-logs.yaml 2026-04-03 08:50:18,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:19,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:19,155 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee6000c489abc2bffac/lava_callback.json.gz 2026-04-03 08:50:19,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:19,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ee6000c489abc2bffac HTTP/1.1" 200 None 2026-04-03 08:50:19,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:19,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:19,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:19,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ee6000c489abc2bffac HTTP/1.1" 200 None 2026-04-03 08:50:19,250 - lava_callback - INFO - Completed processing callback for node 69cf7ee6000c489abc2bffac 2026-04-03 08:50:20,308 - lava_callback - INFO - Received callback for node 69cf7f03000c489abc2c0028 INFO: 172.21.0.1:47836 - "POST /node/69cf7f03000c489abc2c0028 HTTP/1.0" 202 Accepted 2026-04-03 08:50:20,315 - lava_callback - INFO - Starting async job submit for node 69cf7f03000c489abc2c0028 2026-04-03 08:50:20,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:20,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f03000c489abc2c0028 HTTP/1.1" 200 None 2026-04-03 08:50:20,371 - lava_callback - DEBUG - Processing node 69cf7f03000c489abc2c0028: baseline-arm64-kontron 2026-04-03 08:50:20,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:20,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:20,960 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf7f03000c489abc2c0028/log.txt.gz 2026-04-03 08:50:21,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:21,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:21,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-03 08:50:21,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:21,388 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf7f03000c489abc2c0028/lava-logs.yaml 2026-04-03 08:50:21,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:21,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:21,547 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf7f03000c489abc2c0028/lava_callback.json.gz 2026-04-03 08:50:21,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:21,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f03000c489abc2c0028 HTTP/1.1" 200 None 2026-04-03 08:50:21,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:21,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:21,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:21,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f03000c489abc2c0028 HTTP/1.1" 200 None 2026-04-03 08:50:21,632 - lava_callback - INFO - Completed processing callback for node 69cf7f03000c489abc2c0028 2026-04-03 08:50:22,913 - lava_callback - INFO - Received callback for node 69cf7ef5000c489abc2bfffc INFO: 172.21.0.1:47844 - "POST /node/69cf7ef5000c489abc2bfffc HTTP/1.0" 202 Accepted 2026-04-03 08:50:22,923 - lava_callback - INFO - Starting async job submit for node 69cf7ef5000c489abc2bfffc 2026-04-03 08:50:22,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:23,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef5000c489abc2bfffc HTTP/1.1" 200 None 2026-04-03 08:50:23,023 - lava_callback - DEBUG - Processing node 69cf7ef5000c489abc2bfffc: baseline-arm64-broonie 2026-04-03 08:50:23,358 - lava_callback - INFO - Received callback for node 69cf7ee4000c489abc2bffa5 INFO: 172.21.0.1:47852 - "POST /node/69cf7ee4000c489abc2bffa5 HTTP/1.0" 202 Accepted 2026-04-03 08:50:23,379 - lava_callback - INFO - Starting async job submit for node 69cf7ee4000c489abc2bffa5 2026-04-03 08:50:23,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:23,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ee4000c489abc2bffa5 HTTP/1.1" 200 None 2026-04-03 08:50:23,517 - lava_callback - DEBUG - Processing node 69cf7ee4000c489abc2bffa5: baseline-nfs-arm64 2026-04-03 08:50:23,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:24,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:24,118 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef5000c489abc2bfffc/log.txt.gz 2026-04-03 08:50:24,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:24,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:24,828 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:24,831 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee4000c489abc2bffa5/log.txt.gz 2026-04-03 08:50:24,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:24,938 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef5000c489abc2bfffc/lava-logs.yaml 2026-04-03 08:50:25,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:25,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:25,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:25,214 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef5000c489abc2bfffc/lava_callback.json.gz 2026-04-03 08:50:25,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:25,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef5000c489abc2bfffc HTTP/1.1" 200 None 2026-04-03 08:50:25,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:25,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:25,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:25,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ef5000c489abc2bfffc HTTP/1.1" 200 None 2026-04-03 08:50:25,339 - lava_callback - INFO - Completed processing callback for node 69cf7ef5000c489abc2bfffc 2026-04-03 08:50:25,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:25,508 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee4000c489abc2bffa5/lava-logs.yaml 2026-04-03 08:50:25,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:26,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:26,090 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee4000c489abc2bffa5/lava_callback.json.gz 2026-04-03 08:50:26,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:26,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ee4000c489abc2bffa5 HTTP/1.1" 200 None 2026-04-03 08:50:26,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:26,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:26,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:26,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ee4000c489abc2bffa5 HTTP/1.1" 200 None 2026-04-03 08:50:26,266 - lava_callback - INFO - Completed processing callback for node 69cf7ee4000c489abc2bffa5 INFO: 172.21.0.1:47862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37840 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:50:38,963 - lava_callback - INFO - Received callback for node 69cf7ecb000c489abc2bff2e INFO: 172.21.0.1:37846 - "POST /node/69cf7ecb000c489abc2bff2e HTTP/1.0" 202 Accepted 2026-04-03 08:50:38,969 - lava_callback - INFO - Starting async job submit for node 69cf7ecb000c489abc2bff2e 2026-04-03 08:50:39,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:39,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ecb000c489abc2bff2e HTTP/1.1" 200 None 2026-04-03 08:50:39,017 - lava_callback - DEBUG - Processing node 69cf7ecb000c489abc2bff2e: baseline-arm64 2026-04-03 08:50:39,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:39,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:39,541 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ecb000c489abc2bff2e/log.txt.gz 2026-04-03 08:50:39,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:39,892 - lava_callback - INFO - Received callback for node 69cf7f3c000c489abc2c00ff INFO: 172.21.0.1:46482 - "POST /node/69cf7f3c000c489abc2c00ff HTTP/1.0" 202 Accepted 2026-04-03 08:50:39,950 - lava_callback - INFO - Starting async job submit for node 69cf7f3c000c489abc2c00ff 2026-04-03 08:50:40,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:40,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3c000c489abc2c00ff HTTP/1.1" 200 None 2026-04-03 08:50:40,033 - lava_callback - DEBUG - Processing node 69cf7f3c000c489abc2c00ff: baseline-arm 2026-04-03 08:50:40,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:40,177 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ecb000c489abc2bff2e/lava-logs.yaml 2026-04-03 08:50:40,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:40,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:40,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:40,462 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3c000c489abc2c00ff/log.txt.gz 2026-04-03 08:50:40,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:40,526 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ecb000c489abc2bff2e/lava_callback.json.gz 2026-04-03 08:50:40,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:40,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:40,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ecb000c489abc2bff2e HTTP/1.1" 200 None 2026-04-03 08:50:40,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:40,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:40,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:40,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ecb000c489abc2bff2e HTTP/1.1" 200 None 2026-04-03 08:50:40,645 - lava_callback - INFO - Completed processing callback for node 69cf7ecb000c489abc2bff2e 2026-04-03 08:50:40,723 - lava_callback - INFO - Received callback for node 69cf7f3f000c489abc2c0110 INFO: 172.21.0.1:46498 - "POST /node/69cf7f3f000c489abc2c0110 HTTP/1.0" 202 Accepted 2026-04-03 08:50:40,771 - lava_callback - INFO - Starting async job submit for node 69cf7f3f000c489abc2c0110 2026-04-03 08:50:40,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:40,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:40,797 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3c000c489abc2c00ff/lava-logs.yaml 2026-04-03 08:50:40,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:40,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3f000c489abc2c0110 HTTP/1.1" 200 None 2026-04-03 08:50:40,826 - lava_callback - DEBUG - Processing node 69cf7f3f000c489abc2c0110: baseline-arm 2026-04-03 08:50:41,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:41,313 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:41,315 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3c000c489abc2c00ff/lava_callback.json.gz 2026-04-03 08:50:41,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:41,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3c000c489abc2c00ff HTTP/1.1" 200 None 2026-04-03 08:50:41,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:41,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:50:41,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:41,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f3c000c489abc2c00ff HTTP/1.1" 200 None 2026-04-03 08:50:41,437 - lava_callback - INFO - Completed processing callback for node 69cf7f3c000c489abc2c00ff 2026-04-03 08:50:41,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:41,482 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3f000c489abc2c0110/log.txt.gz 2026-04-03 08:50:41,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:41,764 - lava_callback - INFO - Received callback for node 69cf7eee000c489abc2bffe8 INFO: 172.21.0.1:46512 - "POST /node/69cf7eee000c489abc2bffe8 HTTP/1.0" 202 Accepted 2026-04-03 08:50:41,771 - lava_callback - INFO - Starting async job submit for node 69cf7eee000c489abc2bffe8 2026-04-03 08:50:41,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:41,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7eee000c489abc2bffe8 HTTP/1.1" 200 None 2026-04-03 08:50:41,854 - lava_callback - DEBUG - Processing node 69cf7eee000c489abc2bffe8: baseline-arm64-broonie 2026-04-03 08:50:41,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:41,904 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3f000c489abc2c0110/lava-logs.yaml 2026-04-03 08:50:41,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:42,106 - lava_callback - INFO - Received callback for node 69cf7eff000c489abc2c001d INFO: 172.21.0.1:46516 - "POST /node/69cf7eff000c489abc2c001d HTTP/1.0" 202 Accepted 2026-04-03 08:50:42,132 - lava_callback - INFO - Starting async job submit for node 69cf7eff000c489abc2c001d 2026-04-03 08:50:42,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:42,186 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3f000c489abc2c0110/lava_callback.json.gz 2026-04-03 08:50:42,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:42,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:42,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7eff000c489abc2c001d HTTP/1.1" 200 None 2026-04-03 08:50:42,364 - lava_callback - DEBUG - Processing node 69cf7eff000c489abc2c001d: baseline-arm64-clabbe 2026-04-03 08:50:42,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3f000c489abc2c0110 HTTP/1.1" 200 None 2026-04-03 08:50:42,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:42,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:42,475 - lava_callback - INFO - Received callback for node 69cf7ef9000c489abc2c0005 INFO: 172.21.0.1:46528 - "POST /node/69cf7ef9000c489abc2c0005 HTTP/1.0" 202 Accepted 2026-04-03 08:50:42,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:50:42,492 - lava_callback - INFO - Starting async job submit for node 69cf7ef9000c489abc2c0005 2026-04-03 08:50:42,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:42,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:42,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f3f000c489abc2c0110 HTTP/1.1" 200 None 2026-04-03 08:50:42,659 - lava_callback - INFO - Completed processing callback for node 69cf7f3f000c489abc2c0110 2026-04-03 08:50:42,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef9000c489abc2c0005 HTTP/1.1" 200 None 2026-04-03 08:50:42,721 - lava_callback - DEBUG - Processing node 69cf7ef9000c489abc2c0005: baseline-arm64-broonie 2026-04-03 08:50:43,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:43,133 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7eee000c489abc2bffe8/log.txt.gz 2026-04-03 08:50:43,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:43,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:43,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:44,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:44,013 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cf7eff000c489abc2c001d/log.txt.gz 2026-04-03 08:50:44,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:44,058 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef9000c489abc2c0005/log.txt.gz 2026-04-03 08:50:44,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:44,184 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7eee000c489abc2bffe8/lava-logs.yaml 2026-04-03 08:50:44,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:44,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:44,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:44,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:44,578 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7eee000c489abc2bffe8/lava_callback.json.gz 2026-04-03 08:50:44,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:44,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7eee000c489abc2bffe8 HTTP/1.1" 200 None 2026-04-03 08:50:44,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:44,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:44,688 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef9000c489abc2c0005/lava-logs.yaml 2026-04-03 08:50:44,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:44,705 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cf7eff000c489abc2c001d/lava-logs.yaml 2026-04-03 08:50:44,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:44,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:44,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:44,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:50:44,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7eee000c489abc2bffe8 HTTP/1.1" 200 None 2026-04-03 08:50:44,784 - lava_callback - INFO - Completed processing callback for node 69cf7eee000c489abc2bffe8 2026-04-03 08:50:45,394 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:45,396 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef9000c489abc2c0005/lava_callback.json.gz 2026-04-03 08:50:45,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:50:45,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:45,447 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cf7eff000c489abc2c001d/lava_callback.json.gz 2026-04-03 08:50:45,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:45,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef9000c489abc2c0005 HTTP/1.1" 200 None 2026-04-03 08:50:45,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:45,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7eff000c489abc2c001d HTTP/1.1" 200 None 2026-04-03 08:50:45,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:45,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:45,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:45,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:50:45,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:45,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ef9000c489abc2c0005 HTTP/1.1" 200 None 2026-04-03 08:50:45,589 - lava_callback - INFO - Completed processing callback for node 69cf7ef9000c489abc2c0005 2026-04-03 08:50:45,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7eff000c489abc2c001d HTTP/1.1" 200 None 2026-04-03 08:50:45,636 - lava_callback - INFO - Completed processing callback for node 69cf7eff000c489abc2c001d INFO: 172.21.0.1:46542 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:50:51,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:51,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:60834 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:50:52,746 - lava_callback - INFO - Received callback for node 69ccdc6074bda3ec93a41835 INFO: 172.21.0.1:60838 - "POST /node/69ccdc6074bda3ec93a41835 HTTP/1.0" 202 Accepted 2026-04-03 08:50:52,755 - lava_callback - INFO - Starting async job submit for node 69ccdc6074bda3ec93a41835 2026-04-03 08:50:52,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:52,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdc6074bda3ec93a41835 HTTP/1.1" 200 None 2026-04-03 08:50:52,779 - lava_callback - DEBUG - Processing node 69ccdc6074bda3ec93a41835: baseline-arm-clabbe 2026-04-03 08:50:52,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdc6074bda3ec93a41835 2026-04-03 08:50:52,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:52,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdc6074bda3ec93a41835 HTTP/1.1" 200 None 2026-04-03 08:50:52,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:52,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 08:50:52,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:50:52,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdc6074bda3ec93a41835 HTTP/1.1" 200 None 2026-04-03 08:50:52,856 - lava_callback - ERROR - Error processing callback for node 69ccdc6074bda3ec93a41835: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60840 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:51:03,702 - lava_callback - INFO - Received callback for node 69cf7ef1000c489abc2bffee INFO: 172.21.0.1:42150 - "POST /node/69cf7ef1000c489abc2bffee HTTP/1.0" 202 Accepted 2026-04-03 08:51:03,712 - lava_callback - INFO - Starting async job submit for node 69cf7ef1000c489abc2bffee 2026-04-03 08:51:03,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:03,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef1000c489abc2bffee HTTP/1.1" 200 None 2026-04-03 08:51:03,756 - lava_callback - DEBUG - Processing node 69cf7ef1000c489abc2bffee: baseline-arm64-broonie 2026-04-03 08:51:04,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:04,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:04,550 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef1000c489abc2bffee/log.txt.gz 2026-04-03 08:51:04,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:05,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:05,036 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef1000c489abc2bffee/lava-logs.yaml 2026-04-03 08:51:05,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:05,152 - lava_callback - INFO - Received callback for node 69cf7ef2000c489abc2bfff1 INFO: 172.21.0.1:42164 - "POST /node/69cf7ef2000c489abc2bfff1 HTTP/1.0" 202 Accepted 2026-04-03 08:51:05,162 - lava_callback - INFO - Starting async job submit for node 69cf7ef2000c489abc2bfff1 2026-04-03 08:51:05,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:05,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef2000c489abc2bfff1 HTTP/1.1" 200 None 2026-04-03 08:51:05,225 - lava_callback - DEBUG - Processing node 69cf7ef2000c489abc2bfff1: baseline-arm64-broonie 2026-04-03 08:51:05,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:05,262 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef1000c489abc2bffee/lava_callback.json.gz 2026-04-03 08:51:05,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:05,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef1000c489abc2bffee HTTP/1.1" 200 None 2026-04-03 08:51:05,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:05,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:51:05,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:05,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ef1000c489abc2bffee HTTP/1.1" 200 None 2026-04-03 08:51:05,645 - lava_callback - INFO - Completed processing callback for node 69cf7ef1000c489abc2bffee 2026-04-03 08:51:06,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:06,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:06,348 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef2000c489abc2bfff1/log.txt.gz INFO: 172.21.0.1:42168 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:51:06,609 - lava_callback - INFO - Received callback for node 69cf7ef6000c489abc2bffff INFO: 172.21.0.1:42176 - "POST /node/69cf7ef6000c489abc2bffff HTTP/1.0" 202 Accepted 2026-04-03 08:51:06,633 - lava_callback - INFO - Starting async job submit for node 69cf7ef6000c489abc2bffff 2026-04-03 08:51:06,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:06,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:06,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef6000c489abc2bffff HTTP/1.1" 200 None 2026-04-03 08:51:06,771 - lava_callback - DEBUG - Processing node 69cf7ef6000c489abc2bffff: baseline-arm64-broonie 2026-04-03 08:51:07,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:07,355 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef2000c489abc2bfff1/lava-logs.yaml 2026-04-03 08:51:07,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:07,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:07,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:07,694 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef2000c489abc2bfff1/lava_callback.json.gz 2026-04-03 08:51:07,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:07,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:07,730 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef6000c489abc2bffff/log.txt.gz 2026-04-03 08:51:07,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef2000c489abc2bfff1 HTTP/1.1" 200 None 2026-04-03 08:51:07,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:07,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:51:07,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:07,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:07,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ef2000c489abc2bfff1 HTTP/1.1" 200 None 2026-04-03 08:51:07,989 - lava_callback - INFO - Completed processing callback for node 69cf7ef2000c489abc2bfff1 2026-04-03 08:51:08,036 - lava_callback - INFO - Received callback for node 69cf7efa000c489abc2c000e INFO: 172.21.0.1:42178 - "POST /node/69cf7efa000c489abc2c000e HTTP/1.0" 202 Accepted 2026-04-03 08:51:08,052 - lava_callback - INFO - Starting async job submit for node 69cf7efa000c489abc2c000e 2026-04-03 08:51:08,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:08,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7efa000c489abc2c000e HTTP/1.1" 200 None 2026-04-03 08:51:08,157 - lava_callback - DEBUG - Processing node 69cf7efa000c489abc2c000e: baseline-arm64-broonie 2026-04-03 08:51:08,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:08,391 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef6000c489abc2bffff/lava-logs.yaml 2026-04-03 08:51:08,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:08,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:08,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:08,852 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef6000c489abc2bffff/lava_callback.json.gz 2026-04-03 08:51:08,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:08,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef6000c489abc2bffff HTTP/1.1" 200 None 2026-04-03 08:51:08,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:08,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:51:08,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:08,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ef6000c489abc2bffff HTTP/1.1" 200 None 2026-04-03 08:51:08,963 - lava_callback - INFO - Completed processing callback for node 69cf7ef6000c489abc2bffff 2026-04-03 08:51:09,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:09,062 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7efa000c489abc2c000e/log.txt.gz 2026-04-03 08:51:09,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:09,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:09,675 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7efa000c489abc2c000e/lava-logs.yaml 2026-04-03 08:51:09,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:09,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:09,948 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7efa000c489abc2c000e/lava_callback.json.gz 2026-04-03 08:51:09,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:10,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7efa000c489abc2c000e HTTP/1.1" 200 None 2026-04-03 08:51:10,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:10,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:51:10,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:10,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7efa000c489abc2c000e HTTP/1.1" 200 None 2026-04-03 08:51:10,109 - lava_callback - INFO - Completed processing callback for node 69cf7efa000c489abc2c000e INFO: 172.21.0.1:60514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60528 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:51:20,074 - lava_callback - INFO - Received callback for node 69ccdc8474bda3ec93a41857 INFO: 172.21.0.1:33402 - "POST /node/69ccdc8474bda3ec93a41857 HTTP/1.0" 202 Accepted 2026-04-03 08:51:20,083 - lava_callback - INFO - Starting async job submit for node 69ccdc8474bda3ec93a41857 2026-04-03 08:51:20,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:20,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdc8474bda3ec93a41857 HTTP/1.1" 200 None 2026-04-03 08:51:20,114 - lava_callback - DEBUG - Processing node 69ccdc8474bda3ec93a41857: baseline-arm-baylibre 2026-04-03 08:51:20,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdc8474bda3ec93a41857 2026-04-03 08:51:20,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:20,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdc8474bda3ec93a41857 HTTP/1.1" 200 None 2026-04-03 08:51:20,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:20,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 08:51:20,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:20,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdc8474bda3ec93a41857 HTTP/1.1" 200 None 2026-04-03 08:51:20,217 - lava_callback - ERROR - Error processing callback for node 69ccdc8474bda3ec93a41857: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 08:51:21,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:21,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-03 08:51:22,792 - lava_callback - INFO - Received callback for node 69cf7f3e000c489abc2c010d INFO: 172.21.0.1:33408 - "POST /node/69cf7f3e000c489abc2c010d HTTP/1.0" 202 Accepted 2026-04-03 08:51:22,800 - lava_callback - INFO - Starting async job submit for node 69cf7f3e000c489abc2c010d 2026-04-03 08:51:22,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:22,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3e000c489abc2c010d HTTP/1.1" 200 None 2026-04-03 08:51:22,858 - lava_callback - DEBUG - Processing node 69cf7f3e000c489abc2c010d: baseline-arm 2026-04-03 08:51:23,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:23,294 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:23,296 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3e000c489abc2c010d/log.txt.gz 2026-04-03 08:51:23,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:23,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:23,821 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3e000c489abc2c010d/lava-logs.yaml 2026-04-03 08:51:23,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:23,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:23,971 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3e000c489abc2c010d/lava_callback.json.gz 2026-04-03 08:51:24,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:24,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3e000c489abc2c010d HTTP/1.1" 200 None 2026-04-03 08:51:24,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:24,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:51:24,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:24,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f3e000c489abc2c010d HTTP/1.1" 200 None 2026-04-03 08:51:24,094 - lava_callback - INFO - Completed processing callback for node 69cf7f3e000c489abc2c010d 2026-04-03 08:51:24,865 - lava_callback - INFO - Received callback for node 69cf7eca000c489abc2bff2a INFO: 172.21.0.1:33416 - "POST /node/69cf7eca000c489abc2bff2a HTTP/1.0" 202 Accepted 2026-04-03 08:51:24,889 - lava_callback - INFO - Starting async job submit for node 69cf7eca000c489abc2bff2a 2026-04-03 08:51:24,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:24,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7eca000c489abc2bff2a HTTP/1.1" 200 None 2026-04-03 08:51:24,972 - lava_callback - DEBUG - Processing node 69cf7eca000c489abc2bff2a: baseline-arm64 2026-04-03 08:51:25,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:25,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:25,676 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7eca000c489abc2bff2a/log.txt.gz 2026-04-03 08:51:25,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:26,122 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:26,125 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7eca000c489abc2bff2a/lava-logs.yaml 2026-04-03 08:51:26,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:26,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:26,443 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7eca000c489abc2bff2a/lava_callback.json.gz 2026-04-03 08:51:26,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:26,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7eca000c489abc2bff2a HTTP/1.1" 200 None 2026-04-03 08:51:26,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:33426 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:51:26,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:51:26,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:26,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7eca000c489abc2bff2a HTTP/1.1" 200 None 2026-04-03 08:51:26,550 - lava_callback - INFO - Completed processing callback for node 69cf7eca000c489abc2bff2a 2026-04-03 08:51:30,009 - lava_callback - INFO - Received callback for node 69cf7f3a000c489abc2c00f9 INFO: 172.21.0.1:41088 - "POST /node/69cf7f3a000c489abc2c00f9 HTTP/1.0" 202 Accepted 2026-04-03 08:51:30,019 - lava_callback - INFO - Starting async job submit for node 69cf7f3a000c489abc2c00f9 2026-04-03 08:51:30,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:30,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3a000c489abc2c00f9 HTTP/1.1" 200 None 2026-04-03 08:51:30,089 - lava_callback - DEBUG - Processing node 69cf7f3a000c489abc2c00f9: baseline-arm 2026-04-03 08:51:30,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:30,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:30,448 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3a000c489abc2c00f9/log.txt.gz 2026-04-03 08:51:30,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:30,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:30,958 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3a000c489abc2c00f9/lava-logs.yaml 2026-04-03 08:51:30,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:31,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:31,106 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3a000c489abc2c00f9/lava_callback.json.gz 2026-04-03 08:51:31,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:31,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3a000c489abc2c00f9 HTTP/1.1" 200 None 2026-04-03 08:51:31,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:31,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:51:31,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:31,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f3a000c489abc2c00f9 HTTP/1.1" 200 None 2026-04-03 08:51:31,222 - lava_callback - INFO - Completed processing callback for node 69cf7f3a000c489abc2c00f9 INFO: 172.21.0.1:41104 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:51:41,861 - lava_callback - INFO - Received callback for node 69cf7ef0000c489abc2bffeb INFO: 172.21.0.1:41914 - "POST /node/69cf7ef0000c489abc2bffeb HTTP/1.0" 202 Accepted 2026-04-03 08:51:41,867 - lava_callback - INFO - Starting async job submit for node 69cf7ef0000c489abc2bffeb 2026-04-03 08:51:41,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:41,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef0000c489abc2bffeb HTTP/1.1" 200 None 2026-04-03 08:51:41,908 - lava_callback - DEBUG - Processing node 69cf7ef0000c489abc2bffeb: baseline-arm64-broonie 2026-04-03 08:51:42,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:42,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:42,251 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef0000c489abc2bffeb/log.txt.gz 2026-04-03 08:51:42,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:42,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:42,551 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef0000c489abc2bffeb/lava-logs.yaml 2026-04-03 08:51:42,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:42,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:42,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef0000c489abc2bffeb/lava_callback.json.gz 2026-04-03 08:51:42,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:42,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef0000c489abc2bffeb HTTP/1.1" 200 None 2026-04-03 08:51:42,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:42,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:51:42,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:42,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ef0000c489abc2bffeb HTTP/1.1" 200 None 2026-04-03 08:51:42,763 - lava_callback - INFO - Completed processing callback for node 69cf7ef0000c489abc2bffeb 2026-04-03 08:51:45,453 - lava_callback - INFO - Received callback for node 69cf7f67000c489abc2c0234 INFO: 172.21.0.1:41916 - "POST /node/69cf7f67000c489abc2c0234 HTTP/1.0" 202 Accepted 2026-04-03 08:51:45,501 - lava_callback - INFO - Starting async job submit for node 69cf7f67000c489abc2c0234 2026-04-03 08:51:45,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:45,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f67000c489abc2c0234 HTTP/1.1" 200 None 2026-04-03 08:51:45,537 - lava_callback - DEBUG - Processing node 69cf7f67000c489abc2c0234: baseline-arm-pengutronix 2026-04-03 08:51:45,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:45,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:45,754 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f67000c489abc2c0234/log.txt.gz 2026-04-03 08:51:45,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:46,006 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:46,007 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f67000c489abc2c0234/lava-logs.yaml 2026-04-03 08:51:46,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:46,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:46,120 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f67000c489abc2c0234/lava_callback.json.gz 2026-04-03 08:51:46,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:46,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f67000c489abc2c0234 HTTP/1.1" 200 None 2026-04-03 08:51:46,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:46,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:51:46,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:46,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f67000c489abc2c0234 HTTP/1.1" 200 None 2026-04-03 08:51:46,232 - lava_callback - INFO - Completed processing callback for node 69cf7f67000c489abc2c0234 INFO: 172.21.0.1:41930 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:51:50,135 - lava_callback - INFO - Received callback for node 69cf7f63000c489abc2c01f1 INFO: 172.21.0.1:56492 - "POST /node/69cf7f63000c489abc2c01f1 HTTP/1.0" 202 Accepted 2026-04-03 08:51:50,141 - lava_callback - INFO - Starting async job submit for node 69cf7f63000c489abc2c01f1 2026-04-03 08:51:50,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:50,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f63000c489abc2c01f1 HTTP/1.1" 200 None 2026-04-03 08:51:50,189 - lava_callback - DEBUG - Processing node 69cf7f63000c489abc2c01f1: baseline-arm-pengutronix 2026-04-03 08:51:50,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:50,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:50,455 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f63000c489abc2c01f1/log.txt.gz 2026-04-03 08:51:50,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:50,707 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:50,709 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f63000c489abc2c01f1/lava-logs.yaml 2026-04-03 08:51:50,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:51:50,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:51:50,836 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f63000c489abc2c01f1/lava_callback.json.gz 2026-04-03 08:51:50,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:50,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f63000c489abc2c01f1 HTTP/1.1" 200 None 2026-04-03 08:51:50,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:50,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:51:50,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:50,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f63000c489abc2c01f1 HTTP/1.1" 200 None 2026-04-03 08:51:50,927 - lava_callback - INFO - Completed processing callback for node 69cf7f63000c489abc2c01f1 2026-04-03 08:51:51,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:51:51,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:56496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56502 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:52:00,026 - lava_callback - INFO - Received callback for node 69ccdca974bda3ec93a4186c INFO: 172.21.0.1:47714 - "POST /node/69ccdca974bda3ec93a4186c HTTP/1.0" 202 Accepted 2026-04-03 08:52:00,032 - lava_callback - INFO - Starting async job submit for node 69ccdca974bda3ec93a4186c 2026-04-03 08:52:00,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:00,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdca974bda3ec93a4186c HTTP/1.1" 200 None 2026-04-03 08:52:00,048 - lava_callback - DEBUG - Processing node 69ccdca974bda3ec93a4186c: baseline-arm-baylibre 2026-04-03 08:52:00,048 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdca974bda3ec93a4186c 2026-04-03 08:52:00,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:00,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdca974bda3ec93a4186c HTTP/1.1" 200 None 2026-04-03 08:52:00,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:00,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 08:52:00,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:00,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdca974bda3ec93a4186c HTTP/1.1" 200 None 2026-04-03 08:52:00,107 - lava_callback - ERROR - Error processing callback for node 69ccdca974bda3ec93a4186c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 08:52:01,772 - lava_callback - INFO - Received callback for node 69cf7f4f000c489abc2c01a3 INFO: 172.21.0.1:47716 - "POST /node/69cf7f4f000c489abc2c01a3 HTTP/1.0" 202 Accepted 2026-04-03 08:52:01,778 - lava_callback - INFO - Starting async job submit for node 69cf7f4f000c489abc2c01a3 2026-04-03 08:52:01,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:01,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f4f000c489abc2c01a3 HTTP/1.1" 200 None 2026-04-03 08:52:01,819 - lava_callback - DEBUG - Processing node 69cf7f4f000c489abc2c01a3: baseline-arm-broonie 2026-04-03 08:52:02,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:02,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:02,194 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf7f4f000c489abc2c01a3/log.txt.gz 2026-04-03 08:52:02,268 - lava_callback - INFO - Received callback for node 69cf7f50000c489abc2c01a6 INFO: 172.21.0.1:47720 - "POST /node/69cf7f50000c489abc2c01a6 HTTP/1.0" 202 Accepted 2026-04-03 08:52:02,289 - lava_callback - INFO - Starting async job submit for node 69cf7f50000c489abc2c01a6 2026-04-03 08:52:02,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:02,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:02,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f50000c489abc2c01a6 HTTP/1.1" 200 None 2026-04-03 08:52:02,384 - lava_callback - DEBUG - Processing node 69cf7f50000c489abc2c01a6: baseline-arm-broonie 2026-04-03 08:52:02,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:02,582 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf7f4f000c489abc2c01a3/lava-logs.yaml 2026-04-03 08:52:02,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:02,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:02,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:02,812 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:02,814 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf7f4f000c489abc2c01a3/lava_callback.json.gz 2026-04-03 08:52:02,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:02,837 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf7f50000c489abc2c01a6/log.txt.gz 2026-04-03 08:52:02,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f4f000c489abc2c01a3 HTTP/1.1" 200 None 2026-04-03 08:52:02,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:02,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:02,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:52:02,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:03,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f4f000c489abc2c01a3 HTTP/1.1" 200 None 2026-04-03 08:52:03,032 - lava_callback - INFO - Completed processing callback for node 69cf7f4f000c489abc2c01a3 2026-04-03 08:52:03,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:03,234 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf7f50000c489abc2c01a6/lava-logs.yaml 2026-04-03 08:52:03,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:03,390 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:03,392 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf7f50000c489abc2c01a6/lava_callback.json.gz 2026-04-03 08:52:03,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:03,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f50000c489abc2c01a6 HTTP/1.1" 200 None 2026-04-03 08:52:03,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:03,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:52:03,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:03,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f50000c489abc2c01a6 HTTP/1.1" 200 None 2026-04-03 08:52:03,483 - lava_callback - INFO - Completed processing callback for node 69cf7f50000c489abc2c01a6 INFO: 172.21.0.1:47728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35944 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:52:20,383 - lava_callback - INFO - Received callback for node 69cf7f5f000c489abc2c01da INFO: 172.21.0.1:39762 - "POST /node/69cf7f5f000c489abc2c01da HTTP/1.0" 202 Accepted 2026-04-03 08:52:20,389 - lava_callback - INFO - Starting async job submit for node 69cf7f5f000c489abc2c01da 2026-04-03 08:52:20,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:20,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f5f000c489abc2c01da HTTP/1.1" 200 None 2026-04-03 08:52:20,425 - lava_callback - DEBUG - Processing node 69cf7f5f000c489abc2c01da: baseline-arm-kontron 2026-04-03 08:52:20,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:20,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:20,736 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf7f5f000c489abc2c01da/log.txt.gz 2026-04-03 08:52:20,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:21,068 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:21,070 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf7f5f000c489abc2c01da/lava-logs.yaml 2026-04-03 08:52:21,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:21,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:21,211 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf7f5f000c489abc2c01da/lava_callback.json.gz 2026-04-03 08:52:21,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:21,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f5f000c489abc2c01da HTTP/1.1" 200 None 2026-04-03 08:52:21,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:21,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:21,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:52:21,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:21,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 08:52:21,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f5f000c489abc2c01da HTTP/1.1" 200 None 2026-04-03 08:52:21,303 - lava_callback - INFO - Completed processing callback for node 69cf7f5f000c489abc2c01da 2026-04-03 08:52:22,523 - lava_callback - INFO - Received callback for node 69cf7f4c000c489abc2c0176 INFO: 172.21.0.1:39772 - "POST /node/69cf7f4c000c489abc2c0176 HTTP/1.0" 202 Accepted 2026-04-03 08:52:22,529 - lava_callback - INFO - Starting async job submit for node 69cf7f4c000c489abc2c0176 2026-04-03 08:52:22,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:22,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f4c000c489abc2c0176 HTTP/1.1" 200 None 2026-04-03 08:52:22,568 - lava_callback - DEBUG - Processing node 69cf7f4c000c489abc2c0176: baseline-arm-broonie 2026-04-03 08:52:22,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:22,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:22,906 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf7f4c000c489abc2c0176/log.txt.gz 2026-04-03 08:52:23,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:23,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:23,203 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf7f4c000c489abc2c0176/lava-logs.yaml 2026-04-03 08:52:23,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:23,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:23,347 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf7f4c000c489abc2c0176/lava_callback.json.gz 2026-04-03 08:52:23,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:23,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f4c000c489abc2c0176 HTTP/1.1" 200 None 2026-04-03 08:52:23,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:23,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:52:23,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:23,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f4c000c489abc2c0176 HTTP/1.1" 200 None 2026-04-03 08:52:23,428 - lava_callback - INFO - Completed processing callback for node 69cf7f4c000c489abc2c0176 2026-04-03 08:52:25,057 - lava_callback - INFO - Received callback for node 69cf7f56000c489abc2c01c1 INFO: 172.21.0.1:39786 - "POST /node/69cf7f56000c489abc2c01c1 HTTP/1.0" 202 Accepted 2026-04-03 08:52:25,063 - lava_callback - INFO - Starting async job submit for node 69cf7f56000c489abc2c01c1 2026-04-03 08:52:25,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:25,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f56000c489abc2c01c1 HTTP/1.1" 200 None 2026-04-03 08:52:25,100 - lava_callback - DEBUG - Processing node 69cf7f56000c489abc2c01c1: baseline-arm-clabbe 2026-04-03 08:52:25,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:25,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:25,416 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf7f56000c489abc2c01c1/log.txt.gz 2026-04-03 08:52:25,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:25,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:25,730 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf7f56000c489abc2c01c1/lava-logs.yaml 2026-04-03 08:52:25,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:25,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:25,894 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf7f56000c489abc2c01c1/lava_callback.json.gz 2026-04-03 08:52:25,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:25,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f56000c489abc2c01c1 HTTP/1.1" 200 None 2026-04-03 08:52:25,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:25,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:52:25,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:25,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f56000c489abc2c01c1 HTTP/1.1" 200 None 2026-04-03 08:52:25,984 - lava_callback - INFO - Completed processing callback for node 69cf7f56000c489abc2c01c1 INFO: 172.21.0.1:39794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37558 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:52:40,234 - lava_callback - INFO - Received callback for node 69cf7f5e000c489abc2c01d6 INFO: 172.21.0.1:55898 - "POST /node/69cf7f5e000c489abc2c01d6 HTTP/1.0" 202 Accepted 2026-04-03 08:52:40,241 - lava_callback - INFO - Starting async job submit for node 69cf7f5e000c489abc2c01d6 2026-04-03 08:52:40,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:40,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f5e000c489abc2c01d6 HTTP/1.1" 200 None 2026-04-03 08:52:40,316 - lava_callback - DEBUG - Processing node 69cf7f5e000c489abc2c01d6: baseline-arm-kontron 2026-04-03 08:52:40,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:40,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:40,784 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf7f5e000c489abc2c01d6/log.txt.gz 2026-04-03 08:52:40,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:41,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:41,118 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf7f5e000c489abc2c01d6/lava-logs.yaml 2026-04-03 08:52:41,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:41,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:41,318 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf7f5e000c489abc2c01d6/lava_callback.json.gz 2026-04-03 08:52:41,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:41,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f5e000c489abc2c01d6 HTTP/1.1" 200 None 2026-04-03 08:52:41,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:41,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:52:41,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:41,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f5e000c489abc2c01d6 HTTP/1.1" 200 None 2026-04-03 08:52:41,442 - lava_callback - INFO - Completed processing callback for node 69cf7f5e000c489abc2c01d6 2026-04-03 08:52:41,718 - lava_callback - INFO - Received callback for node 69cf7ef4000c489abc2bfff7 INFO: 172.21.0.1:55908 - "POST /node/69cf7ef4000c489abc2bfff7 HTTP/1.0" 202 Accepted 2026-04-03 08:52:41,730 - lava_callback - INFO - Starting async job submit for node 69cf7ef4000c489abc2bfff7 2026-04-03 08:52:41,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:41,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef4000c489abc2bfff7 HTTP/1.1" 200 None 2026-04-03 08:52:41,795 - lava_callback - DEBUG - Processing node 69cf7ef4000c489abc2bfff7: baseline-arm64-broonie 2026-04-03 08:52:42,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:42,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:42,321 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef4000c489abc2bfff7/log.txt.gz 2026-04-03 08:52:42,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:42,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:42,752 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef4000c489abc2bfff7/lava-logs.yaml 2026-04-03 08:52:42,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:42,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:42,921 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef4000c489abc2bfff7/lava_callback.json.gz 2026-04-03 08:52:42,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:42,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef4000c489abc2bfff7 HTTP/1.1" 200 None 2026-04-03 08:52:42,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:42,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:52:42,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:43,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ef4000c489abc2bfff7 HTTP/1.1" 200 None 2026-04-03 08:52:43,016 - lava_callback - INFO - Completed processing callback for node 69cf7ef4000c489abc2bfff7 INFO: 172.21.0.1:55920 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:52:48,251 - lava_callback - INFO - Received callback for node 69cf7f64000c489abc2c01f4 INFO: 172.21.0.1:55926 - "POST /node/69cf7f64000c489abc2c01f4 HTTP/1.0" 202 Accepted 2026-04-03 08:52:48,257 - lava_callback - INFO - Starting async job submit for node 69cf7f64000c489abc2c01f4 2026-04-03 08:52:48,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:48,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f64000c489abc2c01f4 HTTP/1.1" 200 None 2026-04-03 08:52:48,296 - lava_callback - DEBUG - Processing node 69cf7f64000c489abc2c01f4: baseline-arm-pengutronix 2026-04-03 08:52:48,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:48,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:48,616 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f64000c489abc2c01f4/log.txt.gz 2026-04-03 08:52:48,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:49,031 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:49,034 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f64000c489abc2c01f4/lava-logs.yaml 2026-04-03 08:52:49,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:52:49,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:52:49,189 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f64000c489abc2c01f4/lava_callback.json.gz 2026-04-03 08:52:49,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:49,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f64000c489abc2c01f4 HTTP/1.1" 200 None 2026-04-03 08:52:49,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:49,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:52:49,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:49,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f64000c489abc2c01f4 HTTP/1.1" 200 None 2026-04-03 08:52:49,377 - lava_callback - INFO - Completed processing callback for node 69cf7f64000c489abc2c01f4 2026-04-03 08:52:51,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:51,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:53064 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:52:53,040 - lava_callback - INFO - Received callback for node 69ccdcd174bda3ec93a41888 INFO: 172.21.0.1:53072 - "POST /node/69ccdcd174bda3ec93a41888 HTTP/1.0" 202 Accepted 2026-04-03 08:52:53,048 - lava_callback - INFO - Starting async job submit for node 69ccdcd174bda3ec93a41888 2026-04-03 08:52:53,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:53,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdcd174bda3ec93a41888 HTTP/1.1" 200 None 2026-04-03 08:52:53,065 - lava_callback - DEBUG - Processing node 69ccdcd174bda3ec93a41888: baseline-arm-clabbe 2026-04-03 08:52:53,065 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdcd174bda3ec93a41888 2026-04-03 08:52:53,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:53,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdcd174bda3ec93a41888 HTTP/1.1" 200 None 2026-04-03 08:52:53,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:53,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 08:52:53,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:52:53,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdcd174bda3ec93a41888 HTTP/1.1" 200 None 2026-04-03 08:52:53,126 - lava_callback - ERROR - Error processing callback for node 69ccdcd174bda3ec93a41888: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36874 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:53:33,347 - lava_callback - INFO - Received callback for node 69ccdcf874bda3ec93a41899 INFO: 172.21.0.1:45024 - "POST /node/69ccdcf874bda3ec93a41899 HTTP/1.0" 202 Accepted 2026-04-03 08:53:33,355 - lava_callback - INFO - Starting async job submit for node 69ccdcf874bda3ec93a41899 2026-04-03 08:53:33,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:33,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdcf874bda3ec93a41899 HTTP/1.1" 200 None 2026-04-03 08:53:33,397 - lava_callback - DEBUG - Processing node 69ccdcf874bda3ec93a41899: baseline-arm-clabbe 2026-04-03 08:53:33,398 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdcf874bda3ec93a41899 2026-04-03 08:53:33,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:33,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdcf874bda3ec93a41899 HTTP/1.1" 200 None 2026-04-03 08:53:33,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:33,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 08:53:33,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:33,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdcf874bda3ec93a41899 HTTP/1.1" 200 None 2026-04-03 08:53:33,480 - lava_callback - ERROR - Error processing callback for node 69ccdcf874bda3ec93a41899: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53556 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:53:52,953 - lava_callback - INFO - Received callback for node 69ccdd1e74bda3ec93a418aa INFO: 172.21.0.1:53560 - "POST /node/69ccdd1e74bda3ec93a418aa HTTP/1.0" 202 Accepted 2026-04-03 08:53:52,960 - lava_callback - INFO - Starting async job submit for node 69ccdd1e74bda3ec93a418aa 2026-04-03 08:53:52,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:52,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdd1e74bda3ec93a418aa HTTP/1.1" 200 None 2026-04-03 08:53:52,979 - lava_callback - DEBUG - Processing node 69ccdd1e74bda3ec93a418aa: baseline-arm-clabbe 2026-04-03 08:53:52,979 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdd1e74bda3ec93a418aa 2026-04-03 08:53:52,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:53,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdd1e74bda3ec93a418aa HTTP/1.1" 200 None 2026-04-03 08:53:53,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:53,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 08:53:53,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:53,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdd1e74bda3ec93a418aa HTTP/1.1" 200 None 2026-04-03 08:53:53,068 - lava_callback - ERROR - Error processing callback for node 69ccdd1e74bda3ec93a418aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 08:53:56,231 - lava_callback - INFO - Received callback for node 69cf7f80000c489abc2c0278 INFO: 172.21.0.1:53570 - "POST /node/69cf7f80000c489abc2c0278 HTTP/1.0" 202 Accepted 2026-04-03 08:53:56,239 - lava_callback - INFO - Starting async job submit for node 69cf7f80000c489abc2c0278 2026-04-03 08:53:56,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:56,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f80000c489abc2c0278 HTTP/1.1" 200 None 2026-04-03 08:53:56,283 - lava_callback - DEBUG - Processing node 69cf7f80000c489abc2c0278: ltp-dio INFO: 172.21.0.1:53574 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:53:56,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:53:56,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:53:56,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69cf7f80000c489abc2c0278/log.txt.gz 2026-04-03 08:53:57,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:53:57,424 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:53:57,427 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69cf7f80000c489abc2c0278/lava-logs.yaml 2026-04-03 08:53:57,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:53:57,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:53:57,823 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69cf7f80000c489abc2c0278/lava_callback.json.gz 2026-04-03 08:53:57,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:57,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f80000c489abc2c0278 HTTP/1.1" 200 None 2026-04-03 08:53:57,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:57,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:53:57,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:53:57,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f80000c489abc2c0278 HTTP/1.1" 200 None 2026-04-03 08:53:57,929 - lava_callback - INFO - Completed processing callback for node 69cf7f80000c489abc2c0278 2026-04-03 08:54:00,134 - lava_callback - INFO - Received callback for node 69cf7f5a000c489abc2c01cd INFO: 172.21.0.1:38058 - "POST /node/69cf7f5a000c489abc2c01cd HTTP/1.0" 202 Accepted 2026-04-03 08:54:00,141 - lava_callback - INFO - Starting async job submit for node 69cf7f5a000c489abc2c01cd 2026-04-03 08:54:00,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:00,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f5a000c489abc2c01cd HTTP/1.1" 200 None 2026-04-03 08:54:00,182 - lava_callback - DEBUG - Processing node 69cf7f5a000c489abc2c01cd: baseline-arm-clabbe 2026-04-03 08:54:00,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:54:00,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:54:00,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf7f5a000c489abc2c01cd/log.txt.gz 2026-04-03 08:54:00,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:54:00,953 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:54:00,955 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf7f5a000c489abc2c01cd/lava-logs.yaml 2026-04-03 08:54:00,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:54:01,086 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:54:01,088 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf7f5a000c489abc2c01cd/lava_callback.json.gz 2026-04-03 08:54:01,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:01,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f5a000c489abc2c01cd HTTP/1.1" 200 None 2026-04-03 08:54:01,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:01,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:54:01,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:01,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f5a000c489abc2c01cd HTTP/1.1" 200 None 2026-04-03 08:54:01,207 - lava_callback - INFO - Completed processing callback for node 69cf7f5a000c489abc2c01cd 2026-04-03 08:54:01,937 - lava_callback - INFO - Received callback for node 69cf7ef8000c489abc2c0002 INFO: 172.21.0.1:38066 - "POST /node/69cf7ef8000c489abc2c0002 HTTP/1.0" 202 Accepted 2026-04-03 08:54:01,947 - lava_callback - INFO - Starting async job submit for node 69cf7ef8000c489abc2c0002 2026-04-03 08:54:01,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:02,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef8000c489abc2c0002 HTTP/1.1" 200 None 2026-04-03 08:54:02,003 - lava_callback - DEBUG - Processing node 69cf7ef8000c489abc2c0002: baseline-arm64-broonie 2026-04-03 08:54:02,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:54:02,967 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:54:02,969 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef8000c489abc2c0002/log.txt.gz 2026-04-03 08:54:03,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:54:03,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:54:03,549 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef8000c489abc2c0002/lava-logs.yaml 2026-04-03 08:54:03,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:54:03,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:54:03,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf7ef8000c489abc2c0002/lava_callback.json.gz 2026-04-03 08:54:03,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:03,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ef8000c489abc2c0002 HTTP/1.1" 200 None 2026-04-03 08:54:03,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:03,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:54:03,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:03,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ef8000c489abc2c0002 HTTP/1.1" 200 None 2026-04-03 08:54:03,991 - lava_callback - INFO - Completed processing callback for node 69cf7ef8000c489abc2c0002 INFO: 172.21.0.1:38080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39666 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:54:21,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:21,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57248 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:54:32,906 - lava_callback - INFO - Received callback for node 69ccdd4474bda3ec93a418bf INFO: 172.21.0.1:47586 - "POST /node/69ccdd4474bda3ec93a418bf HTTP/1.0" 202 Accepted 2026-04-03 08:54:32,915 - lava_callback - INFO - Starting async job submit for node 69ccdd4474bda3ec93a418bf 2026-04-03 08:54:32,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:32,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdd4474bda3ec93a418bf HTTP/1.1" 200 None 2026-04-03 08:54:32,946 - lava_callback - DEBUG - Processing node 69ccdd4474bda3ec93a418bf: baseline-arm-clabbe 2026-04-03 08:54:32,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdd4474bda3ec93a418bf 2026-04-03 08:54:32,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:32,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdd4474bda3ec93a418bf HTTP/1.1" 200 None 2026-04-03 08:54:32,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:32,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 08:54:32,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:33,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdd4474bda3ec93a418bf HTTP/1.1" 200 None 2026-04-03 08:54:33,032 - lava_callback - ERROR - Error processing callback for node 69ccdd4474bda3ec93a418bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47590 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:54:37,565 - lava_callback - INFO - Received callback for node 69cf8025000c489abc2c03cc INFO: 172.21.0.1:47602 - "POST /node/69cf8025000c489abc2c03cc HTTP/1.0" 202 Accepted 2026-04-03 08:54:37,573 - lava_callback - INFO - Starting async job submit for node 69cf8025000c489abc2c03cc 2026-04-03 08:54:37,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:37,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8025000c489abc2c03cc HTTP/1.1" 200 None 2026-04-03 08:54:37,606 - lava_callback - DEBUG - Processing node 69cf8025000c489abc2c03cc: baseline-nfs-arm64 2026-04-03 08:54:37,608 - lava_callback - WARNING - Failed to upload log 2026-04-03 08:54:37,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:54:37,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:54:37,764 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf8025000c489abc2c03cc/lava-logs.yaml 2026-04-03 08:54:37,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:54:37,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:54:37,952 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf8025000c489abc2c03cc/lava_callback.json.gz 2026-04-03 08:54:37,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:37,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8025000c489abc2c03cc HTTP/1.1" 200 None 2026-04-03 08:54:37,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:37,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:54:38,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:38,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8025000c489abc2c03cc HTTP/1.1" 200 None 2026-04-03 08:54:38,039 - lava_callback - INFO - Completed processing callback for node 69cf8025000c489abc2c03cc INFO: 172.21.0.1:44696 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:54:51,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:54:51,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36070 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:55:13,085 - lava_callback - INFO - Received callback for node 69ccdd6a74bda3ec93a418d7 INFO: 172.21.0.1:56578 - "POST /node/69ccdd6a74bda3ec93a418d7 HTTP/1.0" 202 Accepted 2026-04-03 08:55:13,094 - lava_callback - INFO - Starting async job submit for node 69ccdd6a74bda3ec93a418d7 2026-04-03 08:55:13,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:13,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdd6a74bda3ec93a418d7 HTTP/1.1" 200 None 2026-04-03 08:55:13,111 - lava_callback - DEBUG - Processing node 69ccdd6a74bda3ec93a418d7: baseline-arm-clabbe 2026-04-03 08:55:13,111 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdd6a74bda3ec93a418d7 2026-04-03 08:55:13,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:13,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdd6a74bda3ec93a418d7 HTTP/1.1" 200 None 2026-04-03 08:55:13,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:13,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 08:55:13,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:13,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdd6a74bda3ec93a418d7 HTTP/1.1" 200 None 2026-04-03 08:55:13,176 - lava_callback - ERROR - Error processing callback for node 69ccdd6a74bda3ec93a418d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56594 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:55:22,087 - lava_callback - INFO - Received callback for node 69cf7f77000c489abc2c025c INFO: 172.21.0.1:37372 - "POST /node/69cf7f77000c489abc2c025c HTTP/1.0" 202 Accepted 2026-04-03 08:55:22,093 - lava_callback - INFO - Starting async job submit for node 69cf7f77000c489abc2c025c 2026-04-03 08:55:22,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:22,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f77000c489abc2c025c HTTP/1.1" 200 None 2026-04-03 08:55:22,134 - lava_callback - DEBUG - Processing node 69cf7f77000c489abc2c025c: ltp-cap_bounds 2026-04-03 08:55:22,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:22,692 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:22,694 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cf7f77000c489abc2c025c/log.txt.gz 2026-04-03 08:55:22,899 - lava_callback - INFO - Received callback for node 69cf7f7b000c489abc2c0266 INFO: 172.21.0.1:37376 - "POST /node/69cf7f7b000c489abc2c0266 HTTP/1.0" 202 Accepted 2026-04-03 08:55:22,914 - lava_callback - INFO - Starting async job submit for node 69cf7f7b000c489abc2c0266 2026-04-03 08:55:22,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:22,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:22,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f7b000c489abc2c0266 HTTP/1.1" 200 None 2026-04-03 08:55:22,989 - lava_callback - DEBUG - Processing node 69cf7f7b000c489abc2c0266: ltp-controllers 2026-04-03 08:55:23,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:23,260 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cf7f77000c489abc2c025c/lava-logs.yaml 2026-04-03 08:55:23,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:23,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:23,632 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:23,634 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cf7f77000c489abc2c025c/lava_callback.json.gz 2026-04-03 08:55:23,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:23,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:23,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f77000c489abc2c025c HTTP/1.1" 200 None 2026-04-03 08:55:23,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:23,694 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69cf7f7b000c489abc2c0266/log.txt.gz 2026-04-03 08:55:23,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:55:23,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:23,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f77000c489abc2c025c HTTP/1.1" 200 None 2026-04-03 08:55:23,921 - lava_callback - INFO - Completed processing callback for node 69cf7f77000c489abc2c025c 2026-04-03 08:55:24,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:24,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:24,291 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69cf7f7b000c489abc2c0266/lava-logs.yaml 2026-04-03 08:55:24,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:24,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:24,462 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69cf7f7b000c489abc2c0266/lava_callback.json.gz 2026-04-03 08:55:24,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:24,490 - lava_callback - INFO - Received callback for node 69cf7f8c000c489abc2c02c2 INFO: 172.21.0.1:37384 - "POST /node/69cf7f8c000c489abc2c02c2 HTTP/1.0" 202 Accepted 2026-04-03 08:55:24,497 - lava_callback - INFO - Starting async job submit for node 69cf7f8c000c489abc2c02c2 2026-04-03 08:55:24,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:24,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f7b000c489abc2c0266 HTTP/1.1" 200 None 2026-04-03 08:55:24,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:24,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f8c000c489abc2c02c2 HTTP/1.1" 200 None 2026-04-03 08:55:24,577 - lava_callback - DEBUG - Processing node 69cf7f8c000c489abc2c02c2: ltp-syscalls 2026-04-03 08:55:24,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:55:24,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:24,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f7b000c489abc2c0266 HTTP/1.1" 200 None 2026-04-03 08:55:24,680 - lava_callback - INFO - Completed processing callback for node 69cf7f7b000c489abc2c0266 2026-04-03 08:55:25,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:25,589 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:25,591 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cf7f8c000c489abc2c02c2/log.txt.gz 2026-04-03 08:55:26,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:26,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:26,310 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cf7f8c000c489abc2c02c2/lava-logs.yaml 2026-04-03 08:55:26,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:37388 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:55:26,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:26,520 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cf7f8c000c489abc2c02c2/lava_callback.json.gz 2026-04-03 08:55:26,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:26,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f8c000c489abc2c02c2 HTTP/1.1" 200 None 2026-04-03 08:55:26,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:26,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:55:26,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:26,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f8c000c489abc2c02c2 HTTP/1.1" 200 None 2026-04-03 08:55:26,617 - lava_callback - INFO - Completed processing callback for node 69cf7f8c000c489abc2c02c2 INFO: 172.21.0.1:59984 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:55:42,840 - lava_callback - INFO - Received callback for node 69cf7f7e000c489abc2c0275 INFO: 172.21.0.1:38156 - "POST /node/69cf7f7e000c489abc2c0275 HTTP/1.0" 202 Accepted 2026-04-03 08:55:42,851 - lava_callback - INFO - Starting async job submit for node 69cf7f7e000c489abc2c0275 2026-04-03 08:55:42,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:42,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f7e000c489abc2c0275 HTTP/1.1" 200 None 2026-04-03 08:55:42,897 - lava_callback - DEBUG - Processing node 69cf7f7e000c489abc2c0275: ltp-cve 2026-04-03 08:55:43,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:43,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:43,435 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69cf7f7e000c489abc2c0275/log.txt.gz 2026-04-03 08:55:43,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:43,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:43,879 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69cf7f7e000c489abc2c0275/lava-logs.yaml 2026-04-03 08:55:43,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:44,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:44,027 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69cf7f7e000c489abc2c0275/lava_callback.json.gz 2026-04-03 08:55:44,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:44,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f7e000c489abc2c0275 HTTP/1.1" 200 None 2026-04-03 08:55:44,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:44,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:55:44,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:44,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f7e000c489abc2c0275 HTTP/1.1" 200 None 2026-04-03 08:55:44,108 - lava_callback - INFO - Completed processing callback for node 69cf7f7e000c489abc2c0275 2026-04-03 08:55:45,928 - lava_callback - INFO - Received callback for node 69cf7eda000c489abc2bff89 INFO: 172.21.0.1:38168 - "POST /node/69cf7eda000c489abc2bff89 HTTP/1.0" 202 Accepted 2026-04-03 08:55:45,935 - lava_callback - INFO - Starting async job submit for node 69cf7eda000c489abc2bff89 2026-04-03 08:55:45,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:45,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7eda000c489abc2bff89 HTTP/1.1" 200 None 2026-04-03 08:55:45,975 - lava_callback - DEBUG - Processing node 69cf7eda000c489abc2bff89: baseline-nfs-arm64 2026-04-03 08:55:46,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:46,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:46,503 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7eda000c489abc2bff89/log.txt.gz INFO: 172.21.0.1:38170 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:55:46,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:46,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:46,947 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7eda000c489abc2bff89/lava-logs.yaml 2026-04-03 08:55:46,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:55:47,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:55:47,149 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7eda000c489abc2bff89/lava_callback.json.gz 2026-04-03 08:55:47,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:47,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7eda000c489abc2bff89 HTTP/1.1" 200 None 2026-04-03 08:55:47,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:47,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:55:47,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:47,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7eda000c489abc2bff89 HTTP/1.1" 200 None 2026-04-03 08:55:47,235 - lava_callback - INFO - Completed processing callback for node 69cf7eda000c489abc2bff89 2026-04-03 08:55:51,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:55:51,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48306 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:56:03,035 - lava_callback - INFO - Received callback for node 69cf7f79000c489abc2c0263 INFO: 172.21.0.1:55940 - "POST /node/69cf7f79000c489abc2c0263 HTTP/1.0" 202 Accepted 2026-04-03 08:56:03,041 - lava_callback - INFO - Starting async job submit for node 69cf7f79000c489abc2c0263 2026-04-03 08:56:03,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:03,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f79000c489abc2c0263 HTTP/1.1" 200 None 2026-04-03 08:56:03,084 - lava_callback - DEBUG - Processing node 69cf7f79000c489abc2c0263: ltp-containers 2026-04-03 08:56:03,340 - lava_callback - INFO - Received callback for node 69cf7ed7000c489abc2bff7f INFO: 172.21.0.1:55952 - "POST /node/69cf7ed7000c489abc2bff7f HTTP/1.0" 202 Accepted 2026-04-03 08:56:03,361 - lava_callback - INFO - Starting async job submit for node 69cf7ed7000c489abc2bff7f 2026-04-03 08:56:03,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:03,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed7000c489abc2bff7f HTTP/1.1" 200 None 2026-04-03 08:56:03,579 - lava_callback - DEBUG - Processing node 69cf7ed7000c489abc2bff7f: baseline-arm64 2026-04-03 08:56:03,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:04,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:04,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69cf7f79000c489abc2c0263/log.txt.gz 2026-04-03 08:56:04,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:04,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:04,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:04,345 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed7000c489abc2bff7f/log.txt.gz 2026-04-03 08:56:04,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:04,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:04,593 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69cf7f79000c489abc2c0263/lava-logs.yaml 2026-04-03 08:56:04,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:04,775 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:04,778 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed7000c489abc2bff7f/lava-logs.yaml 2026-04-03 08:56:04,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:04,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:04,808 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69cf7f79000c489abc2c0263/lava_callback.json.gz 2026-04-03 08:56:04,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:04,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f79000c489abc2c0263 HTTP/1.1" 200 None 2026-04-03 08:56:04,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:04,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:56:04,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:04,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f79000c489abc2c0263 HTTP/1.1" 200 None 2026-04-03 08:56:04,917 - lava_callback - INFO - Completed processing callback for node 69cf7f79000c489abc2c0263 2026-04-03 08:56:04,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:04,974 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed7000c489abc2bff7f/lava_callback.json.gz DEBUG: Setup failed for 69cf7ed7000c489abc2bff7f. Transitting job node result: incomplete -> fail 2026-04-03 08:56:05,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:05,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed7000c489abc2bff7f HTTP/1.1" 200 None 2026-04-03 08:56:05,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:05,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:56:05,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:05,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ed7000c489abc2bff7f HTTP/1.1" 200 None 2026-04-03 08:56:05,225 - lava_callback - INFO - Completed processing callback for node 69cf7ed7000c489abc2bff7f INFO: 172.21.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60980 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:56:21,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:21,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 08:56:22,903 - lava_callback - INFO - Received callback for node 69cf7f81000c489abc2c027b INFO: 172.21.0.1:55720 - "POST /node/69cf7f81000c489abc2c027b HTTP/1.0" 202 Accepted 2026-04-03 08:56:22,910 - lava_callback - INFO - Starting async job submit for node 69cf7f81000c489abc2c027b 2026-04-03 08:56:22,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:22,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f81000c489abc2c027b HTTP/1.1" 200 None 2026-04-03 08:56:22,953 - lava_callback - DEBUG - Processing node 69cf7f81000c489abc2c027b: ltp-fs_bind 2026-04-03 08:56:23,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:23,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:23,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cf7f81000c489abc2c027b/log.txt.gz 2026-04-03 08:56:23,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:23,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:23,940 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cf7f81000c489abc2c027b/lava-logs.yaml 2026-04-03 08:56:23,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:24,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:24,147 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cf7f81000c489abc2c027b/lava_callback.json.gz 2026-04-03 08:56:24,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:24,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f81000c489abc2c027b HTTP/1.1" 200 None 2026-04-03 08:56:24,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:24,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:56:24,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:24,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f81000c489abc2c027b HTTP/1.1" 200 None 2026-04-03 08:56:24,237 - lava_callback - INFO - Completed processing callback for node 69cf7f81000c489abc2c027b INFO: 172.21.0.1:55730 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:56:27,425 - lava_callback - INFO - Received callback for node 69cf8060000c489abc2c042e INFO: 172.21.0.1:55740 - "POST /node/69cf8060000c489abc2c042e HTTP/1.0" 202 Accepted 2026-04-03 08:56:27,432 - lava_callback - INFO - Starting async job submit for node 69cf8060000c489abc2c042e 2026-04-03 08:56:27,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:27,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8060000c489abc2c042e HTTP/1.1" 200 None 2026-04-03 08:56:27,485 - lava_callback - DEBUG - Processing node 69cf8060000c489abc2c042e: baseline-arm64 2026-04-03 08:56:27,645 - lava_callback - WARNING - Failed to upload log 2026-04-03 08:56:27,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:27,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:27,960 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8060000c489abc2c042e/lava-logs.yaml 2026-04-03 08:56:27,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:56:28,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:56:28,112 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8060000c489abc2c042e/lava_callback.json.gz 2026-04-03 08:56:28,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:28,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8060000c489abc2c042e HTTP/1.1" 200 None 2026-04-03 08:56:28,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:28,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:56:28,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:28,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8060000c489abc2c042e HTTP/1.1" 200 None 2026-04-03 08:56:28,252 - lava_callback - INFO - Completed processing callback for node 69cf8060000c489abc2c042e 2026-04-03 08:56:33,030 - lava_callback - INFO - Received callback for node 69ccddb474bda3ec93a41905 INFO: 172.21.0.1:53692 - "POST /node/69ccddb474bda3ec93a41905 HTTP/1.0" 202 Accepted 2026-04-03 08:56:33,036 - lava_callback - INFO - Starting async job submit for node 69ccddb474bda3ec93a41905 2026-04-03 08:56:33,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:33,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccddb474bda3ec93a41905 HTTP/1.1" 200 None 2026-04-03 08:56:33,055 - lava_callback - DEBUG - Processing node 69ccddb474bda3ec93a41905: baseline-arm-clabbe 2026-04-03 08:56:33,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccddb474bda3ec93a41905 2026-04-03 08:56:33,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:33,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccddb474bda3ec93a41905 HTTP/1.1" 200 None 2026-04-03 08:56:33,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:33,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 08:56:33,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:33,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccddb474bda3ec93a41905 HTTP/1.1" 200 None 2026-04-03 08:56:33,137 - lava_callback - ERROR - Error processing callback for node 69ccddb474bda3ec93a41905: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45384 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:56:51,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:56:51,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60048 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:57:19,164 - lava_callback - INFO - Received callback for node 69cf80a7000c489abc2c04a0 INFO: 172.21.0.1:60056 - "POST /node/69cf80a7000c489abc2c04a0 HTTP/1.0" 202 Accepted 2026-04-03 08:57:19,175 - lava_callback - INFO - Starting async job submit for node 69cf80a7000c489abc2c04a0 2026-04-03 08:57:19,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:57:19,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf80a7000c489abc2c04a0 HTTP/1.1" 200 None 2026-04-03 08:57:19,279 - lava_callback - DEBUG - Processing node 69cf80a7000c489abc2c04a0: baseline-arm64 2026-04-03 08:57:19,625 - lava_callback - WARNING - Failed to upload log 2026-04-03 08:57:19,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:57:20,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:57:20,268 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf80a7000c489abc2c04a0/lava-logs.yaml 2026-04-03 08:57:20,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:57:20,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:57:20,417 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf80a7000c489abc2c04a0/lava_callback.json.gz 2026-04-03 08:57:20,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:57:20,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf80a7000c489abc2c04a0 HTTP/1.1" 200 None 2026-04-03 08:57:20,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:57:20,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:57:20,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:57:20,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf80a7000c489abc2c04a0 HTTP/1.1" 200 None 2026-04-03 08:57:20,655 - lava_callback - INFO - Completed processing callback for node 69cf80a7000c489abc2c04a0 2026-04-03 08:57:21,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:57:21,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52748 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:58:34,484 - lava_callback - INFO - Received callback for node 69cf80f1000c489abc2c0504 INFO: 172.21.0.1:57866 - "POST /node/69cf80f1000c489abc2c0504 HTTP/1.0" 202 Accepted 2026-04-03 08:58:34,491 - lava_callback - INFO - Starting async job submit for node 69cf80f1000c489abc2c0504 2026-04-03 08:58:34,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:58:34,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf80f1000c489abc2c0504 HTTP/1.1" 200 None 2026-04-03 08:58:34,530 - lava_callback - DEBUG - Processing node 69cf80f1000c489abc2c0504: baseline-arm64 2026-04-03 08:58:34,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:58:35,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:58:35,038 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf80f1000c489abc2c0504/log.txt.gz 2026-04-03 08:58:35,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:58:35,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:58:35,475 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf80f1000c489abc2c0504/lava-logs.yaml 2026-04-03 08:58:35,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:58:35,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:58:35,626 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf80f1000c489abc2c0504/lava_callback.json.gz 2026-04-03 08:58:35,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:58:35,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf80f1000c489abc2c0504 HTTP/1.1" 200 None 2026-04-03 08:58:35,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:58:35,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:58:35,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:58:35,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf80f1000c489abc2c0504 HTTP/1.1" 200 None 2026-04-03 08:58:35,716 - lava_callback - INFO - Completed processing callback for node 69cf80f1000c489abc2c0504 INFO: 172.21.0.1:57882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48750 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:58:51,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:58:51,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40612 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:59:01,319 - lava_callback - INFO - Received callback for node 69cf7ed2000c489abc2bff6f INFO: 172.21.0.1:42376 - "POST /node/69cf7ed2000c489abc2bff6f HTTP/1.0" 202 Accepted 2026-04-03 08:59:01,325 - lava_callback - INFO - Starting async job submit for node 69cf7ed2000c489abc2bff6f 2026-04-03 08:59:01,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:01,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed2000c489abc2bff6f HTTP/1.1" 200 None 2026-04-03 08:59:01,368 - lava_callback - DEBUG - Processing node 69cf7ed2000c489abc2bff6f: baseline-arm64 2026-04-03 08:59:01,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:02,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:02,003 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed2000c489abc2bff6f/log.txt.gz 2026-04-03 08:59:02,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:02,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:02,471 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed2000c489abc2bff6f/lava-logs.yaml 2026-04-03 08:59:02,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:02,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:02,667 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed2000c489abc2bff6f/lava_callback.json.gz 2026-04-03 08:59:02,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:02,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed2000c489abc2bff6f HTTP/1.1" 200 None 2026-04-03 08:59:02,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:02,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:59:02,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:02,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ed2000c489abc2bff6f HTTP/1.1" 200 None 2026-04-03 08:59:02,859 - lava_callback - INFO - Completed processing callback for node 69cf7ed2000c489abc2bff6f 2026-04-03 08:59:03,905 - lava_callback - INFO - Received callback for node 69cf7da5000c489abc2bfe77 INFO: 172.21.0.1:42384 - "POST /node/69cf7da5000c489abc2bfe77 HTTP/1.0" 202 Accepted 2026-04-03 08:59:03,912 - lava_callback - INFO - Starting async job submit for node 69cf7da5000c489abc2bfe77 2026-04-03 08:59:03,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:03,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7da5000c489abc2bfe77 HTTP/1.1" 200 None 2026-04-03 08:59:03,972 - lava_callback - DEBUG - Processing node 69cf7da5000c489abc2bfe77: baseline-x86 2026-04-03 08:59:04,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:04,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:04,894 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da5000c489abc2bfe77/log.txt.gz 2026-04-03 08:59:05,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:05,555 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:05,558 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da5000c489abc2bfe77/lava-logs.yaml 2026-04-03 08:59:05,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:05,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:05,748 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da5000c489abc2bfe77/lava_callback.json.gz DEBUG: Setup failed for 69cf7da5000c489abc2bfe77. Transitting job node result: incomplete -> fail 2026-04-03 08:59:05,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:05,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7da5000c489abc2bfe77 HTTP/1.1" 200 None 2026-04-03 08:59:05,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:05,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77d1000c489abc2bf955 HTTP/1.1" 200 None 2026-04-03 08:59:05,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:05,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7da5000c489abc2bfe77 HTTP/1.1" 200 None 2026-04-03 08:59:06,022 - lava_callback - INFO - Completed processing callback for node 69cf7da5000c489abc2bfe77 INFO: 172.21.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40272 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:59:21,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:21,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 08:59:21,803 - lava_callback - INFO - Received callback for node 69cf7ee2000c489abc2bffa2 INFO: 172.21.0.1:55976 - "POST /node/69cf7ee2000c489abc2bffa2 HTTP/1.0" 202 Accepted 2026-04-03 08:59:21,811 - lava_callback - INFO - Starting async job submit for node 69cf7ee2000c489abc2bffa2 2026-04-03 08:59:21,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:21,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ee2000c489abc2bffa2 HTTP/1.1" 200 None 2026-04-03 08:59:21,868 - lava_callback - DEBUG - Processing node 69cf7ee2000c489abc2bffa2: baseline-nfs-arm64 2026-04-03 08:59:22,150 - lava_callback - INFO - Received callback for node 69cf7f83000c489abc2c029d INFO: 172.21.0.1:55990 - "POST /node/69cf7f83000c489abc2c029d HTTP/1.0" 202 Accepted 2026-04-03 08:59:22,175 - lava_callback - INFO - Starting async job submit for node 69cf7f83000c489abc2c029d 2026-04-03 08:59:22,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:22,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f83000c489abc2c029d HTTP/1.1" 200 None 2026-04-03 08:59:22,345 - lava_callback - DEBUG - Processing node 69cf7f83000c489abc2c029d: ltp-fsx 2026-04-03 08:59:22,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:22,818 - lava_callback - INFO - Received callback for node 69cf7f85000c489abc2c02a0 INFO: 172.21.0.1:55996 - "POST /node/69cf7f85000c489abc2c02a0 HTTP/1.0" 202 Accepted 2026-04-03 08:59:22,853 - lava_callback - INFO - Starting async job submit for node 69cf7f85000c489abc2c02a0 2026-04-03 08:59:22,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:23,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f85000c489abc2c02a0 HTTP/1.1" 200 None 2026-04-03 08:59:23,041 - lava_callback - DEBUG - Processing node 69cf7f85000c489abc2c02a0: ltp-input 2026-04-03 08:59:23,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:23,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:23,355 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee2000c489abc2bffa2/log.txt.gz 2026-04-03 08:59:23,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:23,577 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69cf7f83000c489abc2c029d/log.txt.gz 2026-04-03 08:59:23,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:24,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:24,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:24,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:24,143 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69cf7f85000c489abc2c02a0/log.txt.gz 2026-04-03 08:59:24,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:24,273 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee2000c489abc2bffa2/lava-logs.yaml 2026-04-03 08:59:24,288 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:24,340 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69cf7f83000c489abc2c029d/lava-logs.yaml 2026-04-03 08:59:24,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:24,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:24,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:24,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:24,563 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69cf7f83000c489abc2c029d/lava_callback.json.gz 2026-04-03 08:59:24,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:24,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:24,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f83000c489abc2c029d HTTP/1.1" 200 None 2026-04-03 08:59:24,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:24,611 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69cf7f85000c489abc2c02a0/lava-logs.yaml 2026-04-03 08:59:24,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:24,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:24,625 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee2000c489abc2bffa2/lava_callback.json.gz 2026-04-03 08:59:24,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:59:24,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:24,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:24,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ee2000c489abc2bffa2 HTTP/1.1" 200 None 2026-04-03 08:59:24,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:24,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f83000c489abc2c029d HTTP/1.1" 200 None 2026-04-03 08:59:24,696 - lava_callback - INFO - Completed processing callback for node 69cf7f83000c489abc2c029d 2026-04-03 08:59:24,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:59:24,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:24,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ee2000c489abc2bffa2 HTTP/1.1" 200 None 2026-04-03 08:59:24,867 - lava_callback - INFO - Completed processing callback for node 69cf7ee2000c489abc2bffa2 2026-04-03 08:59:24,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:24,929 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69cf7f85000c489abc2c02a0/lava_callback.json.gz 2026-04-03 08:59:24,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:24,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f85000c489abc2c02a0 HTTP/1.1" 200 None 2026-04-03 08:59:25,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:25,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 08:59:25,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:25,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f85000c489abc2c02a0 HTTP/1.1" 200 None 2026-04-03 08:59:25,075 - lava_callback - INFO - Completed processing callback for node 69cf7f85000c489abc2c02a0 INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:59:34,740 - lava_callback - INFO - Received callback for node 69cf8138000c489abc2c0534 INFO: 172.21.0.1:40920 - "POST /node/69cf8138000c489abc2c0534 HTTP/1.0" 202 Accepted 2026-04-03 08:59:34,746 - lava_callback - INFO - Starting async job submit for node 69cf8138000c489abc2c0534 2026-04-03 08:59:34,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:34,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8138000c489abc2c0534 HTTP/1.1" 200 None 2026-04-03 08:59:34,784 - lava_callback - DEBUG - Processing node 69cf8138000c489abc2c0534: baseline-arm64 2026-04-03 08:59:35,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:35,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:35,208 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8138000c489abc2c0534/log.txt.gz 2026-04-03 08:59:35,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:35,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:35,600 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8138000c489abc2c0534/lava-logs.yaml 2026-04-03 08:59:35,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 08:59:35,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 08:59:35,745 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8138000c489abc2c0534/lava_callback.json.gz 2026-04-03 08:59:35,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:35,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8138000c489abc2c0534 HTTP/1.1" 200 None 2026-04-03 08:59:35,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:35,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 08:59:35,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:35,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8138000c489abc2c0534 HTTP/1.1" 200 None 2026-04-03 08:59:35,961 - lava_callback - INFO - Completed processing callback for node 69cf8138000c489abc2c0534 INFO: 172.21.0.1:40930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54636 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:59:51,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:59:51,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53940 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:00:02,775 - lava_callback - INFO - Received callback for node 69cf7f88000c489abc2c02a9 INFO: 172.21.0.1:48846 - "POST /node/69cf7f88000c489abc2c02a9 HTTP/1.0" 202 Accepted 2026-04-03 09:00:02,781 - lava_callback - INFO - Starting async job submit for node 69cf7f88000c489abc2c02a9 2026-04-03 09:00:02,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:00:02,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f88000c489abc2c02a9 HTTP/1.1" 200 None 2026-04-03 09:00:02,824 - lava_callback - DEBUG - Processing node 69cf7f88000c489abc2c02a9: ltp-pty 2026-04-03 09:00:03,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:00:03,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:00:03,386 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69cf7f88000c489abc2c02a9/log.txt.gz 2026-04-03 09:00:03,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:00:03,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:00:03,975 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69cf7f88000c489abc2c02a9/lava-logs.yaml 2026-04-03 09:00:03,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:00:04,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:00:04,163 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69cf7f88000c489abc2c02a9/lava_callback.json.gz 2026-04-03 09:00:04,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:00:04,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f88000c489abc2c02a9 HTTP/1.1" 200 None 2026-04-03 09:00:04,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:00:04,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:00:04,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:00:04,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f88000c489abc2c02a9 HTTP/1.1" 200 None 2026-04-03 09:00:04,259 - lava_callback - INFO - Completed processing callback for node 69cf7f88000c489abc2c02a9 INFO: 172.21.0.1:48852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48746 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:00:21,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:00:21,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 09:00:23,069 - lava_callback - INFO - Received callback for node 69cf7f8b000c489abc2c02bb INFO: 172.21.0.1:46840 - "POST /node/69cf7f8b000c489abc2c02bb HTTP/1.0" 202 Accepted 2026-04-03 09:00:23,075 - lava_callback - INFO - Starting async job submit for node 69cf7f8b000c489abc2c02bb 2026-04-03 09:00:23,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:00:23,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f8b000c489abc2c02bb HTTP/1.1" 200 None 2026-04-03 09:00:23,115 - lava_callback - DEBUG - Processing node 69cf7f8b000c489abc2c02bb: ltp-smoke 2026-04-03 09:00:23,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:00:23,598 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:00:23,600 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf7f8b000c489abc2c02bb/log.txt.gz 2026-04-03 09:00:23,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:00:24,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:00:24,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf7f8b000c489abc2c02bb/lava-logs.yaml 2026-04-03 09:00:24,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:00:24,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:00:24,229 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf7f8b000c489abc2c02bb/lava_callback.json.gz 2026-04-03 09:00:24,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:00:24,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f8b000c489abc2c02bb HTTP/1.1" 200 None 2026-04-03 09:00:24,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:00:24,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:00:24,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:00:24,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f8b000c489abc2c02bb HTTP/1.1" 200 None 2026-04-03 09:00:24,310 - lava_callback - INFO - Completed processing callback for node 69cf7f8b000c489abc2c02bb INFO: 172.21.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38024 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:00:51,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:00:51,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57326 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:01:20,673 - lava_callback - INFO - Received callback for node 69ccdedc74bda3ec93a419f3 INFO: 172.21.0.1:38860 - "POST /node/69ccdedc74bda3ec93a419f3 HTTP/1.0" 202 Accepted 2026-04-03 09:01:20,679 - lava_callback - INFO - Starting async job submit for node 69ccdedc74bda3ec93a419f3 2026-04-03 09:01:20,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:20,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdedc74bda3ec93a419f3 HTTP/1.1" 200 None 2026-04-03 09:01:20,695 - lava_callback - DEBUG - Processing node 69ccdedc74bda3ec93a419f3: baseline-arm64-baylibre 2026-04-03 09:01:20,695 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdedc74bda3ec93a419f3 2026-04-03 09:01:20,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:20,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdedc74bda3ec93a419f3 HTTP/1.1" 200 None 2026-04-03 09:01:20,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:20,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 09:01:20,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:20,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdedc74bda3ec93a419f3 HTTP/1.1" 200 None 2026-04-03 09:01:20,745 - lava_callback - ERROR - Error processing callback for node 69ccdedc74bda3ec93a419f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39050 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:01:38,584 - lava_callback - INFO - Received callback for node 69cf81dd000c489abc2c060e INFO: 172.21.0.1:39058 - "POST /node/69cf81dd000c489abc2c060e HTTP/1.0" 202 Accepted 2026-04-03 09:01:38,590 - lava_callback - INFO - Starting async job submit for node 69cf81dd000c489abc2c060e 2026-04-03 09:01:38,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:38,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf81dd000c489abc2c060e HTTP/1.1" 200 None 2026-04-03 09:01:38,609 - lava_callback - DEBUG - Processing node 69cf81dd000c489abc2c060e: baseline-arm 2026-04-03 09:01:38,614 - lava_callback - WARNING - Failed to upload log 2026-04-03 09:01:38,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:01:38,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:01:38,948 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf81dd000c489abc2c060e/lava-logs.yaml 2026-04-03 09:01:38,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:01:39,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:01:39,237 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf81dd000c489abc2c060e/lava_callback.json.gz 2026-04-03 09:01:39,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:39,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf81dd000c489abc2c060e HTTP/1.1" 200 None 2026-04-03 09:01:39,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:39,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:01:39,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:39,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf81dd000c489abc2c060e HTTP/1.1" 200 None 2026-04-03 09:01:39,305 - lava_callback - INFO - Completed processing callback for node 69cf81dd000c489abc2c060e 2026-04-03 09:01:40,502 - lava_callback - INFO - Received callback for node 69ccdedf74bda3ec93a419fc INFO: 172.21.0.1:43378 - "POST /node/69ccdedf74bda3ec93a419fc HTTP/1.0" 202 Accepted 2026-04-03 09:01:40,507 - lava_callback - INFO - Starting async job submit for node 69ccdedf74bda3ec93a419fc 2026-04-03 09:01:40,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:40,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdedf74bda3ec93a419fc HTTP/1.1" 200 None 2026-04-03 09:01:40,544 - lava_callback - DEBUG - Processing node 69ccdedf74bda3ec93a419fc: baseline-arm64-baylibre 2026-04-03 09:01:40,544 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdedf74bda3ec93a419fc 2026-04-03 09:01:40,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:40,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdedf74bda3ec93a419fc HTTP/1.1" 200 None 2026-04-03 09:01:40,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:40,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:40,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:40,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdedf74bda3ec93a419fc HTTP/1.1" 200 None 2026-04-03 09:01:40,603 - lava_callback - ERROR - Error processing callback for node 69ccdedf74bda3ec93a419fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 09:01:41,187 - lava_callback - INFO - Received callback for node 69ccdee174bda3ec93a419ff INFO: 172.21.0.1:43392 - "POST /node/69ccdee174bda3ec93a419ff HTTP/1.0" 202 Accepted 2026-04-03 09:01:41,193 - lava_callback - INFO - Starting async job submit for node 69ccdee174bda3ec93a419ff 2026-04-03 09:01:41,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:41,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdee174bda3ec93a419ff HTTP/1.1" 200 None 2026-04-03 09:01:41,209 - lava_callback - DEBUG - Processing node 69ccdee174bda3ec93a419ff: baseline-arm64-baylibre 2026-04-03 09:01:41,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdee174bda3ec93a419ff 2026-04-03 09:01:41,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:41,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdee174bda3ec93a419ff HTTP/1.1" 200 None 2026-04-03 09:01:41,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:41,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:41,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:41,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdee174bda3ec93a419ff HTTP/1.1" 200 None 2026-04-03 09:01:41,265 - lava_callback - ERROR - Error processing callback for node 69ccdee174bda3ec93a419ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 09:01:41,864 - lava_callback - INFO - Received callback for node 69ccdee374bda3ec93a41a06 INFO: 172.21.0.1:43404 - "POST /node/69ccdee374bda3ec93a41a06 HTTP/1.0" 202 Accepted 2026-04-03 09:01:41,870 - lava_callback - INFO - Starting async job submit for node 69ccdee374bda3ec93a41a06 2026-04-03 09:01:41,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:41,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdee374bda3ec93a41a06 HTTP/1.1" 200 None 2026-04-03 09:01:41,886 - lava_callback - DEBUG - Processing node 69ccdee374bda3ec93a41a06: baseline-arm64-baylibre 2026-04-03 09:01:41,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdee374bda3ec93a41a06 2026-04-03 09:01:41,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:41,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdee374bda3ec93a41a06 HTTP/1.1" 200 None 2026-04-03 09:01:41,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:41,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:41,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:41,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdee374bda3ec93a41a06 HTTP/1.1" 200 None 2026-04-03 09:01:41,951 - lava_callback - ERROR - Error processing callback for node 69ccdee374bda3ec93a41a06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 09:01:42,552 - lava_callback - INFO - Received callback for node 69ccdee574bda3ec93a41a26 INFO: 172.21.0.1:43416 - "POST /node/69ccdee574bda3ec93a41a26 HTTP/1.0" 202 Accepted 2026-04-03 09:01:42,557 - lava_callback - INFO - Starting async job submit for node 69ccdee574bda3ec93a41a26 2026-04-03 09:01:42,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:42,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdee574bda3ec93a41a26 HTTP/1.1" 200 None 2026-04-03 09:01:42,573 - lava_callback - DEBUG - Processing node 69ccdee574bda3ec93a41a26: baseline-arm64-baylibre 2026-04-03 09:01:42,574 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdee574bda3ec93a41a26 2026-04-03 09:01:42,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:42,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdee574bda3ec93a41a26 HTTP/1.1" 200 None 2026-04-03 09:01:42,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:42,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:42,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:42,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdee574bda3ec93a41a26 HTTP/1.1" 200 None 2026-04-03 09:01:42,625 - lava_callback - ERROR - Error processing callback for node 69ccdee574bda3ec93a41a26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 09:01:43,219 - lava_callback - INFO - Received callback for node 69ccdee674bda3ec93a41a29 INFO: 172.21.0.1:43426 - "POST /node/69ccdee674bda3ec93a41a29 HTTP/1.0" 202 Accepted 2026-04-03 09:01:43,225 - lava_callback - INFO - Starting async job submit for node 69ccdee674bda3ec93a41a29 2026-04-03 09:01:43,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:43,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdee674bda3ec93a41a29 HTTP/1.1" 200 None 2026-04-03 09:01:43,242 - lava_callback - DEBUG - Processing node 69ccdee674bda3ec93a41a29: baseline-arm64-baylibre 2026-04-03 09:01:43,242 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdee674bda3ec93a41a29 2026-04-03 09:01:43,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:43,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdee674bda3ec93a41a29 HTTP/1.1" 200 None 2026-04-03 09:01:43,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:43,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:43,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:43,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdee674bda3ec93a41a29 HTTP/1.1" 200 None 2026-04-03 09:01:43,299 - lava_callback - ERROR - Error processing callback for node 69ccdee674bda3ec93a41a29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43438 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:01:51,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:51,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46862 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:01:53,406 - lava_callback - INFO - Received callback for node 69ccdef674bda3ec93a41a46 INFO: 172.21.0.1:46878 - "POST /node/69ccdef674bda3ec93a41a46 HTTP/1.0" 202 Accepted 2026-04-03 09:01:53,411 - lava_callback - INFO - Starting async job submit for node 69ccdef674bda3ec93a41a46 2026-04-03 09:01:53,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:53,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdef674bda3ec93a41a46 HTTP/1.1" 200 None 2026-04-03 09:01:53,428 - lava_callback - DEBUG - Processing node 69ccdef674bda3ec93a41a46: baseline-arm64-clabbe 2026-04-03 09:01:53,428 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdef674bda3ec93a41a46 2026-04-03 09:01:53,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:53,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdef674bda3ec93a41a46 HTTP/1.1" 200 None 2026-04-03 09:01:53,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:53,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:53,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:53,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdef674bda3ec93a41a46 HTTP/1.1" 200 None 2026-04-03 09:01:53,499 - lava_callback - ERROR - Error processing callback for node 69ccdef674bda3ec93a41a46: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 09:01:54,100 - lava_callback - INFO - Received callback for node 69ccdef974bda3ec93a41a4d INFO: 172.21.0.1:46892 - "POST /node/69ccdef974bda3ec93a41a4d HTTP/1.0" 202 Accepted 2026-04-03 09:01:54,105 - lava_callback - INFO - Starting async job submit for node 69ccdef974bda3ec93a41a4d 2026-04-03 09:01:54,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:54,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdef974bda3ec93a41a4d HTTP/1.1" 200 None 2026-04-03 09:01:54,122 - lava_callback - DEBUG - Processing node 69ccdef974bda3ec93a41a4d: baseline-arm64-clabbe 2026-04-03 09:01:54,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdef974bda3ec93a41a4d 2026-04-03 09:01:54,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:54,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdef974bda3ec93a41a4d HTTP/1.1" 200 None 2026-04-03 09:01:54,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:54,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:54,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:54,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdef974bda3ec93a41a4d HTTP/1.1" 200 None 2026-04-03 09:01:54,184 - lava_callback - ERROR - Error processing callback for node 69ccdef974bda3ec93a41a4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 09:01:54,811 - lava_callback - INFO - Received callback for node 69ccdef974bda3ec93a41a50 INFO: 172.21.0.1:46896 - "POST /node/69ccdef974bda3ec93a41a50 HTTP/1.0" 202 Accepted 2026-04-03 09:01:54,817 - lava_callback - INFO - Starting async job submit for node 69ccdef974bda3ec93a41a50 2026-04-03 09:01:54,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:54,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdef974bda3ec93a41a50 HTTP/1.1" 200 None 2026-04-03 09:01:54,834 - lava_callback - DEBUG - Processing node 69ccdef974bda3ec93a41a50: baseline-arm64-clabbe 2026-04-03 09:01:54,834 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdef974bda3ec93a41a50 2026-04-03 09:01:54,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:54,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdef974bda3ec93a41a50 HTTP/1.1" 200 None 2026-04-03 09:01:54,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:54,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:54,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:54,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdef974bda3ec93a41a50 HTTP/1.1" 200 None 2026-04-03 09:01:54,890 - lava_callback - ERROR - Error processing callback for node 69ccdef974bda3ec93a41a50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 09:01:55,501 - lava_callback - INFO - Received callback for node 69ccdefa74bda3ec93a41a53 INFO: 172.21.0.1:46904 - "POST /node/69ccdefa74bda3ec93a41a53 HTTP/1.0" 202 Accepted 2026-04-03 09:01:55,506 - lava_callback - INFO - Starting async job submit for node 69ccdefa74bda3ec93a41a53 2026-04-03 09:01:55,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:55,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdefa74bda3ec93a41a53 HTTP/1.1" 200 None 2026-04-03 09:01:55,522 - lava_callback - DEBUG - Processing node 69ccdefa74bda3ec93a41a53: baseline-arm64-clabbe 2026-04-03 09:01:55,522 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdefa74bda3ec93a41a53 2026-04-03 09:01:55,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:55,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdefa74bda3ec93a41a53 HTTP/1.1" 200 None 2026-04-03 09:01:55,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:55,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:55,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:55,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdefa74bda3ec93a41a53 HTTP/1.1" 200 None 2026-04-03 09:01:55,574 - lava_callback - ERROR - Error processing callback for node 69ccdefa74bda3ec93a41a53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 09:01:56,196 - lava_callback - INFO - Received callback for node 69ccdefd74bda3ec93a41a60 INFO: 172.21.0.1:46906 - "POST /node/69ccdefd74bda3ec93a41a60 HTTP/1.0" 202 Accepted 2026-04-03 09:01:56,202 - lava_callback - INFO - Starting async job submit for node 69ccdefd74bda3ec93a41a60 2026-04-03 09:01:56,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:56,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdefd74bda3ec93a41a60 HTTP/1.1" 200 None 2026-04-03 09:01:56,218 - lava_callback - DEBUG - Processing node 69ccdefd74bda3ec93a41a60: baseline-arm64-clabbe 2026-04-03 09:01:56,218 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdefd74bda3ec93a41a60 2026-04-03 09:01:56,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:56,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdefd74bda3ec93a41a60 HTTP/1.1" 200 None 2026-04-03 09:01:56,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:56,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:56,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:56,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdefd74bda3ec93a41a60 HTTP/1.1" 200 None 2026-04-03 09:01:56,301 - lava_callback - ERROR - Error processing callback for node 69ccdefd74bda3ec93a41a60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46910 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:01:56,893 - lava_callback - INFO - Received callback for node 69ccdefc74bda3ec93a41a5d INFO: 172.21.0.1:46914 - "POST /node/69ccdefc74bda3ec93a41a5d HTTP/1.0" 202 Accepted 2026-04-03 09:01:56,899 - lava_callback - INFO - Starting async job submit for node 69ccdefc74bda3ec93a41a5d 2026-04-03 09:01:56,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:56,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdefc74bda3ec93a41a5d HTTP/1.1" 200 None 2026-04-03 09:01:56,914 - lava_callback - DEBUG - Processing node 69ccdefc74bda3ec93a41a5d: baseline-arm64-clabbe 2026-04-03 09:01:56,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdefc74bda3ec93a41a5d 2026-04-03 09:01:56,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:56,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdefc74bda3ec93a41a5d HTTP/1.1" 200 None 2026-04-03 09:01:56,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:56,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 09:01:56,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:01:56,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdefc74bda3ec93a41a5d HTTP/1.1" 200 None 2026-04-03 09:01:56,981 - lava_callback - ERROR - Error processing callback for node 69ccdefc74bda3ec93a41a5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46642 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:03:23,197 - lava_callback - INFO - Received callback for node 69cf7f92000c489abc2c02cc INFO: 172.21.0.1:60326 - "POST /node/69cf7f92000c489abc2c02cc HTTP/1.0" 202 Accepted 2026-04-03 09:03:23,204 - lava_callback - INFO - Starting async job submit for node 69cf7f92000c489abc2c02cc 2026-04-03 09:03:23,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:03:23,234 - lava_callback - INFO - Received callback for node 69cf7f8e000c489abc2c02c5 INFO: 172.21.0.1:60332 - "POST /node/69cf7f8e000c489abc2c02c5 HTTP/1.0" 202 Accepted 2026-04-03 09:03:23,241 - lava_callback - INFO - Starting async job submit for node 69cf7f8e000c489abc2c02c5 2026-04-03 09:03:23,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:03:23,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f92000c489abc2c02cc HTTP/1.1" 200 None 2026-04-03 09:03:23,274 - lava_callback - DEBUG - Processing node 69cf7f92000c489abc2c02cc: ltp-timers 2026-04-03 09:03:23,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f8e000c489abc2c02c5 HTTP/1.1" 200 None 2026-04-03 09:03:23,383 - lava_callback - DEBUG - Processing node 69cf7f8e000c489abc2c02c5: ltp-syscalls-ipc 2026-04-03 09:03:24,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:03:24,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:03:24,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:03:24,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:03:24,200 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cf7f8e000c489abc2c02c5/log.txt.gz 2026-04-03 09:03:24,201 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69cf7f92000c489abc2c02cc/log.txt.gz 2026-04-03 09:03:24,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:03:24,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:03:24,835 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:03:24,837 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cf7f8e000c489abc2c02c5/lava-logs.yaml 2026-04-03 09:03:24,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:03:24,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:03:24,927 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69cf7f92000c489abc2c02cc/lava-logs.yaml 2026-04-03 09:03:24,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:03:25,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:03:25,022 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cf7f8e000c489abc2c02c5/lava_callback.json.gz 2026-04-03 09:03:25,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:03:25,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f8e000c489abc2c02c5 HTTP/1.1" 200 None 2026-04-03 09:03:25,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:03:25,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:03:25,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:03:25,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f8e000c489abc2c02c5 HTTP/1.1" 200 None 2026-04-03 09:03:25,108 - lava_callback - INFO - Completed processing callback for node 69cf7f8e000c489abc2c02c5 2026-04-03 09:03:25,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:03:25,342 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69cf7f92000c489abc2c02cc/lava_callback.json.gz 2026-04-03 09:03:25,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:03:25,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f92000c489abc2c02cc HTTP/1.1" 200 None 2026-04-03 09:03:25,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:03:25,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:03:25,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:03:25,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f92000c489abc2c02cc HTTP/1.1" 200 None 2026-04-03 09:03:25,443 - lava_callback - INFO - Completed processing callback for node 69cf7f92000c489abc2c02cc INFO: 172.21.0.1:60338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33690 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:03:51,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:03:51,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36460 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:04:20,738 - lava_callback - INFO - Received callback for node 69ccdf8a74bda3ec93a41b64 INFO: 172.21.0.1:50758 - "POST /node/69ccdf8a74bda3ec93a41b64 HTTP/1.0" 202 Accepted 2026-04-03 09:04:20,744 - lava_callback - INFO - Starting async job submit for node 69ccdf8a74bda3ec93a41b64 2026-04-03 09:04:20,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:20,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdf8a74bda3ec93a41b64 HTTP/1.1" 200 None 2026-04-03 09:04:20,766 - lava_callback - DEBUG - Processing node 69ccdf8a74bda3ec93a41b64: baseline-arm64-baylibre 2026-04-03 09:04:20,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdf8a74bda3ec93a41b64 2026-04-03 09:04:20,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:20,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdf8a74bda3ec93a41b64 HTTP/1.1" 200 None 2026-04-03 09:04:20,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:20,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 09:04:20,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:20,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdf8a74bda3ec93a41b64 HTTP/1.1" 200 None 2026-04-03 09:04:20,822 - lava_callback - ERROR - Error processing callback for node 69ccdf8a74bda3ec93a41b64: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 09:04:22,278 - lava_callback - INFO - Received callback for node 69cf8242000c489abc2c0699 INFO: 172.21.0.1:50770 - "POST /node/69cf8242000c489abc2c0699 HTTP/1.0" 202 Accepted 2026-04-03 09:04:22,285 - lava_callback - INFO - Starting async job submit for node 69cf8242000c489abc2c0699 2026-04-03 09:04:22,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:22,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8242000c489abc2c0699 HTTP/1.1" 200 None 2026-04-03 09:04:22,335 - lava_callback - DEBUG - Processing node 69cf8242000c489abc2c0699: baseline-arm-broonie 2026-04-03 09:04:22,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:04:22,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:04:22,677 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf8242000c489abc2c0699/log.txt.gz 2026-04-03 09:04:22,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:04:22,974 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:04:22,976 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf8242000c489abc2c0699/lava-logs.yaml 2026-04-03 09:04:22,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:04:23,008 - lava_callback - INFO - Received callback for node 69cf7f94000c489abc2c02d3 INFO: 172.21.0.1:50784 - "POST /node/69cf7f94000c489abc2c02d3 HTTP/1.0" 202 Accepted 2026-04-03 09:04:23,018 - lava_callback - INFO - Starting async job submit for node 69cf7f94000c489abc2c02d3 2026-04-03 09:04:23,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:23,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f94000c489abc2c02d3 HTTP/1.1" 200 None 2026-04-03 09:04:23,062 - lava_callback - DEBUG - Processing node 69cf7f94000c489abc2c02d3: ltp-watchqueue 2026-04-03 09:04:23,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:04:23,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf8242000c489abc2c0699/lava_callback.json.gz 2026-04-03 09:04:23,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:23,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8242000c489abc2c0699 HTTP/1.1" 200 None 2026-04-03 09:04:23,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:23,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:04:23,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:23,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:04:23,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8242000c489abc2c0699 HTTP/1.1" 200 None 2026-04-03 09:04:23,540 - lava_callback - INFO - Completed processing callback for node 69cf8242000c489abc2c0699 2026-04-03 09:04:24,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:04:24,558 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cf7f94000c489abc2c02d3/log.txt.gz 2026-04-03 09:04:24,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:04:25,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:04:25,083 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cf7f94000c489abc2c02d3/lava-logs.yaml 2026-04-03 09:04:25,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:04:25,259 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:04:25,261 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cf7f94000c489abc2c02d3/lava_callback.json.gz 2026-04-03 09:04:25,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:25,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f94000c489abc2c02d3 HTTP/1.1" 200 None 2026-04-03 09:04:25,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:25,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:04:25,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:25,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f94000c489abc2c02d3 HTTP/1.1" 200 None 2026-04-03 09:04:25,370 - lava_callback - INFO - Completed processing callback for node 69cf7f94000c489abc2c02d3 INFO: 172.21.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42302 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:04:51,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:04:51,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45242 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:05:40,825 - lava_callback - INFO - Received callback for node 69ccdfe074bda3ec93a41b91 INFO: 172.21.0.1:58058 - "POST /node/69ccdfe074bda3ec93a41b91 HTTP/1.0" 202 Accepted 2026-04-03 09:05:40,833 - lava_callback - INFO - Starting async job submit for node 69ccdfe074bda3ec93a41b91 2026-04-03 09:05:40,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:05:40,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdfe074bda3ec93a41b91 HTTP/1.1" 200 None 2026-04-03 09:05:40,850 - lava_callback - DEBUG - Processing node 69ccdfe074bda3ec93a41b91: baseline-arm64-baylibre 2026-04-03 09:05:40,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccdfe074bda3ec93a41b91 2026-04-03 09:05:40,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:05:40,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccdfe074bda3ec93a41b91 HTTP/1.1" 200 None 2026-04-03 09:05:40,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:05:40,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 09:05:40,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:05:40,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccdfe074bda3ec93a41b91 HTTP/1.1" 200 None 2026-04-03 09:05:40,920 - lava_callback - ERROR - Error processing callback for node 69ccdfe074bda3ec93a41b91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48804 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:07:20,916 - lava_callback - INFO - Received callback for node 69cce03574bda3ec93a41bb1 INFO: 172.21.0.1:52646 - "POST /node/69cce03574bda3ec93a41bb1 HTTP/1.0" 202 Accepted 2026-04-03 09:07:20,922 - lava_callback - INFO - Starting async job submit for node 69cce03574bda3ec93a41bb1 2026-04-03 09:07:20,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:07:20,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce03574bda3ec93a41bb1 HTTP/1.1" 200 None 2026-04-03 09:07:20,939 - lava_callback - DEBUG - Processing node 69cce03574bda3ec93a41bb1: baseline-arm64-baylibre 2026-04-03 09:07:20,939 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce03574bda3ec93a41bb1 2026-04-03 09:07:20,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:07:20,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce03574bda3ec93a41bb1 HTTP/1.1" 200 None 2026-04-03 09:07:20,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:07:20,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 09:07:20,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:07:20,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce03574bda3ec93a41bb1 HTTP/1.1" 200 None 2026-04-03 09:07:20,994 - lava_callback - ERROR - Error processing callback for node 69cce03574bda3ec93a41bb1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44114 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:07:52,675 - lava_callback - INFO - Received callback for node 69cf7f3c000c489abc2c0102 INFO: 172.21.0.1:44118 - "POST /node/69cf7f3c000c489abc2c0102 HTTP/1.0" 202 Accepted 2026-04-03 09:07:52,682 - lava_callback - INFO - Starting async job submit for node 69cf7f3c000c489abc2c0102 2026-04-03 09:07:52,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:07:52,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3c000c489abc2c0102 HTTP/1.1" 200 None 2026-04-03 09:07:52,752 - lava_callback - DEBUG - Processing node 69cf7f3c000c489abc2c0102: baseline-arm 2026-04-03 09:07:53,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:07:53,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:07:53,642 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3c000c489abc2c0102/log.txt.gz 2026-04-03 09:07:54,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:07:54,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:07:54,283 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3c000c489abc2c0102/lava-logs.yaml 2026-04-03 09:07:54,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:07:54,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:07:54,466 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf7f3c000c489abc2c0102/lava_callback.json.gz DEBUG: Setup failed for 69cf7f3c000c489abc2c0102. Transitting job node result: incomplete -> fail 2026-04-03 09:07:54,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:07:54,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f3c000c489abc2c0102 HTTP/1.1" 200 None 2026-04-03 09:07:54,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:07:54,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:07:54,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:07:54,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f3c000c489abc2c0102 HTTP/1.1" 200 None 2026-04-03 09:07:54,743 - lava_callback - INFO - Completed processing callback for node 69cf7f3c000c489abc2c0102 INFO: 172.21.0.1:44126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37794 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:08:21,043 - lava_callback - INFO - Received callback for node 69cf7edc000c489abc2bff93 INFO: 172.21.0.1:42216 - "POST /node/69cf7edc000c489abc2bff93 HTTP/1.0" 202 Accepted 2026-04-03 09:08:21,053 - lava_callback - INFO - Starting async job submit for node 69cf7edc000c489abc2bff93 2026-04-03 09:08:21,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:08:21,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7edc000c489abc2bff93 HTTP/1.1" 200 None 2026-04-03 09:08:21,120 - lava_callback - DEBUG - Processing node 69cf7edc000c489abc2bff93: baseline-nfs-arm64 2026-04-03 09:08:21,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:08:21,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 09:08:22,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:08:22,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:08:22,521 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edc000c489abc2bff93/log.txt.gz 2026-04-03 09:08:23,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:08:23,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:08:23,379 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edc000c489abc2bff93/lava-logs.yaml 2026-04-03 09:08:23,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:08:23,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:08:23,573 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7edc000c489abc2bff93/lava_callback.json.gz DEBUG: Setup failed for 69cf7edc000c489abc2bff93. Transitting job node result: incomplete -> fail 2026-04-03 09:08:23,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:08:23,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7edc000c489abc2bff93 HTTP/1.1" 200 None 2026-04-03 09:08:23,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:08:23,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:08:23,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:08:23,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7edc000c489abc2bff93 HTTP/1.1" 200 None 2026-04-03 09:08:23,818 - lava_callback - INFO - Completed processing callback for node 69cf7edc000c489abc2bff93 INFO: 172.21.0.1:42226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48494 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:08:51,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:08:51,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36434 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:08:53,645 - lava_callback - INFO - Received callback for node 69cce08f74bda3ec93a41bcb INFO: 172.21.0.1:36446 - "POST /node/69cce08f74bda3ec93a41bcb HTTP/1.0" 202 Accepted 2026-04-03 09:08:53,651 - lava_callback - INFO - Starting async job submit for node 69cce08f74bda3ec93a41bcb 2026-04-03 09:08:53,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:08:53,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce08f74bda3ec93a41bcb HTTP/1.1" 200 None 2026-04-03 09:08:53,667 - lava_callback - DEBUG - Processing node 69cce08f74bda3ec93a41bcb: baseline-arm64-clabbe 2026-04-03 09:08:53,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce08f74bda3ec93a41bcb 2026-04-03 09:08:53,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:08:53,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce08f74bda3ec93a41bcb HTTP/1.1" 200 None 2026-04-03 09:08:53,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:08:53,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 09:08:53,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:08:53,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce08f74bda3ec93a41bcb HTTP/1.1" 200 None 2026-04-03 09:08:53,726 - lava_callback - ERROR - Error processing callback for node 69cce08f74bda3ec93a41bcb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60644 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:10:13,931 - lava_callback - INFO - Received callback for node 69cce0e574bda3ec93a41bed INFO: 172.21.0.1:53366 - "POST /node/69cce0e574bda3ec93a41bed HTTP/1.0" 202 Accepted 2026-04-03 09:10:13,937 - lava_callback - INFO - Starting async job submit for node 69cce0e574bda3ec93a41bed 2026-04-03 09:10:13,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:13,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce0e574bda3ec93a41bed HTTP/1.1" 200 None 2026-04-03 09:10:13,957 - lava_callback - DEBUG - Processing node 69cce0e574bda3ec93a41bed: baseline-arm64-clabbe 2026-04-03 09:10:13,957 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce0e574bda3ec93a41bed 2026-04-03 09:10:13,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:13,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce0e574bda3ec93a41bed HTTP/1.1" 200 None 2026-04-03 09:10:13,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:14,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 09:10:14,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:14,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce0e574bda3ec93a41bed HTTP/1.1" 200 None 2026-04-03 09:10:14,035 - lava_callback - ERROR - Error processing callback for node 69cce0e574bda3ec93a41bed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53378 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 09:10:15,850 - lava_callback - INFO - Received callback for node 69cf7f66000c489abc2c0231 INFO: 172.21.0.1:53390 - "POST /node/69cf7f66000c489abc2c0231 HTTP/1.0" 202 Accepted 2026-04-03 09:10:15,897 - lava_callback - INFO - Starting async job submit for node 69cf7f66000c489abc2c0231 2026-04-03 09:10:15,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:15,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f66000c489abc2c0231 HTTP/1.1" 200 None 2026-04-03 09:10:15,937 - lava_callback - DEBUG - Processing node 69cf7f66000c489abc2c0231: baseline-arm-pengutronix 2026-04-03 09:10:16,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:10:16,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:10:16,191 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f66000c489abc2c0231/log.txt.gz 2026-04-03 09:10:16,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:10:16,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:10:16,424 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f66000c489abc2c0231/lava-logs.yaml 2026-04-03 09:10:16,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:53404 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:10:16,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:10:16,565 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf7f66000c489abc2c0231/lava_callback.json.gz DEBUG: Setup failed for 69cf7f66000c489abc2c0231. Transitting job node result: incomplete -> fail 2026-04-03 09:10:16,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:16,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f66000c489abc2c0231 HTTP/1.1" 200 None 2026-04-03 09:10:16,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:16,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:10:16,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:16,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f66000c489abc2c0231 HTTP/1.1" 200 None 2026-04-03 09:10:16,754 - lava_callback - INFO - Completed processing callback for node 69cf7f66000c489abc2c0231 2026-04-03 09:10:21,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:21,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33518 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:10:57,016 - lava_callback - INFO - Received callback for node 69cf7ed1000c489abc2bff6c INFO: 172.21.0.1:33530 - "POST /node/69cf7ed1000c489abc2bff6c HTTP/1.0" 202 Accepted 2026-04-03 09:10:57,022 - lava_callback - INFO - Starting async job submit for node 69cf7ed1000c489abc2bff6c 2026-04-03 09:10:57,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:57,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed1000c489abc2bff6c HTTP/1.1" 200 None 2026-04-03 09:10:57,058 - lava_callback - DEBUG - Processing node 69cf7ed1000c489abc2bff6c: baseline-arm64 2026-04-03 09:10:57,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:10:57,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:10:57,435 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed1000c489abc2bff6c/log.txt.gz 2026-04-03 09:10:57,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:10:57,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:10:57,763 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed1000c489abc2bff6c/lava-logs.yaml 2026-04-03 09:10:57,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:10:57,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:10:57,911 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ed1000c489abc2bff6c/lava_callback.json.gz 2026-04-03 09:10:57,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:57,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed1000c489abc2bff6c HTTP/1.1" 200 None 2026-04-03 09:10:57,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:57,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:10:57,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:10:57,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ed1000c489abc2bff6c HTTP/1.1" 200 None 2026-04-03 09:10:57,991 - lava_callback - INFO - Completed processing callback for node 69cf7ed1000c489abc2bff6c INFO: 172.21.0.1:33992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59802 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:11:21,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:11:21,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53752 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:11:33,916 - lava_callback - INFO - Received callback for node 69cce13c74bda3ec93a41bfd INFO: 172.21.0.1:37068 - "POST /node/69cce13c74bda3ec93a41bfd HTTP/1.0" 202 Accepted 2026-04-03 09:11:33,921 - lava_callback - INFO - Starting async job submit for node 69cce13c74bda3ec93a41bfd 2026-04-03 09:11:33,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:11:33,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce13c74bda3ec93a41bfd HTTP/1.1" 200 None 2026-04-03 09:11:33,937 - lava_callback - DEBUG - Processing node 69cce13c74bda3ec93a41bfd: baseline-arm64-clabbe 2026-04-03 09:11:33,937 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce13c74bda3ec93a41bfd 2026-04-03 09:11:33,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:11:33,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce13c74bda3ec93a41bfd HTTP/1.1" 200 None 2026-04-03 09:11:33,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:11:33,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 09:11:33,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:11:33,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce13c74bda3ec93a41bfd HTTP/1.1" 200 None 2026-04-03 09:11:34,000 - lava_callback - ERROR - Error processing callback for node 69cce13c74bda3ec93a41bfd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37074 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:11:43,241 - lava_callback - INFO - Received callback for node 69cf835b000c489abc2c07fb INFO: 172.21.0.1:51538 - "POST /node/69cf835b000c489abc2c07fb HTTP/1.0" 202 Accepted 2026-04-03 09:11:43,247 - lava_callback - INFO - Starting async job submit for node 69cf835b000c489abc2c07fb 2026-04-03 09:11:43,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:11:43,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf835b000c489abc2c07fb HTTP/1.1" 200 None 2026-04-03 09:11:43,313 - lava_callback - DEBUG - Processing node 69cf835b000c489abc2c07fb: ltp-ipc 2026-04-03 09:11:43,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:11:43,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:11:43,819 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69cf835b000c489abc2c07fb/log.txt.gz 2026-04-03 09:11:44,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:11:44,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:11:44,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69cf835b000c489abc2c07fb/lava-logs.yaml 2026-04-03 09:11:44,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:11:44,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:11:44,437 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69cf835b000c489abc2c07fb/lava_callback.json.gz 2026-04-03 09:11:44,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:11:44,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf835b000c489abc2c07fb HTTP/1.1" 200 None 2026-04-03 09:11:44,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:11:44,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:11:44,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:11:44,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf835b000c489abc2c07fb HTTP/1.1" 200 None 2026-04-03 09:11:44,520 - lava_callback - INFO - Completed processing callback for node 69cf835b000c489abc2c07fb INFO: 172.21.0.1:51540 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:11:51,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:11:51,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40026 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:12:04,087 - lava_callback - INFO - Received callback for node 69cf83f3000c489abc2c0895 INFO: 172.21.0.1:48952 - "POST /node/69cf83f3000c489abc2c0895 HTTP/1.0" 202 Accepted 2026-04-03 09:12:04,093 - lava_callback - INFO - Starting async job submit for node 69cf83f3000c489abc2c0895 2026-04-03 09:12:04,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:12:04,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf83f3000c489abc2c0895 HTTP/1.1" 200 None 2026-04-03 09:12:04,132 - lava_callback - DEBUG - Processing node 69cf83f3000c489abc2c0895: baseline-arm-pengutronix 2026-04-03 09:12:04,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:12:04,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:12:04,481 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf83f3000c489abc2c0895/log.txt.gz 2026-04-03 09:12:04,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:12:04,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:12:04,800 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf83f3000c489abc2c0895/lava-logs.yaml 2026-04-03 09:12:04,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:12:04,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:12:04,966 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf83f3000c489abc2c0895/lava_callback.json.gz 2026-04-03 09:12:05,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:12:05,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf83f3000c489abc2c0895 HTTP/1.1" 200 None 2026-04-03 09:12:05,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:12:05,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:12:05,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:12:05,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf83f3000c489abc2c0895 HTTP/1.1" 200 None 2026-04-03 09:12:05,139 - lava_callback - INFO - Completed processing callback for node 69cf83f3000c489abc2c0895 INFO: 172.21.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47690 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:12:21,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:12:21,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51520 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:12:53,849 - lava_callback - INFO - Received callback for node 69cce19274bda3ec93a41c20 INFO: 172.21.0.1:51524 - "POST /node/69cce19274bda3ec93a41c20 HTTP/1.0" 202 Accepted 2026-04-03 09:12:53,855 - lava_callback - INFO - Starting async job submit for node 69cce19274bda3ec93a41c20 2026-04-03 09:12:53,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:12:53,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce19274bda3ec93a41c20 HTTP/1.1" 200 None 2026-04-03 09:12:53,890 - lava_callback - DEBUG - Processing node 69cce19274bda3ec93a41c20: baseline-arm64-clabbe 2026-04-03 09:12:53,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce19274bda3ec93a41c20 2026-04-03 09:12:53,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:12:53,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce19274bda3ec93a41c20 HTTP/1.1" 200 None 2026-04-03 09:12:53,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:12:53,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 09:12:53,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:12:53,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce19274bda3ec93a41c20 HTTP/1.1" 200 None 2026-04-03 09:12:53,958 - lava_callback - ERROR - Error processing callback for node 69cce19274bda3ec93a41c20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:13:17,881 - lava_callback - INFO - Received callback for node 69cf7ee1000c489abc2bff9f INFO: 172.21.0.1:49186 - "POST /node/69cf7ee1000c489abc2bff9f HTTP/1.0" 202 Accepted 2026-04-03 09:13:17,887 - lava_callback - INFO - Starting async job submit for node 69cf7ee1000c489abc2bff9f 2026-04-03 09:13:17,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:13:17,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ee1000c489abc2bff9f HTTP/1.1" 200 None 2026-04-03 09:13:17,933 - lava_callback - DEBUG - Processing node 69cf7ee1000c489abc2bff9f: baseline-nfs-arm64 2026-04-03 09:13:18,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:13:18,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:13:18,412 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee1000c489abc2bff9f/log.txt.gz 2026-04-03 09:13:18,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:13:18,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:13:18,831 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee1000c489abc2bff9f/lava-logs.yaml 2026-04-03 09:13:18,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:13:19,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:13:19,178 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ee1000c489abc2bff9f/lava_callback.json.gz 2026-04-03 09:13:19,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:13:19,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ee1000c489abc2bff9f HTTP/1.1" 200 None 2026-04-03 09:13:19,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:13:19,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:13:19,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:13:19,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ee1000c489abc2bff9f HTTP/1.1" 200 None 2026-04-03 09:13:19,277 - lava_callback - INFO - Completed processing callback for node 69cf7ee1000c489abc2bff9f 2026-04-03 09:13:21,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:13:21,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58138 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:14:23,126 - lava_callback - INFO - Received callback for node 69cf84df000c489abc2c0978 INFO: 172.21.0.1:42978 - "POST /node/69cf84df000c489abc2c0978 HTTP/1.0" 202 Accepted 2026-04-03 09:14:23,135 - lava_callback - INFO - Starting async job submit for node 69cf84df000c489abc2c0978 2026-04-03 09:14:23,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:14:23,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf84df000c489abc2c0978 HTTP/1.1" 200 None 2026-04-03 09:14:23,156 - lava_callback - DEBUG - Processing node 69cf84df000c489abc2c0978: baseline-nfs-arm64 2026-04-03 09:14:23,157 - lava_callback - WARNING - Failed to upload log 2026-04-03 09:14:23,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:14:23,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:14:23,319 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf84df000c489abc2c0978/lava-logs.yaml 2026-04-03 09:14:23,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:14:23,610 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:14:23,612 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf84df000c489abc2c0978/lava_callback.json.gz 2026-04-03 09:14:23,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:14:23,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf84df000c489abc2c0978 HTTP/1.1" 200 None 2026-04-03 09:14:23,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:14:23,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:14:23,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:14:23,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf84df000c489abc2c0978 HTTP/1.1" 200 None 2026-04-03 09:14:23,668 - lava_callback - INFO - Completed processing callback for node 69cf84df000c489abc2c0978 INFO: 172.21.0.1:42992 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:14:34,057 - lava_callback - INFO - Received callback for node 69cce1e774bda3ec93a41c39 INFO: 172.21.0.1:47362 - "POST /node/69cce1e774bda3ec93a41c39 HTTP/1.0" 202 Accepted 2026-04-03 09:14:34,063 - lava_callback - INFO - Starting async job submit for node 69cce1e774bda3ec93a41c39 2026-04-03 09:14:34,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:14:34,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce1e774bda3ec93a41c39 HTTP/1.1" 200 None 2026-04-03 09:14:34,079 - lava_callback - DEBUG - Processing node 69cce1e774bda3ec93a41c39: baseline-arm64-clabbe 2026-04-03 09:14:34,080 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce1e774bda3ec93a41c39 2026-04-03 09:14:34,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:14:34,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce1e774bda3ec93a41c39 HTTP/1.1" 200 None 2026-04-03 09:14:34,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:14:34,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 09:14:34,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:14:34,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce1e774bda3ec93a41c39 HTTP/1.1" 200 None 2026-04-03 09:14:34,135 - lava_callback - ERROR - Error processing callback for node 69cce1e774bda3ec93a41c39: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38058 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:14:51,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:14:51,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55228 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:15:35,558 - lava_callback - INFO - Received callback for node 69cf7ec6000c489abc2bff1a INFO: 172.21.0.1:34452 - "POST /node/69cf7ec6000c489abc2bff1a HTTP/1.0" 202 Accepted 2026-04-03 09:15:35,565 - lava_callback - INFO - Starting async job submit for node 69cf7ec6000c489abc2bff1a 2026-04-03 09:15:35,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:15:35,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec6000c489abc2bff1a HTTP/1.1" 200 None 2026-04-03 09:15:35,609 - lava_callback - DEBUG - Processing node 69cf7ec6000c489abc2bff1a: baseline-arm64 2026-04-03 09:15:35,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:15:35,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:15:35,983 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec6000c489abc2bff1a/log.txt.gz 2026-04-03 09:15:36,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:15:36,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:15:36,340 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec6000c489abc2bff1a/lava-logs.yaml 2026-04-03 09:15:36,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:15:36,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:15:36,492 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf7ec6000c489abc2bff1a/lava_callback.json.gz 2026-04-03 09:15:36,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:34464 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:15:36,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ec6000c489abc2bff1a HTTP/1.1" 200 None 2026-04-03 09:15:36,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:15:36,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:15:36,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:15:36,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ec6000c489abc2bff1a HTTP/1.1" 200 None 2026-04-03 09:15:36,571 - lava_callback - INFO - Completed processing callback for node 69cf7ec6000c489abc2bff1a INFO: 172.21.0.1:45080 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:15:51,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:15:51,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58008 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:15:54,078 - lava_callback - INFO - Received callback for node 69cce23d74bda3ec93a41c5c INFO: 172.21.0.1:58014 - "POST /node/69cce23d74bda3ec93a41c5c HTTP/1.0" 202 Accepted 2026-04-03 09:15:54,084 - lava_callback - INFO - Starting async job submit for node 69cce23d74bda3ec93a41c5c 2026-04-03 09:15:54,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:15:54,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce23d74bda3ec93a41c5c HTTP/1.1" 200 None 2026-04-03 09:15:54,105 - lava_callback - DEBUG - Processing node 69cce23d74bda3ec93a41c5c: baseline-arm64-clabbe 2026-04-03 09:15:54,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce23d74bda3ec93a41c5c 2026-04-03 09:15:54,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:15:54,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce23d74bda3ec93a41c5c HTTP/1.1" 200 None 2026-04-03 09:15:54,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:15:54,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 09:15:54,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:15:54,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce23d74bda3ec93a41c5c HTTP/1.1" 200 None 2026-04-03 09:15:54,164 - lava_callback - ERROR - Error processing callback for node 69cce23d74bda3ec93a41c5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37098 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:16:22,884 - lava_callback - INFO - Received callback for node 69cf7ed8000c489abc2bff82 INFO: 172.21.0.1:33070 - "POST /node/69cf7ed8000c489abc2bff82 HTTP/1.0" 202 Accepted 2026-04-03 09:16:22,890 - lava_callback - INFO - Starting async job submit for node 69cf7ed8000c489abc2bff82 2026-04-03 09:16:22,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:16:22,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed8000c489abc2bff82 HTTP/1.1" 200 None 2026-04-03 09:16:22,935 - lava_callback - DEBUG - Processing node 69cf7ed8000c489abc2bff82: baseline-nfs-arm64 2026-04-03 09:16:23,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:16:23,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:16:23,388 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ed8000c489abc2bff82/log.txt.gz 2026-04-03 09:16:23,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:16:23,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:16:23,826 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ed8000c489abc2bff82/lava-logs.yaml 2026-04-03 09:16:23,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:16:23,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:16:23,967 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf7ed8000c489abc2bff82/lava_callback.json.gz 2026-04-03 09:16:24,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:16:24,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7ed8000c489abc2bff82 HTTP/1.1" 200 None 2026-04-03 09:16:24,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:16:24,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:16:24,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:16:24,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7ed8000c489abc2bff82 HTTP/1.1" 200 None 2026-04-03 09:16:24,102 - lava_callback - INFO - Completed processing callback for node 69cf7ed8000c489abc2bff82 INFO: 172.21.0.1:33080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59170 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:16:51,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:16:51,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54756 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:17:01,422 - lava_callback - INFO - Received callback for node 69cce27e74bda3ec93a41c78 INFO: 172.21.0.1:46674 - "POST /node/69cce27e74bda3ec93a41c78 HTTP/1.0" 202 Accepted 2026-04-03 09:17:01,428 - lava_callback - INFO - Starting async job submit for node 69cce27e74bda3ec93a41c78 2026-04-03 09:17:01,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:01,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce27e74bda3ec93a41c78 HTTP/1.1" 200 None 2026-04-03 09:17:01,445 - lava_callback - DEBUG - Processing node 69cce27e74bda3ec93a41c78: baseline-arm-baylibre 2026-04-03 09:17:01,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce27e74bda3ec93a41c78 2026-04-03 09:17:01,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:01,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce27e74bda3ec93a41c78 HTTP/1.1" 200 None 2026-04-03 09:17:01,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:01,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-04-03 09:17:01,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:01,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce27e74bda3ec93a41c78 HTTP/1.1" 200 None 2026-04-03 09:17:01,517 - lava_callback - ERROR - Error processing callback for node 69cce27e74bda3ec93a41c78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44574 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:17:34,061 - lava_callback - INFO - Received callback for node 69cce2a874bda3ec93a41c91 INFO: 172.21.0.1:59930 - "POST /node/69cce2a874bda3ec93a41c91 HTTP/1.0" 202 Accepted 2026-04-03 09:17:34,069 - lava_callback - INFO - Starting async job submit for node 69cce2a874bda3ec93a41c91 2026-04-03 09:17:34,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:34,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce2a874bda3ec93a41c91 HTTP/1.1" 200 None 2026-04-03 09:17:34,085 - lava_callback - DEBUG - Processing node 69cce2a874bda3ec93a41c91: baseline-arm-clabbe 2026-04-03 09:17:34,085 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce2a874bda3ec93a41c91 2026-04-03 09:17:34,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:34,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce2a874bda3ec93a41c91 HTTP/1.1" 200 None 2026-04-03 09:17:34,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:34,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-04-03 09:17:34,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:34,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce2a874bda3ec93a41c91 HTTP/1.1" 200 None 2026-04-03 09:17:34,168 - lava_callback - ERROR - Error processing callback for node 69cce2a874bda3ec93a41c91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32894 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:17:48,947 - lava_callback - INFO - Received callback for node 69cf8583000c489abc2c0a32 INFO: 172.21.0.1:32904 - "POST /node/69cf8583000c489abc2c0a32 HTTP/1.0" 202 Accepted 2026-04-03 09:17:48,953 - lava_callback - INFO - Starting async job submit for node 69cf8583000c489abc2c0a32 2026-04-03 09:17:48,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:48,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8583000c489abc2c0a32 HTTP/1.1" 200 None 2026-04-03 09:17:48,997 - lava_callback - DEBUG - Processing node 69cf8583000c489abc2c0a32: baseline-arm64 2026-04-03 09:17:49,180 - lava_callback - WARNING - Failed to upload log 2026-04-03 09:17:49,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:17:49,438 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:17:49,440 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8583000c489abc2c0a32/lava-logs.yaml 2026-04-03 09:17:49,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:17:49,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:17:49,607 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8583000c489abc2c0a32/lava_callback.json.gz 2026-04-03 09:17:49,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:49,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8583000c489abc2c0a32 HTTP/1.1" 200 None 2026-04-03 09:17:49,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:49,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:17:49,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:49,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8583000c489abc2c0a32 HTTP/1.1" 200 None 2026-04-03 09:17:49,781 - lava_callback - INFO - Completed processing callback for node 69cf8583000c489abc2c0a32 2026-04-03 09:17:51,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:17:51,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36566 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:18:03,143 - lava_callback - INFO - Received callback for node 69cf7f89000c489abc2c02b8 INFO: 172.21.0.1:38334 - "POST /node/69cf7f89000c489abc2c02b8 HTTP/1.0" 202 Accepted 2026-04-03 09:18:03,190 - lava_callback - INFO - Starting async job submit for node 69cf7f89000c489abc2c02b8 2026-04-03 09:18:03,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:03,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f89000c489abc2c02b8 HTTP/1.1" 200 None 2026-04-03 09:18:03,235 - lava_callback - DEBUG - Processing node 69cf7f89000c489abc2c02b8: ltp-sched 2026-04-03 09:18:03,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:18:03,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:18:03,516 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69cf7f89000c489abc2c02b8/log.txt.gz 2026-04-03 09:18:03,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:18:03,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:18:03,746 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69cf7f89000c489abc2c02b8/lava-logs.yaml 2026-04-03 09:18:03,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:18:03,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:18:03,892 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69cf7f89000c489abc2c02b8/lava_callback.json.gz 2026-04-03 09:18:03,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:03,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7f89000c489abc2c02b8 HTTP/1.1" 200 None 2026-04-03 09:18:03,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:03,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:18:03,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:03,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7f89000c489abc2c02b8 HTTP/1.1" 200 None 2026-04-03 09:18:04,049 - lava_callback - INFO - Completed processing callback for node 69cf7f89000c489abc2c02b8 INFO: 172.21.0.1:38346 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:18:14,100 - lava_callback - INFO - Received callback for node 69cce2cc74bda3ec93a41ca2 INFO: 172.21.0.1:39630 - "POST /node/69cce2cc74bda3ec93a41ca2 HTTP/1.0" 202 Accepted 2026-04-03 09:18:14,106 - lava_callback - INFO - Starting async job submit for node 69cce2cc74bda3ec93a41ca2 2026-04-03 09:18:14,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:14,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce2cc74bda3ec93a41ca2 HTTP/1.1" 200 None 2026-04-03 09:18:14,125 - lava_callback - DEBUG - Processing node 69cce2cc74bda3ec93a41ca2: baseline-arm-clabbe 2026-04-03 09:18:14,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce2cc74bda3ec93a41ca2 2026-04-03 09:18:14,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:14,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce2cc74bda3ec93a41ca2 HTTP/1.1" 200 None 2026-04-03 09:18:14,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:14,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-04-03 09:18:14,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:14,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce2cc74bda3ec93a41ca2 HTTP/1.1" 200 None 2026-04-03 09:18:14,204 - lava_callback - ERROR - Error processing callback for node 69cce2cc74bda3ec93a41ca2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39642 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:18:22,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:22,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43278 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:18:54,127 - lava_callback - INFO - Received callback for node 69cce2f074bda3ec93a41cb0 INFO: 172.21.0.1:43282 - "POST /node/69cce2f074bda3ec93a41cb0 HTTP/1.0" 202 Accepted 2026-04-03 09:18:54,133 - lava_callback - INFO - Starting async job submit for node 69cce2f074bda3ec93a41cb0 2026-04-03 09:18:54,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:54,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce2f074bda3ec93a41cb0 HTTP/1.1" 200 None 2026-04-03 09:18:54,152 - lava_callback - DEBUG - Processing node 69cce2f074bda3ec93a41cb0: baseline-arm-clabbe 2026-04-03 09:18:54,153 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce2f074bda3ec93a41cb0 2026-04-03 09:18:54,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:54,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce2f074bda3ec93a41cb0 HTTP/1.1" 200 None 2026-04-03 09:18:54,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:54,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-04-03 09:18:54,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:18:54,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce2f074bda3ec93a41cb0 HTTP/1.1" 200 None 2026-04-03 09:18:54,225 - lava_callback - ERROR - Error processing callback for node 69cce2f074bda3ec93a41cb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40960 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:19:19,004 - lava_callback - INFO - Received callback for node 69cf85e9000c489abc2c0a86 INFO: 172.21.0.1:40968 - "POST /node/69cf85e9000c489abc2c0a86 HTTP/1.0" 202 Accepted 2026-04-03 09:19:19,011 - lava_callback - INFO - Starting async job submit for node 69cf85e9000c489abc2c0a86 2026-04-03 09:19:19,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:19:19,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf85e9000c489abc2c0a86 HTTP/1.1" 200 None 2026-04-03 09:19:19,057 - lava_callback - DEBUG - Processing node 69cf85e9000c489abc2c0a86: baseline-arm64 2026-04-03 09:19:19,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:19:19,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:19:19,327 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf85e9000c489abc2c0a86/log.txt.gz 2026-04-03 09:19:19,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:19:19,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:19:19,581 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf85e9000c489abc2c0a86/lava-logs.yaml 2026-04-03 09:19:19,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:19:19,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:19:19,715 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf85e9000c489abc2c0a86/lava_callback.json.gz 2026-04-03 09:19:19,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:19:19,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf85e9000c489abc2c0a86 HTTP/1.1" 200 None 2026-04-03 09:19:19,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:19:19,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:19:19,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:19:19,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf85e9000c489abc2c0a86 HTTP/1.1" 200 None 2026-04-03 09:19:19,862 - lava_callback - INFO - Completed processing callback for node 69cf85e9000c489abc2c0a86 2026-04-03 09:19:22,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:19:22,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32940 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:20:14,192 - lava_callback - INFO - Received callback for node 69cce33974bda3ec93a41ccd INFO: 172.21.0.1:51090 - "POST /node/69cce33974bda3ec93a41ccd HTTP/1.0" 202 Accepted 2026-04-03 09:20:14,198 - lava_callback - INFO - Starting async job submit for node 69cce33974bda3ec93a41ccd 2026-04-03 09:20:14,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:20:14,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce33974bda3ec93a41ccd HTTP/1.1" 200 None 2026-04-03 09:20:14,213 - lava_callback - DEBUG - Processing node 69cce33974bda3ec93a41ccd: baseline-arm-clabbe 2026-04-03 09:20:14,214 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce33974bda3ec93a41ccd 2026-04-03 09:20:14,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:20:14,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce33974bda3ec93a41ccd HTTP/1.1" 200 None 2026-04-03 09:20:14,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:20:14,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-04-03 09:20:14,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:20:14,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce33974bda3ec93a41ccd HTTP/1.1" 200 None 2026-04-03 09:20:14,278 - lava_callback - ERROR - Error processing callback for node 69cce33974bda3ec93a41ccd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60532 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:21:14,262 - lava_callback - INFO - Received callback for node 69cce38274bda3ec93a41ce9 INFO: 172.21.0.1:51596 - "POST /node/69cce38274bda3ec93a41ce9 HTTP/1.0" 202 Accepted 2026-04-03 09:21:14,267 - lava_callback - INFO - Starting async job submit for node 69cce38274bda3ec93a41ce9 2026-04-03 09:21:14,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:21:14,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce38274bda3ec93a41ce9 HTTP/1.1" 200 None 2026-04-03 09:21:14,283 - lava_callback - DEBUG - Processing node 69cce38274bda3ec93a41ce9: baseline-arm-clabbe 2026-04-03 09:21:14,283 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce38274bda3ec93a41ce9 2026-04-03 09:21:14,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:21:14,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce38274bda3ec93a41ce9 HTTP/1.1" 200 None 2026-04-03 09:21:14,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:21:14,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebcbf63ce05331e2a47f HTTP/1.1" 200 None 2026-04-03 09:21:14,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:21:14,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce38274bda3ec93a41ce9 HTTP/1.1" 200 None 2026-04-03 09:21:14,347 - lava_callback - ERROR - Error processing callback for node 69cce38274bda3ec93a41ce9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37172 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:23:04,013 - lava_callback - INFO - Received callback for node 69cf8545000c489abc2c09e8 INFO: 172.21.0.1:60166 - "POST /node/69cf8545000c489abc2c09e8 HTTP/1.0" 202 Accepted 2026-04-03 09:23:04,021 - lava_callback - INFO - Starting async job submit for node 69cf8545000c489abc2c09e8 2026-04-03 09:23:04,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:23:04,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8545000c489abc2c09e8 HTTP/1.1" 200 None 2026-04-03 09:23:04,071 - lava_callback - DEBUG - Processing node 69cf8545000c489abc2c09e8: baseline-arm64 2026-04-03 09:23:04,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:23:04,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:23:04,747 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8545000c489abc2c09e8/log.txt.gz 2026-04-03 09:23:05,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:23:05,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:23:05,304 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8545000c489abc2c09e8/lava-logs.yaml 2026-04-03 09:23:05,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:23:05,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:23:05,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8545000c489abc2c09e8/lava_callback.json.gz DEBUG: Setup failed for 69cf8545000c489abc2c09e8. Transitting job node result: incomplete -> fail 2026-04-03 09:23:05,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:23:05,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8545000c489abc2c09e8 HTTP/1.1" 200 None 2026-04-03 09:23:05,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:23:05,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:23:05,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:23:05,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8545000c489abc2c09e8 HTTP/1.1" 200 None 2026-04-03 09:23:05,722 - lava_callback - INFO - Completed processing callback for node 69cf8545000c489abc2c09e8 INFO: 172.21.0.1:60182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 09:23:15,733 - lava_callback - INFO - Received callback for node 69cf86ac000c489abc2c0b01 INFO: 172.21.0.1:33344 - "POST /node/69cf86ac000c489abc2c0b01 HTTP/1.0" 202 Accepted 2026-04-03 09:23:15,739 - lava_callback - INFO - Starting async job submit for node 69cf86ac000c489abc2c0b01 2026-04-03 09:23:15,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:23:15,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf86ac000c489abc2c0b01 HTTP/1.1" 200 None 2026-04-03 09:23:15,780 - lava_callback - DEBUG - Processing node 69cf86ac000c489abc2c0b01: baseline-x86 2026-04-03 09:23:16,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:23:16,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:23:16,371 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf86ac000c489abc2c0b01/log.txt.gz INFO: 172.21.0.1:33346 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:23:16,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:23:16,711 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:23:16,713 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf86ac000c489abc2c0b01/lava-logs.yaml 2026-04-03 09:23:16,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:23:16,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:23:16,874 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf86ac000c489abc2c0b01/lava_callback.json.gz 2026-04-03 09:23:16,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:23:16,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf86ac000c489abc2c0b01 HTTP/1.1" 200 None 2026-04-03 09:23:16,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:23:16,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77d1000c489abc2bf955 HTTP/1.1" 200 None 2026-04-03 09:23:16,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:23:16,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf86ac000c489abc2c0b01 HTTP/1.1" 200 None 2026-04-03 09:23:16,951 - lava_callback - INFO - Completed processing callback for node 69cf86ac000c489abc2c0b01 2026-04-03 09:23:22,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:23:22,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42022 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:24:25,943 - lava_callback - INFO - Received callback for node 69cf86f8000c489abc2c0b29 INFO: 172.21.0.1:42010 - "POST /node/69cf86f8000c489abc2c0b29 HTTP/1.0" 202 Accepted 2026-04-03 09:24:25,950 - lava_callback - INFO - Starting async job submit for node 69cf86f8000c489abc2c0b29 2026-04-03 09:24:25,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:24:26,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf86f8000c489abc2c0b29 HTTP/1.1" 200 None 2026-04-03 09:24:26,012 - lava_callback - DEBUG - Processing node 69cf86f8000c489abc2c0b29: baseline-arm64 INFO: 172.21.0.1:42020 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:24:26,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:24:26,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:24:26,808 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf86f8000c489abc2c0b29/log.txt.gz 2026-04-03 09:24:27,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:24:27,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:24:27,366 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf86f8000c489abc2c0b29/lava-logs.yaml 2026-04-03 09:24:27,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:24:27,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:24:27,648 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf86f8000c489abc2c0b29/lava_callback.json.gz 2026-04-03 09:24:27,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:24:27,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf86f8000c489abc2c0b29 HTTP/1.1" 200 None 2026-04-03 09:24:27,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:24:27,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:24:27,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:24:27,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf86f8000c489abc2c0b29 HTTP/1.1" 200 None 2026-04-03 09:24:27,761 - lava_callback - INFO - Completed processing callback for node 69cf86f8000c489abc2c0b29 INFO: 172.21.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53806 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:24:52,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:24:52,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59358 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:25:31,035 - lava_callback - INFO - Received callback for node 69cf877e000c489abc2c0bbe INFO: 172.21.0.1:48848 - "POST /node/69cf877e000c489abc2c0bbe HTTP/1.0" 202 Accepted 2026-04-03 09:25:31,041 - lava_callback - INFO - Starting async job submit for node 69cf877e000c489abc2c0bbe 2026-04-03 09:25:31,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:25:31,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf877e000c489abc2c0bbe HTTP/1.1" 200 None 2026-04-03 09:25:31,069 - lava_callback - DEBUG - Processing node 69cf877e000c489abc2c0bbe: baseline-arm 2026-04-03 09:25:31,074 - lava_callback - WARNING - Failed to upload log 2026-04-03 09:25:31,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:25:31,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:25:31,308 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf877e000c489abc2c0bbe/lava-logs.yaml 2026-04-03 09:25:31,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:25:31,466 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:25:31,468 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf877e000c489abc2c0bbe/lava_callback.json.gz 2026-04-03 09:25:31,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:25:31,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf877e000c489abc2c0bbe HTTP/1.1" 200 None 2026-04-03 09:25:31,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:25:31,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:25:31,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:25:31,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf877e000c489abc2c0bbe HTTP/1.1" 200 None 2026-04-03 09:25:31,569 - lava_callback - INFO - Completed processing callback for node 69cf877e000c489abc2c0bbe INFO: 172.21.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41672 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:25:52,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:25:52,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42262 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:29:59,574 - lava_callback - INFO - Received callback for node 69cf862f000c489abc2c0ab9 INFO: 172.21.0.1:42274 - "POST /node/69cf862f000c489abc2c0ab9 HTTP/1.0" 202 Accepted 2026-04-03 09:29:59,581 - lava_callback - INFO - Starting async job submit for node 69cf862f000c489abc2c0ab9 2026-04-03 09:29:59,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:29:59,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf862f000c489abc2c0ab9 HTTP/1.1" 200 None 2026-04-03 09:29:59,620 - lava_callback - DEBUG - Processing node 69cf862f000c489abc2c0ab9: baseline-arm64 2026-04-03 09:30:00,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:30:00,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:30:00,381 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf862f000c489abc2c0ab9/log.txt.gz 2026-04-03 09:30:00,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:30:00,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:30:00,901 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf862f000c489abc2c0ab9/lava-logs.yaml 2026-04-03 09:30:00,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:30:01,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:30:01,112 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf862f000c489abc2c0ab9/lava_callback.json.gz 2026-04-03 09:30:01,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:30:01,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf862f000c489abc2c0ab9 HTTP/1.1" 200 None 2026-04-03 09:30:01,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:30:01,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:30:01,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:30:01,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf862f000c489abc2c0ab9 HTTP/1.1" 200 None 2026-04-03 09:30:01,267 - lava_callback - INFO - Completed processing callback for node 69cf862f000c489abc2c0ab9 INFO: 172.21.0.1:59038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39468 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:30:22,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:30:22,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59652 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:32:46,276 - lava_callback - INFO - Received callback for node 69cf86b9000c489abc2c0b04 INFO: 172.21.0.1:41252 - "POST /node/69cf86b9000c489abc2c0b04 HTTP/1.0" 202 Accepted 2026-04-03 09:32:46,283 - lava_callback - INFO - Starting async job submit for node 69cf86b9000c489abc2c0b04 2026-04-03 09:32:46,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:32:46,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf86b9000c489abc2c0b04 HTTP/1.1" 200 None 2026-04-03 09:32:46,330 - lava_callback - DEBUG - Processing node 69cf86b9000c489abc2c0b04: baseline-nfs-arm64 INFO: 172.21.0.1:41266 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:32:46,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:32:47,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:32:47,028 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf86b9000c489abc2c0b04/log.txt.gz 2026-04-03 09:32:47,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:32:47,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:32:47,495 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf86b9000c489abc2c0b04/lava-logs.yaml 2026-04-03 09:32:47,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:32:47,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:32:47,682 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf86b9000c489abc2c0b04/lava_callback.json.gz 2026-04-03 09:32:47,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:32:47,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf86b9000c489abc2c0b04 HTTP/1.1" 200 None 2026-04-03 09:32:47,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:32:47,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:32:47,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:32:47,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf86b9000c489abc2c0b04 HTTP/1.1" 200 None 2026-04-03 09:32:47,851 - lava_callback - INFO - Completed processing callback for node 69cf86b9000c489abc2c0b04 2026-04-03 09:32:52,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:32:52,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48060 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:33:42,355 - lava_callback - INFO - Received callback for node 69cce66274bda3ec93a41e23 INFO: 172.21.0.1:37128 - "POST /node/69cce66274bda3ec93a41e23 HTTP/1.0" 202 Accepted 2026-04-03 09:33:42,364 - lava_callback - INFO - Starting async job submit for node 69cce66274bda3ec93a41e23 2026-04-03 09:33:42,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:33:42,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce66274bda3ec93a41e23 HTTP/1.1" 200 None 2026-04-03 09:33:42,387 - lava_callback - DEBUG - Processing node 69cce66274bda3ec93a41e23: baseline-arm64-baylibre 2026-04-03 09:33:42,388 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce66274bda3ec93a41e23 2026-04-03 09:33:42,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:33:42,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce66274bda3ec93a41e23 HTTP/1.1" 200 None 2026-04-03 09:33:42,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:33:42,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:33:42,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:33:42,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce66274bda3ec93a41e23 HTTP/1.1" 200 None 2026-04-03 09:33:42,439 - lava_callback - ERROR - Error processing callback for node 69cce66274bda3ec93a41e23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58242 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:37:42,480 - lava_callback - INFO - Received callback for node 69cce75b74bda3ec93a41ea1 INFO: 172.21.0.1:47402 - "POST /node/69cce75b74bda3ec93a41ea1 HTTP/1.0" 202 Accepted 2026-04-03 09:37:42,486 - lava_callback - INFO - Starting async job submit for node 69cce75b74bda3ec93a41ea1 2026-04-03 09:37:42,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:37:42,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce75b74bda3ec93a41ea1 HTTP/1.1" 200 None 2026-04-03 09:37:42,501 - lava_callback - DEBUG - Processing node 69cce75b74bda3ec93a41ea1: baseline-arm64-baylibre 2026-04-03 09:37:42,502 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce75b74bda3ec93a41ea1 2026-04-03 09:37:42,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:37:42,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce75b74bda3ec93a41ea1 HTTP/1.1" 200 None 2026-04-03 09:37:42,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:37:42,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:37:42,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:37:42,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce75b74bda3ec93a41ea1 HTTP/1.1" 200 None 2026-04-03 09:37:42,562 - lava_callback - ERROR - Error processing callback for node 69cce75b74bda3ec93a41ea1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40540 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:39:02,613 - lava_callback - INFO - Received callback for node 69cce7b074bda3ec93a41ec8 INFO: 172.21.0.1:34910 - "POST /node/69cce7b074bda3ec93a41ec8 HTTP/1.0" 202 Accepted 2026-04-03 09:39:02,618 - lava_callback - INFO - Starting async job submit for node 69cce7b074bda3ec93a41ec8 2026-04-03 09:39:02,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:39:02,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce7b074bda3ec93a41ec8 HTTP/1.1" 200 None 2026-04-03 09:39:02,633 - lava_callback - DEBUG - Processing node 69cce7b074bda3ec93a41ec8: baseline-arm64-baylibre 2026-04-03 09:39:02,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce7b074bda3ec93a41ec8 2026-04-03 09:39:02,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:39:02,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce7b074bda3ec93a41ec8 HTTP/1.1" 200 None 2026-04-03 09:39:02,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:39:02,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:39:02,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:39:02,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce7b074bda3ec93a41ec8 HTTP/1.1" 200 None 2026-04-03 09:39:02,700 - lava_callback - ERROR - Error processing callback for node 69cce7b074bda3ec93a41ec8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39390 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:40:22,607 - lava_callback - INFO - Received callback for node 69cce80374bda3ec93a41ee9 INFO: 172.21.0.1:51390 - "POST /node/69cce80374bda3ec93a41ee9 HTTP/1.0" 202 Accepted 2026-04-03 09:40:22,614 - lava_callback - INFO - Starting async job submit for node 69cce80374bda3ec93a41ee9 2026-04-03 09:40:22,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:40:22,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce80374bda3ec93a41ee9 HTTP/1.1" 200 None 2026-04-03 09:40:22,630 - lava_callback - DEBUG - Processing node 69cce80374bda3ec93a41ee9: baseline-arm64-baylibre 2026-04-03 09:40:22,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce80374bda3ec93a41ee9 2026-04-03 09:40:22,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:40:22,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce80374bda3ec93a41ee9 HTTP/1.1" 200 None 2026-04-03 09:40:22,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:40:22,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:40:22,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:40:22,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce80374bda3ec93a41ee9 HTTP/1.1" 200 None 2026-04-03 09:40:22,687 - lava_callback - ERROR - Error processing callback for node 69cce80374bda3ec93a41ee9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47340 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:42:02,703 - lava_callback - INFO - Received callback for node 69cce85974bda3ec93a41f10 INFO: 172.21.0.1:55788 - "POST /node/69cce85974bda3ec93a41f10 HTTP/1.0" 202 Accepted 2026-04-03 09:42:02,708 - lava_callback - INFO - Starting async job submit for node 69cce85974bda3ec93a41f10 2026-04-03 09:42:02,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:42:02,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce85974bda3ec93a41f10 HTTP/1.1" 200 None 2026-04-03 09:42:02,727 - lava_callback - DEBUG - Processing node 69cce85974bda3ec93a41f10: baseline-arm64-baylibre 2026-04-03 09:42:02,727 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce85974bda3ec93a41f10 2026-04-03 09:42:02,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:42:02,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce85974bda3ec93a41f10 HTTP/1.1" 200 None 2026-04-03 09:42:02,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:42:02,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:42:02,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:42:02,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce85974bda3ec93a41f10 HTTP/1.1" 200 None 2026-04-03 09:42:02,808 - lava_callback - ERROR - Error processing callback for node 69cce85974bda3ec93a41f10: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53182 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:43:22,761 - lava_callback - INFO - Received callback for node 69cce8b274bda3ec93a41f3a INFO: 172.21.0.1:57816 - "POST /node/69cce8b274bda3ec93a41f3a HTTP/1.0" 202 Accepted 2026-04-03 09:43:22,767 - lava_callback - INFO - Starting async job submit for node 69cce8b274bda3ec93a41f3a 2026-04-03 09:43:22,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:43:22,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce8b274bda3ec93a41f3a HTTP/1.1" 200 None 2026-04-03 09:43:22,783 - lava_callback - DEBUG - Processing node 69cce8b274bda3ec93a41f3a: baseline-arm64-baylibre 2026-04-03 09:43:22,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce8b274bda3ec93a41f3a 2026-04-03 09:43:22,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:43:22,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce8b274bda3ec93a41f3a HTTP/1.1" 200 None 2026-04-03 09:43:22,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:43:22,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:43:22,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:43:22,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce8b274bda3ec93a41f3a HTTP/1.1" 200 None 2026-04-03 09:43:22,850 - lava_callback - ERROR - Error processing callback for node 69cce8b274bda3ec93a41f3a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48918 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:44:55,259 - lava_callback - INFO - Received callback for node 69cce91074bda3ec93a41f5e INFO: 172.21.0.1:48922 - "POST /node/69cce91074bda3ec93a41f5e HTTP/1.0" 202 Accepted 2026-04-03 09:44:55,265 - lava_callback - INFO - Starting async job submit for node 69cce91074bda3ec93a41f5e 2026-04-03 09:44:55,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:44:55,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce91074bda3ec93a41f5e HTTP/1.1" 200 None 2026-04-03 09:44:55,286 - lava_callback - DEBUG - Processing node 69cce91074bda3ec93a41f5e: baseline-arm64-clabbe 2026-04-03 09:44:55,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce91074bda3ec93a41f5e 2026-04-03 09:44:55,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:44:55,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce91074bda3ec93a41f5e HTTP/1.1" 200 None 2026-04-03 09:44:55,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:44:55,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:44:55,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:44:55,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce91074bda3ec93a41f5e HTTP/1.1" 200 None 2026-04-03 09:44:55,345 - lava_callback - ERROR - Error processing callback for node 69cce91074bda3ec93a41f5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48944 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 09:46:15,253 - lava_callback - INFO - Received callback for node 69cce96174bda3ec93a41f88 INFO: 172.21.0.1:48958 - "POST /node/69cce96174bda3ec93a41f88 HTTP/1.0" 202 Accepted 2026-04-03 09:46:15,259 - lava_callback - INFO - Starting async job submit for node 69cce96174bda3ec93a41f88 2026-04-03 09:46:15,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:46:15,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce96174bda3ec93a41f88 HTTP/1.1" 200 None 2026-04-03 09:46:15,275 - lava_callback - DEBUG - Processing node 69cce96174bda3ec93a41f88: baseline-arm64-clabbe 2026-04-03 09:46:15,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cce96174bda3ec93a41f88 2026-04-03 09:46:15,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:46:15,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cce96174bda3ec93a41f88 HTTP/1.1" 200 None 2026-04-03 09:46:15,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:46:15,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:46:15,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:46:15,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cce96174bda3ec93a41f88 HTTP/1.1" 200 None 2026-04-03 09:46:15,326 - lava_callback - ERROR - Error processing callback for node 69cce96174bda3ec93a41f88: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53240 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 09:50:15,418 - lava_callback - INFO - Received callback for node 69ccea5374bda3ec93a41fde INFO: 172.21.0.1:53244 - "POST /node/69ccea5374bda3ec93a41fde HTTP/1.0" 202 Accepted 2026-04-03 09:50:15,424 - lava_callback - INFO - Starting async job submit for node 69ccea5374bda3ec93a41fde 2026-04-03 09:50:15,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:50:15,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccea5374bda3ec93a41fde HTTP/1.1" 200 None 2026-04-03 09:50:15,439 - lava_callback - DEBUG - Processing node 69ccea5374bda3ec93a41fde: baseline-arm64-clabbe 2026-04-03 09:50:15,440 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccea5374bda3ec93a41fde 2026-04-03 09:50:15,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:50:15,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccea5374bda3ec93a41fde HTTP/1.1" 200 None 2026-04-03 09:50:15,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:50:15,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:50:15,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:50:15,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccea5374bda3ec93a41fde HTTP/1.1" 200 None 2026-04-03 09:50:15,509 - lava_callback - ERROR - Error processing callback for node 69ccea5374bda3ec93a41fde: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36690 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 09:54:15,630 - lava_callback - INFO - Received callback for node 69cceb3b74bda3ec93a4202a INFO: 172.21.0.1:36700 - "POST /node/69cceb3b74bda3ec93a4202a HTTP/1.0" 202 Accepted 2026-04-03 09:54:15,635 - lava_callback - INFO - Starting async job submit for node 69cceb3b74bda3ec93a4202a 2026-04-03 09:54:15,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:54:15,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cceb3b74bda3ec93a4202a HTTP/1.1" 200 None 2026-04-03 09:54:15,651 - lava_callback - DEBUG - Processing node 69cceb3b74bda3ec93a4202a: baseline-arm64-clabbe 2026-04-03 09:54:15,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cceb3b74bda3ec93a4202a 2026-04-03 09:54:15,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:54:15,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cceb3b74bda3ec93a4202a HTTP/1.1" 200 None 2026-04-03 09:54:15,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:54:15,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:54:15,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:54:15,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cceb3b74bda3ec93a4202a HTTP/1.1" 200 None 2026-04-03 09:54:15,722 - lava_callback - ERROR - Error processing callback for node 69cceb3b74bda3ec93a4202a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53734 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:54:46,396 - lava_callback - INFO - Received callback for node 69cf8e37000c489abc2c10ec INFO: 172.21.0.1:44598 - "POST /node/69cf8e37000c489abc2c10ec HTTP/1.0" 202 Accepted 2026-04-03 09:54:46,404 - lava_callback - INFO - Starting async job submit for node 69cf8e37000c489abc2c10ec 2026-04-03 09:54:46,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:54:46,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8e37000c489abc2c10ec HTTP/1.1" 200 None 2026-04-03 09:54:46,437 - lava_callback - DEBUG - Processing node 69cf8e37000c489abc2c10ec: baseline-arm-pengutronix INFO: 172.21.0.1:44608 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:54:46,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:54:46,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:54:46,755 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf8e37000c489abc2c10ec/log.txt.gz 2026-04-03 09:54:46,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:54:47,037 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:54:47,039 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf8e37000c489abc2c10ec/lava-logs.yaml 2026-04-03 09:54:47,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:54:47,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:54:47,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf8e37000c489abc2c10ec/lava_callback.json.gz 2026-04-03 09:54:47,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:54:47,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8e37000c489abc2c10ec HTTP/1.1" 200 None 2026-04-03 09:54:47,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:54:47,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 09:54:47,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:54:47,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8e37000c489abc2c10ec HTTP/1.1" 200 None 2026-04-03 09:54:47,273 - lava_callback - INFO - Completed processing callback for node 69cf8e37000c489abc2c10ec 2026-04-03 09:54:52,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:54:52,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58126 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:56:04,721 - lava_callback - INFO - Received callback for node 69cf8e9b000c489abc2c1154 INFO: 172.21.0.1:34984 - "POST /node/69cf8e9b000c489abc2c1154 HTTP/1.0" 202 Accepted 2026-04-03 09:56:04,727 - lava_callback - INFO - Starting async job submit for node 69cf8e9b000c489abc2c1154 2026-04-03 09:56:04,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:56:04,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8e9b000c489abc2c1154 HTTP/1.1" 200 None 2026-04-03 09:56:04,759 - lava_callback - DEBUG - Processing node 69cf8e9b000c489abc2c1154: baseline-nfs-arm64 2026-04-03 09:56:04,761 - lava_callback - WARNING - Failed to upload log 2026-04-03 09:56:04,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:56:04,903 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:56:04,904 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf8e9b000c489abc2c1154/lava-logs.yaml 2026-04-03 09:56:04,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 09:56:05,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 09:56:05,026 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf8e9b000c489abc2c1154/lava_callback.json.gz 2026-04-03 09:56:05,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:56:05,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8e9b000c489abc2c1154 HTTP/1.1" 200 None 2026-04-03 09:56:05,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:56:05,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 09:56:05,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:56:05,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8e9b000c489abc2c1154 HTTP/1.1" 200 None 2026-04-03 09:56:05,087 - lava_callback - INFO - Completed processing callback for node 69cf8e9b000c489abc2c1154 INFO: 172.21.0.1:34996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54840 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:56:22,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:56:22,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34852 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:56:55,779 - lava_callback - INFO - Received callback for node 69ccebd874bda3ec93a4206a INFO: 172.21.0.1:34862 - "POST /node/69ccebd874bda3ec93a4206a HTTP/1.0" 202 Accepted 2026-04-03 09:56:55,785 - lava_callback - INFO - Starting async job submit for node 69ccebd874bda3ec93a4206a 2026-04-03 09:56:55,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:56:55,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccebd874bda3ec93a4206a HTTP/1.1" 200 None 2026-04-03 09:56:55,811 - lava_callback - DEBUG - Processing node 69ccebd874bda3ec93a4206a: baseline-arm64-clabbe 2026-04-03 09:56:55,811 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccebd874bda3ec93a4206a 2026-04-03 09:56:55,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:56:55,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccebd874bda3ec93a4206a HTTP/1.1" 200 None 2026-04-03 09:56:55,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:56:55,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:56:55,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:56:55,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccebd874bda3ec93a4206a HTTP/1.1" 200 None 2026-04-03 09:56:55,878 - lava_callback - ERROR - Error processing callback for node 69ccebd874bda3ec93a4206a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36200 - "GET / HTTP/1.0" 200 OK 2026-04-03 09:59:35,927 - lava_callback - INFO - Received callback for node 69ccec8074bda3ec93a4209a INFO: 172.21.0.1:33898 - "POST /node/69ccec8074bda3ec93a4209a HTTP/1.0" 202 Accepted 2026-04-03 09:59:35,933 - lava_callback - INFO - Starting async job submit for node 69ccec8074bda3ec93a4209a 2026-04-03 09:59:35,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:59:35,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccec8074bda3ec93a4209a HTTP/1.1" 200 None 2026-04-03 09:59:35,949 - lava_callback - DEBUG - Processing node 69ccec8074bda3ec93a4209a: baseline-arm64-clabbe 2026-04-03 09:59:35,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccec8074bda3ec93a4209a 2026-04-03 09:59:35,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:59:35,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccec8074bda3ec93a4209a HTTP/1.1" 200 None 2026-04-03 09:59:35,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:59:35,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 09:59:35,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 09:59:36,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccec8074bda3ec93a4209a HTTP/1.1" 200 None 2026-04-03 09:59:36,001 - lava_callback - ERROR - Error processing callback for node 69ccec8074bda3ec93a4209a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47708 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:00:10,045 - lava_callback - INFO - Received callback for node 69cf8f69000c489abc2c1212 INFO: 172.21.0.1:56260 - "POST /node/69cf8f69000c489abc2c1212 HTTP/1.0" 202 Accepted 2026-04-03 10:00:10,052 - lava_callback - INFO - Starting async job submit for node 69cf8f69000c489abc2c1212 2026-04-03 10:00:10,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:00:10,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8f69000c489abc2c1212 HTTP/1.1" 200 None 2026-04-03 10:00:10,100 - lava_callback - DEBUG - Processing node 69cf8f69000c489abc2c1212: baseline-arm64 2026-04-03 10:00:10,236 - lava_callback - WARNING - Failed to upload log 2026-04-03 10:00:10,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:00:10,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:00:10,530 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8f69000c489abc2c1212/lava-logs.yaml 2026-04-03 10:00:10,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:00:10,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:00:10,685 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8f69000c489abc2c1212/lava_callback.json.gz 2026-04-03 10:00:10,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:00:10,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8f69000c489abc2c1212 HTTP/1.1" 200 None 2026-04-03 10:00:10,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:00:10,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 10:00:10,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:00:10,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8f69000c489abc2c1212 HTTP/1.1" 200 None 2026-04-03 10:00:10,817 - lava_callback - INFO - Completed processing callback for node 69cf8f69000c489abc2c1212 INFO: 172.21.0.1:56266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56268 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:00:22,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:00:22,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60736 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:01:37,658 - lava_callback - INFO - Received callback for node 69cf8fcf000c489abc2c125a INFO: 172.21.0.1:60742 - "POST /node/69cf8fcf000c489abc2c125a HTTP/1.0" 202 Accepted 2026-04-03 10:01:37,665 - lava_callback - INFO - Starting async job submit for node 69cf8fcf000c489abc2c125a 2026-04-03 10:01:37,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:01:37,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8fcf000c489abc2c125a HTTP/1.1" 200 None 2026-04-03 10:01:37,714 - lava_callback - DEBUG - Processing node 69cf8fcf000c489abc2c125a: baseline-arm64 2026-04-03 10:01:37,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:01:38,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:01:38,019 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8fcf000c489abc2c125a/log.txt.gz 2026-04-03 10:01:38,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:01:38,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:01:38,412 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8fcf000c489abc2c125a/lava-logs.yaml 2026-04-03 10:01:38,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:01:38,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:01:38,538 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf8fcf000c489abc2c125a/lava_callback.json.gz 2026-04-03 10:01:38,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:01:38,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8fcf000c489abc2c125a HTTP/1.1" 200 None 2026-04-03 10:01:38,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:01:38,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 10:01:38,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:01:38,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8fcf000c489abc2c125a HTTP/1.1" 200 None 2026-04-03 10:01:38,731 - lava_callback - INFO - Completed processing callback for node 69cf8fcf000c489abc2c125a INFO: 172.21.0.1:58758 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:01:52,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:01:52,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42326 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:03:22,021 - lava_callback - INFO - Received callback for node 69cf905c000c489abc2c12e0 INFO: 172.21.0.1:49540 - "POST /node/69cf905c000c489abc2c12e0 HTTP/1.0" 202 Accepted 2026-04-03 10:03:22,027 - lava_callback - INFO - Starting async job submit for node 69cf905c000c489abc2c12e0 2026-04-03 10:03:22,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:03:22,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf905c000c489abc2c12e0 HTTP/1.1" 200 None 2026-04-03 10:03:22,053 - lava_callback - DEBUG - Processing node 69cf905c000c489abc2c12e0: baseline-arm 2026-04-03 10:03:22,058 - lava_callback - WARNING - Failed to upload log 2026-04-03 10:03:22,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:03:22,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:03:22,230 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf905c000c489abc2c12e0/lava-logs.yaml 2026-04-03 10:03:22,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:03:22,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:03:22,385 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf905c000c489abc2c12e0/lava_callback.json.gz 2026-04-03 10:03:22,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:03:22,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf905c000c489abc2c12e0 HTTP/1.1" 200 None 2026-04-03 10:03:22,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:03:22,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 10:03:22,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:03:22,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf905c000c489abc2c12e0 HTTP/1.1" 200 None 2026-04-03 10:03:22,453 - lava_callback - INFO - Completed processing callback for node 69cf905c000c489abc2c12e0 INFO: 172.21.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54018 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:03:52,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:03:52,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41764 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:04:45,448 - lava_callback - INFO - Received callback for node 69cf8c88000c489abc2c0fb2 INFO: 172.21.0.1:43314 - "POST /node/69cf8c88000c489abc2c0fb2 HTTP/1.0" 202 Accepted 2026-04-03 10:04:45,455 - lava_callback - INFO - Starting async job submit for node 69cf8c88000c489abc2c0fb2 2026-04-03 10:04:45,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:04:45,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8c88000c489abc2c0fb2 HTTP/1.1" 200 None 2026-04-03 10:04:45,523 - lava_callback - DEBUG - Processing node 69cf8c88000c489abc2c0fb2: baseline-arm 2026-04-03 10:04:45,604 - lava_callback - INFO - Received callback for node 69cf8fca000c489abc2c1257 INFO: 172.21.0.1:43322 - "POST /node/69cf8fca000c489abc2c1257 HTTP/1.0" 202 Accepted 2026-04-03 10:04:45,615 - lava_callback - INFO - Starting async job submit for node 69cf8fca000c489abc2c1257 2026-04-03 10:04:45,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:04:45,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8fca000c489abc2c1257 HTTP/1.1" 200 None 2026-04-03 10:04:45,771 - lava_callback - DEBUG - Processing node 69cf8fca000c489abc2c1257: ltp-sched INFO: 172.21.0.1:43324 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:04:46,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:04:46,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:04:46,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:04:46,741 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf8c88000c489abc2c0fb2/log.txt.gz 2026-04-03 10:04:46,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:04:46,832 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69cf8fca000c489abc2c1257/log.txt.gz 2026-04-03 10:04:47,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:04:47,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:04:47,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:04:47,545 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69cf8fca000c489abc2c1257/lava-logs.yaml 2026-04-03 10:04:47,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:04:47,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:04:47,663 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf8c88000c489abc2c0fb2/lava-logs.yaml 2026-04-03 10:04:47,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:04:47,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:04:47,755 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69cf8fca000c489abc2c1257/lava_callback.json.gz 2026-04-03 10:04:47,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:04:47,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8fca000c489abc2c1257 HTTP/1.1" 200 None 2026-04-03 10:04:47,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:04:47,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 10:04:47,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:04:47,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:04:47,838 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf8c88000c489abc2c0fb2/lava_callback.json.gz 2026-04-03 10:04:47,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8fca000c489abc2c1257 HTTP/1.1" 200 None 2026-04-03 10:04:47,857 - lava_callback - INFO - Completed processing callback for node 69cf8fca000c489abc2c1257 DEBUG: Setup failed for 69cf8c88000c489abc2c0fb2. Transitting job node result: incomplete -> fail 2026-04-03 10:04:48,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:04:48,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8c88000c489abc2c0fb2 HTTP/1.1" 200 None 2026-04-03 10:04:48,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:04:48,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 10:04:48,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:04:48,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8c88000c489abc2c0fb2 HTTP/1.1" 200 None 2026-04-03 10:04:48,232 - lava_callback - INFO - Completed processing callback for node 69cf8c88000c489abc2c0fb2 2026-04-03 10:04:52,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:04:52,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35332 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:07:27,272 - lava_callback - INFO - Received callback for node 69cf8cb1000c489abc2c0fd0 INFO: 172.21.0.1:35340 - "POST /node/69cf8cb1000c489abc2c0fd0 HTTP/1.0" 202 Accepted 2026-04-03 10:07:27,280 - lava_callback - INFO - Starting async job submit for node 69cf8cb1000c489abc2c0fd0 2026-04-03 10:07:27,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:07:27,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8cb1000c489abc2c0fd0 HTTP/1.1" 200 None 2026-04-03 10:07:27,349 - lava_callback - DEBUG - Processing node 69cf8cb1000c489abc2c0fd0: baseline-nfs-arm64 2026-04-03 10:07:28,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:07:28,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:07:28,757 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf8cb1000c489abc2c0fd0/log.txt.gz 2026-04-03 10:07:29,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:07:29,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:07:29,563 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf8cb1000c489abc2c0fd0/lava-logs.yaml 2026-04-03 10:07:29,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:07:29,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:07:29,794 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf8cb1000c489abc2c0fd0/lava_callback.json.gz DEBUG: Setup failed for 69cf8cb1000c489abc2c0fd0. Transitting job node result: incomplete -> fail 2026-04-03 10:07:29,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:07:29,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8cb1000c489abc2c0fd0 HTTP/1.1" 200 None 2026-04-03 10:07:29,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:07:29,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 10:07:29,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:07:29,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8cb1000c489abc2c0fd0 HTTP/1.1" 200 None 2026-04-03 10:07:30,076 - lava_callback - INFO - Completed processing callback for node 69cf8cb1000c489abc2c0fd0 INFO: 172.21.0.1:45816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41920 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:07:52,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:07:52,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53166 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:09:24,587 - lava_callback - INFO - Received callback for node 69cceec574bda3ec93a42148 INFO: 172.21.0.1:58196 - "POST /node/69cceec574bda3ec93a42148 HTTP/1.0" 202 Accepted 2026-04-03 10:09:24,593 - lava_callback - INFO - Starting async job submit for node 69cceec574bda3ec93a42148 2026-04-03 10:09:24,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:09:24,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cceec574bda3ec93a42148 HTTP/1.1" 200 None 2026-04-03 10:09:24,609 - lava_callback - DEBUG - Processing node 69cceec574bda3ec93a42148: baseline-arm64-baylibre 2026-04-03 10:09:24,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cceec574bda3ec93a42148 2026-04-03 10:09:24,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:09:24,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cceec574bda3ec93a42148 HTTP/1.1" 200 None 2026-04-03 10:09:24,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:09:24,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:09:24,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:09:24,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cceec574bda3ec93a42148 HTTP/1.1" 200 None 2026-04-03 10:09:24,661 - lava_callback - ERROR - Error processing callback for node 69cceec574bda3ec93a42148: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 10:09:25,226 - lava_callback - INFO - Received callback for node 69cf901c000c489abc2c12b8 INFO: 172.21.0.1:58206 - "POST /node/69cf901c000c489abc2c12b8 HTTP/1.0" 202 Accepted 2026-04-03 10:09:25,237 - lava_callback - INFO - Starting async job submit for node 69cf901c000c489abc2c12b8 2026-04-03 10:09:25,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:09:25,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf901c000c489abc2c12b8 HTTP/1.1" 200 None 2026-04-03 10:09:25,305 - lava_callback - DEBUG - Processing node 69cf901c000c489abc2c12b8: baseline-arm64 2026-04-03 10:09:25,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:09:25,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:09:25,947 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf901c000c489abc2c12b8/log.txt.gz 2026-04-03 10:09:26,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:09:26,425 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:09:26,427 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf901c000c489abc2c12b8/lava-logs.yaml 2026-04-03 10:09:26,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:58222 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:09:26,589 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:09:26,591 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf901c000c489abc2c12b8/lava_callback.json.gz DEBUG: Setup failed for 69cf901c000c489abc2c12b8. Transitting job node result: incomplete -> fail 2026-04-03 10:09:26,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:09:26,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf901c000c489abc2c12b8 HTTP/1.1" 200 None 2026-04-03 10:09:26,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:09:26,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 10:09:26,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:09:26,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf901c000c489abc2c12b8 HTTP/1.1" 200 None 2026-04-03 10:09:26,825 - lava_callback - INFO - Completed processing callback for node 69cf901c000c489abc2c12b8 INFO: 172.21.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34042 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:09:52,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:09:52,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43368 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:11:29,661 - lava_callback - INFO - Received callback for node 69cf8dcf000c489abc2c1089 INFO: 172.21.0.1:43384 - "POST /node/69cf8dcf000c489abc2c1089 HTTP/1.0" 202 Accepted 2026-04-03 10:11:29,709 - lava_callback - INFO - Starting async job submit for node 69cf8dcf000c489abc2c1089 2026-04-03 10:11:29,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:11:29,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8dcf000c489abc2c1089 HTTP/1.1" 200 None 2026-04-03 10:11:29,750 - lava_callback - DEBUG - Processing node 69cf8dcf000c489abc2c1089: baseline-arm-pengutronix 2026-04-03 10:11:29,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:11:29,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:11:29,991 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf8dcf000c489abc2c1089/log.txt.gz 2026-04-03 10:11:30,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:11:30,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:11:30,391 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf8dcf000c489abc2c1089/lava-logs.yaml 2026-04-03 10:11:30,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:11:30,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:11:30,532 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf8dcf000c489abc2c1089/lava_callback.json.gz DEBUG: Setup failed for 69cf8dcf000c489abc2c1089. Transitting job node result: incomplete -> fail 2026-04-03 10:11:30,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:11:30,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf8dcf000c489abc2c1089 HTTP/1.1" 200 None 2026-04-03 10:11:30,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:11:30,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 10:11:30,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:11:30,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf8dcf000c489abc2c1089 HTTP/1.1" 200 None 2026-04-03 10:11:30,697 - lava_callback - INFO - Completed processing callback for node 69cf8dcf000c489abc2c1089 INFO: 172.21.0.1:49656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53422 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:11:52,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:11:52,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42014 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:12:04,431 - lava_callback - INFO - Received callback for node 69ccef6b74bda3ec93a4216c INFO: 172.21.0.1:45654 - "POST /node/69ccef6b74bda3ec93a4216c HTTP/1.0" 202 Accepted 2026-04-03 10:12:04,437 - lava_callback - INFO - Starting async job submit for node 69ccef6b74bda3ec93a4216c 2026-04-03 10:12:04,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:12:04,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccef6b74bda3ec93a4216c HTTP/1.1" 200 None 2026-04-03 10:12:04,454 - lava_callback - DEBUG - Processing node 69ccef6b74bda3ec93a4216c: baseline-arm64-baylibre 2026-04-03 10:12:04,455 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccef6b74bda3ec93a4216c 2026-04-03 10:12:04,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:12:04,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccef6b74bda3ec93a4216c HTTP/1.1" 200 None 2026-04-03 10:12:04,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:12:04,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:12:04,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:12:04,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccef6b74bda3ec93a4216c HTTP/1.1" 200 None 2026-04-03 10:12:04,508 - lava_callback - ERROR - Error processing callback for node 69ccef6b74bda3ec93a4216c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52154 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:13:24,401 - lava_callback - INFO - Received callback for node 69ccefbf74bda3ec93a42182 INFO: 172.21.0.1:55260 - "POST /node/69ccefbf74bda3ec93a42182 HTTP/1.0" 202 Accepted 2026-04-03 10:13:24,407 - lava_callback - INFO - Starting async job submit for node 69ccefbf74bda3ec93a42182 2026-04-03 10:13:24,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:13:24,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccefbf74bda3ec93a42182 HTTP/1.1" 200 None 2026-04-03 10:13:24,433 - lava_callback - DEBUG - Processing node 69ccefbf74bda3ec93a42182: baseline-arm64-baylibre 2026-04-03 10:13:24,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccefbf74bda3ec93a42182 2026-04-03 10:13:24,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:13:24,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccefbf74bda3ec93a42182 HTTP/1.1" 200 None 2026-04-03 10:13:24,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:13:24,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:13:24,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:13:24,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccefbf74bda3ec93a42182 HTTP/1.1" 200 None 2026-04-03 10:13:24,493 - lava_callback - ERROR - Error processing callback for node 69ccefbf74bda3ec93a42182: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46846 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:14:31,203 - lava_callback - INFO - Received callback for node 69cf9082000c489abc2c1306 INFO: 172.21.0.1:53346 - "POST /node/69cf9082000c489abc2c1306 HTTP/1.0" 202 Accepted 2026-04-03 10:14:31,210 - lava_callback - INFO - Starting async job submit for node 69cf9082000c489abc2c1306 2026-04-03 10:14:31,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:14:31,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9082000c489abc2c1306 HTTP/1.1" 200 None 2026-04-03 10:14:31,249 - lava_callback - DEBUG - Processing node 69cf9082000c489abc2c1306: baseline-arm64 2026-04-03 10:14:31,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:14:31,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:14:31,914 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf9082000c489abc2c1306/log.txt.gz 2026-04-03 10:14:32,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:14:32,394 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:14:32,396 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf9082000c489abc2c1306/lava-logs.yaml 2026-04-03 10:14:32,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:14:32,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:14:32,602 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf9082000c489abc2c1306/lava_callback.json.gz 2026-04-03 10:14:32,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:14:32,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9082000c489abc2c1306 HTTP/1.1" 200 None 2026-04-03 10:14:32,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:14:32,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 10:14:32,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:14:32,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf9082000c489abc2c1306 HTTP/1.1" 200 None 2026-04-03 10:14:32,765 - lava_callback - INFO - Completed processing callback for node 69cf9082000c489abc2c1306 INFO: 172.21.0.1:53348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47922 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:14:52,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:14:52,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43260 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:15:04,536 - lava_callback - INFO - Received callback for node 69ccf01b74bda3ec93a42199 INFO: 172.21.0.1:53896 - "POST /node/69ccf01b74bda3ec93a42199 HTTP/1.0" 202 Accepted 2026-04-03 10:15:04,542 - lava_callback - INFO - Starting async job submit for node 69ccf01b74bda3ec93a42199 2026-04-03 10:15:04,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:15:04,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf01b74bda3ec93a42199 HTTP/1.1" 200 None 2026-04-03 10:15:04,563 - lava_callback - DEBUG - Processing node 69ccf01b74bda3ec93a42199: baseline-arm64-baylibre 2026-04-03 10:15:04,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccf01b74bda3ec93a42199 2026-04-03 10:15:04,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:15:04,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf01b74bda3ec93a42199 HTTP/1.1" 200 None 2026-04-03 10:15:04,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:15:04,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:15:04,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:15:04,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccf01b74bda3ec93a42199 HTTP/1.1" 200 None 2026-04-03 10:15:04,618 - lava_callback - ERROR - Error processing callback for node 69ccf01b74bda3ec93a42199: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46106 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:15:50,219 - lava_callback - INFO - Received callback for node 69cf90cd000c489abc2c135e INFO: 172.21.0.1:55322 - "POST /node/69cf90cd000c489abc2c135e HTTP/1.0" 202 Accepted 2026-04-03 10:15:50,226 - lava_callback - INFO - Starting async job submit for node 69cf90cd000c489abc2c135e 2026-04-03 10:15:50,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:15:50,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf90cd000c489abc2c135e HTTP/1.1" 200 None 2026-04-03 10:15:50,268 - lava_callback - DEBUG - Processing node 69cf90cd000c489abc2c135e: baseline-nfs-arm64 2026-04-03 10:15:50,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:15:51,035 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:15:51,037 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf90cd000c489abc2c135e/log.txt.gz 2026-04-03 10:15:51,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:15:51,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:15:51,531 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf90cd000c489abc2c135e/lava-logs.yaml 2026-04-03 10:15:51,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:15:51,732 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:15:51,733 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf90cd000c489abc2c135e/lava_callback.json.gz 2026-04-03 10:15:51,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:15:51,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf90cd000c489abc2c135e HTTP/1.1" 200 None 2026-04-03 10:15:51,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:15:51,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 10:15:51,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:15:51,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf90cd000c489abc2c135e HTTP/1.1" 200 None 2026-04-03 10:15:51,875 - lava_callback - INFO - Completed processing callback for node 69cf90cd000c489abc2c135e 2026-04-03 10:15:52,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:15:52,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56750 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:16:24,551 - lava_callback - INFO - Received callback for node 69ccf06d74bda3ec93a421b3 INFO: 172.21.0.1:56306 - "POST /node/69ccf06d74bda3ec93a421b3 HTTP/1.0" 202 Accepted 2026-04-03 10:16:24,557 - lava_callback - INFO - Starting async job submit for node 69ccf06d74bda3ec93a421b3 2026-04-03 10:16:24,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:16:24,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf06d74bda3ec93a421b3 HTTP/1.1" 200 None 2026-04-03 10:16:24,574 - lava_callback - DEBUG - Processing node 69ccf06d74bda3ec93a421b3: baseline-arm64-baylibre 2026-04-03 10:16:24,574 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccf06d74bda3ec93a421b3 2026-04-03 10:16:24,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:16:24,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf06d74bda3ec93a421b3 HTTP/1.1" 200 None 2026-04-03 10:16:24,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:16:24,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:16:24,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:16:24,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccf06d74bda3ec93a421b3 HTTP/1.1" 200 None 2026-04-03 10:16:24,627 - lava_callback - ERROR - Error processing callback for node 69ccf06d74bda3ec93a421b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44170 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:19:24,725 - lava_callback - INFO - Received callback for node 69ccf11b74bda3ec93a421fb INFO: 172.21.0.1:46652 - "POST /node/69ccf11b74bda3ec93a421fb HTTP/1.0" 202 Accepted 2026-04-03 10:19:24,733 - lava_callback - INFO - Starting async job submit for node 69ccf11b74bda3ec93a421fb 2026-04-03 10:19:24,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:19:24,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf11b74bda3ec93a421fb HTTP/1.1" 200 None 2026-04-03 10:19:24,757 - lava_callback - DEBUG - Processing node 69ccf11b74bda3ec93a421fb: baseline-arm64-baylibre 2026-04-03 10:19:24,757 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccf11b74bda3ec93a421fb 2026-04-03 10:19:24,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:19:24,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf11b74bda3ec93a421fb HTTP/1.1" 200 None 2026-04-03 10:19:24,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:19:24,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:19:24,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:19:24,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccf11b74bda3ec93a421fb HTTP/1.1" 200 None 2026-04-03 10:19:24,830 - lava_callback - ERROR - Error processing callback for node 69ccf11b74bda3ec93a421fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57500 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:22:16,966 - lava_callback - INFO - Received callback for node 69ccf1cd74bda3ec93a42246 INFO: 172.21.0.1:57508 - "POST /node/69ccf1cd74bda3ec93a42246 HTTP/1.0" 202 Accepted 2026-04-03 10:22:16,975 - lava_callback - INFO - Starting async job submit for node 69ccf1cd74bda3ec93a42246 2026-04-03 10:22:16,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:22:16,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf1cd74bda3ec93a42246 HTTP/1.1" 200 None 2026-04-03 10:22:16,994 - lava_callback - DEBUG - Processing node 69ccf1cd74bda3ec93a42246: baseline-arm64-clabbe 2026-04-03 10:22:16,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccf1cd74bda3ec93a42246 2026-04-03 10:22:16,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:22:17,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf1cd74bda3ec93a42246 HTTP/1.1" 200 None 2026-04-03 10:22:17,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:22:17,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:22:17,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:22:17,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccf1cd74bda3ec93a42246 HTTP/1.1" 200 None 2026-04-03 10:22:17,070 - lava_callback - ERROR - Error processing callback for node 69ccf1cd74bda3ec93a42246: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45822 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:23:36,983 - lava_callback - INFO - Received callback for node 69ccf22474bda3ec93a4225f INFO: 172.21.0.1:45824 - "POST /node/69ccf22474bda3ec93a4225f HTTP/1.0" 202 Accepted 2026-04-03 10:23:36,989 - lava_callback - INFO - Starting async job submit for node 69ccf22474bda3ec93a4225f 2026-04-03 10:23:36,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:23:37,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf22474bda3ec93a4225f HTTP/1.1" 200 None 2026-04-03 10:23:37,005 - lava_callback - DEBUG - Processing node 69ccf22474bda3ec93a4225f: baseline-arm64-clabbe 2026-04-03 10:23:37,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccf22474bda3ec93a4225f 2026-04-03 10:23:37,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:23:37,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf22474bda3ec93a4225f HTTP/1.1" 200 None 2026-04-03 10:23:37,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:23:37,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:23:37,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:23:37,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccf22474bda3ec93a4225f HTTP/1.1" 200 None 2026-04-03 10:23:37,085 - lava_callback - ERROR - Error processing callback for node 69ccf22474bda3ec93a4225f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48492 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:26:37,128 - lava_callback - INFO - Received callback for node 69ccf2c974bda3ec93a42289 INFO: 172.21.0.1:48500 - "POST /node/69ccf2c974bda3ec93a42289 HTTP/1.0" 202 Accepted 2026-04-03 10:26:37,134 - lava_callback - INFO - Starting async job submit for node 69ccf2c974bda3ec93a42289 2026-04-03 10:26:37,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:26:37,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf2c974bda3ec93a42289 HTTP/1.1" 200 None 2026-04-03 10:26:37,150 - lava_callback - DEBUG - Processing node 69ccf2c974bda3ec93a42289: baseline-arm64-clabbe 2026-04-03 10:26:37,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccf2c974bda3ec93a42289 2026-04-03 10:26:37,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:26:37,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf2c974bda3ec93a42289 HTTP/1.1" 200 None 2026-04-03 10:26:37,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:26:37,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:26:37,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:26:37,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccf2c974bda3ec93a42289 HTTP/1.1" 200 None 2026-04-03 10:26:37,205 - lava_callback - ERROR - Error processing callback for node 69ccf2c974bda3ec93a42289: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42858 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:29:17,238 - lava_callback - INFO - Received callback for node 69ccf37574bda3ec93a422cf INFO: 172.21.0.1:42862 - "POST /node/69ccf37574bda3ec93a422cf HTTP/1.0" 202 Accepted 2026-04-03 10:29:17,243 - lava_callback - INFO - Starting async job submit for node 69ccf37574bda3ec93a422cf 2026-04-03 10:29:17,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:29:17,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf37574bda3ec93a422cf HTTP/1.1" 200 None 2026-04-03 10:29:17,259 - lava_callback - DEBUG - Processing node 69ccf37574bda3ec93a422cf: baseline-arm64-clabbe 2026-04-03 10:29:17,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccf37574bda3ec93a422cf 2026-04-03 10:29:17,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:29:17,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf37574bda3ec93a422cf HTTP/1.1" 200 None 2026-04-03 10:29:17,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:29:17,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:29:17,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:29:17,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccf37574bda3ec93a422cf HTTP/1.1" 200 None 2026-04-03 10:29:17,337 - lava_callback - ERROR - Error processing callback for node 69ccf37574bda3ec93a422cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54684 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:30:57,479 - lava_callback - INFO - Received callback for node 69ccf3cb74bda3ec93a422e7 INFO: 172.21.0.1:54700 - "POST /node/69ccf3cb74bda3ec93a422e7 HTTP/1.0" 202 Accepted 2026-04-03 10:30:57,484 - lava_callback - INFO - Starting async job submit for node 69ccf3cb74bda3ec93a422e7 2026-04-03 10:30:57,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:30:57,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf3cb74bda3ec93a422e7 HTTP/1.1" 200 None 2026-04-03 10:30:57,510 - lava_callback - DEBUG - Processing node 69ccf3cb74bda3ec93a422e7: baseline-arm64-clabbe 2026-04-03 10:30:57,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccf3cb74bda3ec93a422e7 2026-04-03 10:30:57,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:30:57,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf3cb74bda3ec93a422e7 HTTP/1.1" 200 None 2026-04-03 10:30:57,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:30:57,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:30:57,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:30:57,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccf3cb74bda3ec93a422e7 HTTP/1.1" 200 None 2026-04-03 10:30:57,566 - lava_callback - ERROR - Error processing callback for node 69ccf3cb74bda3ec93a422e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38712 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:32:45,133 - lava_callback - INFO - Received callback for node 69cf9598000c489abc2c1719 INFO: 172.21.0.1:37250 - "POST /node/69cf9598000c489abc2c1719 HTTP/1.0" 202 Accepted 2026-04-03 10:32:45,141 - lava_callback - INFO - Starting async job submit for node 69cf9598000c489abc2c1719 2026-04-03 10:32:45,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:32:45,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9598000c489abc2c1719 HTTP/1.1" 200 None 2026-04-03 10:32:45,201 - lava_callback - DEBUG - Processing node 69cf9598000c489abc2c1719: baseline-arm64 2026-04-03 10:32:45,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:32:45,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:32:45,764 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf9598000c489abc2c1719/log.txt.gz 2026-04-03 10:32:45,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:32:46,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:32:46,230 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf9598000c489abc2c1719/lava-logs.yaml 2026-04-03 10:32:46,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:32:46,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:32:46,404 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf9598000c489abc2c1719/lava_callback.json.gz DEBUG: Setup failed for 69cf9598000c489abc2c1719. Transitting job node result: incomplete -> fail 2026-04-03 10:32:46,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:32:46,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9598000c489abc2c1719 HTTP/1.1" 200 None 2026-04-03 10:32:46,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:37264 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:32:46,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 10:32:46,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:32:46,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf9598000c489abc2c1719 HTTP/1.1" 200 None 2026-04-03 10:32:46,609 - lava_callback - INFO - Completed processing callback for node 69cf9598000c489abc2c1719 2026-04-03 10:32:52,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:32:52,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59098 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:33:37,489 - lava_callback - INFO - Received callback for node 69ccf47174bda3ec93a42312 INFO: 172.21.0.1:59104 - "POST /node/69ccf47174bda3ec93a42312 HTTP/1.0" 202 Accepted 2026-04-03 10:33:37,494 - lava_callback - INFO - Starting async job submit for node 69ccf47174bda3ec93a42312 2026-04-03 10:33:37,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:33:37,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf47174bda3ec93a42312 HTTP/1.1" 200 None 2026-04-03 10:33:37,511 - lava_callback - DEBUG - Processing node 69ccf47174bda3ec93a42312: baseline-arm64-clabbe 2026-04-03 10:33:37,511 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccf47174bda3ec93a42312 2026-04-03 10:33:37,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:33:37,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccf47174bda3ec93a42312 HTTP/1.1" 200 None 2026-04-03 10:33:37,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:33:37,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-04-03 10:33:37,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:33:37,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccf47174bda3ec93a42312 HTTP/1.1" 200 None 2026-04-03 10:33:37,569 - lava_callback - ERROR - Error processing callback for node 69ccf47174bda3ec93a42312: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:37:15,116 - lava_callback - INFO - Received callback for node 69cf95f8000c489abc2c1781 INFO: 172.21.0.1:45814 - "POST /node/69cf95f8000c489abc2c1781 HTTP/1.0" 202 Accepted 2026-04-03 10:37:15,124 - lava_callback - INFO - Starting async job submit for node 69cf95f8000c489abc2c1781 2026-04-03 10:37:15,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:45818 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 10:37:15,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf95f8000c489abc2c1781 HTTP/1.1" 200 None 2026-04-03 10:37:15,167 - lava_callback - DEBUG - Processing node 69cf95f8000c489abc2c1781: baseline-arm64 2026-04-03 10:37:15,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:37:15,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:37:15,847 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf95f8000c489abc2c1781/log.txt.gz 2026-04-03 10:37:16,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:37:16,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:37:16,318 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf95f8000c489abc2c1781/lava-logs.yaml 2026-04-03 10:37:16,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:37:16,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:37:16,483 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf95f8000c489abc2c1781/lava_callback.json.gz 2026-04-03 10:37:16,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:45834 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:37:16,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf95f8000c489abc2c1781 HTTP/1.1" 200 None 2026-04-03 10:37:16,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:37:16,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 10:37:16,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:37:16,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf95f8000c489abc2c1781 HTTP/1.1" 200 None 2026-04-03 10:37:16,612 - lava_callback - INFO - Completed processing callback for node 69cf95f8000c489abc2c1781 2026-04-03 10:37:22,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:37:22,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60606 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:39:08,929 - lava_callback - INFO - Received callback for node 69cf9643000c489abc2c17bd INFO: 172.21.0.1:60614 - "POST /node/69cf9643000c489abc2c17bd HTTP/1.0" 202 Accepted 2026-04-03 10:39:08,936 - lava_callback - INFO - Starting async job submit for node 69cf9643000c489abc2c17bd 2026-04-03 10:39:08,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:39:08,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9643000c489abc2c17bd HTTP/1.1" 200 None 2026-04-03 10:39:08,986 - lava_callback - DEBUG - Processing node 69cf9643000c489abc2c17bd: baseline-nfs-arm64 2026-04-03 10:39:09,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:39:09,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:39:09,863 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf9643000c489abc2c17bd/log.txt.gz 2026-04-03 10:39:10,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:39:10,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:39:10,346 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf9643000c489abc2c17bd/lava-logs.yaml 2026-04-03 10:39:10,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:39:10,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:39:10,538 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf9643000c489abc2c17bd/lava_callback.json.gz 2026-04-03 10:39:10,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:39:10,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9643000c489abc2c17bd HTTP/1.1" 200 None 2026-04-03 10:39:10,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:39:10,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 10:39:10,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:39:10,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf9643000c489abc2c17bd HTTP/1.1" 200 None 2026-04-03 10:39:10,727 - lava_callback - INFO - Completed processing callback for node 69cf9643000c489abc2c17bd INFO: 172.21.0.1:48776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48778 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:39:22,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:39:22,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40160 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:40:39,961 - lava_callback - INFO - Received callback for node 69cf952e000c489abc2c169c INFO: 172.21.0.1:44344 - "POST /node/69cf952e000c489abc2c169c HTTP/1.0" 202 Accepted 2026-04-03 10:40:39,968 - lava_callback - INFO - Starting async job submit for node 69cf952e000c489abc2c169c 2026-04-03 10:40:40,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:40:40,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf952e000c489abc2c169c HTTP/1.1" 200 None 2026-04-03 10:40:40,042 - lava_callback - DEBUG - Processing node 69cf952e000c489abc2c169c: baseline-arm 2026-04-03 10:40:40,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:40:40,883 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:40:40,885 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf952e000c489abc2c169c/log.txt.gz 2026-04-03 10:40:41,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:40:41,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:40:41,538 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf952e000c489abc2c169c/lava-logs.yaml 2026-04-03 10:40:41,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:40:41,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:40:41,737 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf952e000c489abc2c169c/lava_callback.json.gz DEBUG: Setup failed for 69cf952e000c489abc2c169c. Transitting job node result: incomplete -> fail 2026-04-03 10:40:41,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:40:41,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf952e000c489abc2c169c HTTP/1.1" 200 None 2026-04-03 10:40:41,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:40:41,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 10:40:41,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:40:41,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf952e000c489abc2c169c HTTP/1.1" 200 None 2026-04-03 10:40:42,026 - lava_callback - INFO - Completed processing callback for node 69cf952e000c489abc2c169c INFO: 172.21.0.1:44352 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:40:52,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:40:52,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55822 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:44:26,808 - lava_callback - INFO - Received callback for node 69cf9555000c489abc2c16dd INFO: 172.21.0.1:55830 - "POST /node/69cf9555000c489abc2c16dd HTTP/1.0" 202 Accepted 2026-04-03 10:44:26,815 - lava_callback - INFO - Starting async job submit for node 69cf9555000c489abc2c16dd 2026-04-03 10:44:26,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:44:26,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9555000c489abc2c16dd HTTP/1.1" 200 None 2026-04-03 10:44:26,875 - lava_callback - DEBUG - Processing node 69cf9555000c489abc2c16dd: baseline-nfs-arm64 2026-04-03 10:44:28,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:44:28,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:44:28,225 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf9555000c489abc2c16dd/log.txt.gz 2026-04-03 10:44:28,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:44:29,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:44:29,121 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf9555000c489abc2c16dd/lava-logs.yaml 2026-04-03 10:44:29,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:44:29,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:44:29,322 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf9555000c489abc2c16dd/lava_callback.json.gz DEBUG: Setup failed for 69cf9555000c489abc2c16dd. Transitting job node result: incomplete -> fail 2026-04-03 10:44:29,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:44:29,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9555000c489abc2c16dd HTTP/1.1" 200 None 2026-04-03 10:44:29,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:44:29,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 10:44:29,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:44:29,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf9555000c489abc2c16dd HTTP/1.1" 200 None 2026-04-03 10:44:29,549 - lava_callback - INFO - Completed processing callback for node 69cf9555000c489abc2c16dd INFO: 172.21.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55126 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:44:52,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:44:52,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58492 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:45:51,517 - lava_callback - INFO - Received callback for node 69cf95df000c489abc2c1757 INFO: 172.21.0.1:47868 - "POST /node/69cf95df000c489abc2c1757 HTTP/1.0" 202 Accepted 2026-04-03 10:45:51,565 - lava_callback - INFO - Starting async job submit for node 69cf95df000c489abc2c1757 2026-04-03 10:45:51,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:45:51,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf95df000c489abc2c1757 HTTP/1.1" 200 None 2026-04-03 10:45:51,605 - lava_callback - DEBUG - Processing node 69cf95df000c489abc2c1757: baseline-arm-pengutronix 2026-04-03 10:45:51,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:45:51,849 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:45:51,851 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf95df000c489abc2c1757/log.txt.gz 2026-04-03 10:45:51,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:45:52,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:45:52,095 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf95df000c489abc2c1757/lava-logs.yaml 2026-04-03 10:45:52,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:45:52,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:45:52,214 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf95df000c489abc2c1757/lava_callback.json.gz DEBUG: Setup failed for 69cf95df000c489abc2c1757. Transitting job node result: incomplete -> fail 2026-04-03 10:45:52,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:45:52,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf95df000c489abc2c1757 HTTP/1.1" 200 None 2026-04-03 10:45:52,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:45:52,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 10:45:52,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:45:52,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf95df000c489abc2c1757 HTTP/1.1" 200 None 2026-04-03 10:45:52,362 - lava_callback - INFO - Completed processing callback for node 69cf95df000c489abc2c1757 2026-04-03 10:45:52,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:45:52,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49418 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:58:56,365 - lava_callback - INFO - Received callback for node 69cf992b000c489abc2c1800 INFO: 172.21.0.1:49434 - "POST /node/69cf992b000c489abc2c1800 HTTP/1.0" 202 Accepted 2026-04-03 10:58:56,372 - lava_callback - INFO - Starting async job submit for node 69cf992b000c489abc2c1800 2026-04-03 10:58:56,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:58:56,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf992b000c489abc2c1800 HTTP/1.1" 200 None 2026-04-03 10:58:56,442 - lava_callback - DEBUG - Processing node 69cf992b000c489abc2c1800: baseline-arm INFO: 172.21.0.1:49444 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:58:57,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:58:57,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:58:57,193 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf992b000c489abc2c1800/log.txt.gz 2026-04-03 10:58:57,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:58:57,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:58:57,844 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf992b000c489abc2c1800/lava-logs.yaml 2026-04-03 10:58:57,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 10:58:58,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 10:58:58,036 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf992b000c489abc2c1800/lava_callback.json.gz DEBUG: Setup failed for 69cf992b000c489abc2c1800. Transitting job node result: incomplete -> fail 2026-04-03 10:58:58,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:58:58,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf992b000c489abc2c1800 HTTP/1.1" 200 None 2026-04-03 10:58:58,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:58:58,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 10:58:58,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:58:58,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf992b000c489abc2c1800 HTTP/1.1" 200 None 2026-04-03 10:58:58,338 - lava_callback - INFO - Completed processing callback for node 69cf992b000c489abc2c1800 INFO: 172.21.0.1:48656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59210 - "GET / HTTP/1.0" 200 OK 2026-04-03 10:59:22,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 10:59:22,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34892 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:04:23,973 - lava_callback - INFO - Received callback for node 69cf9a16000c489abc2c181e INFO: 172.21.0.1:58514 - "POST /node/69cf9a16000c489abc2c181e HTTP/1.0" 202 Accepted 2026-04-03 11:04:23,981 - lava_callback - INFO - Starting async job submit for node 69cf9a16000c489abc2c181e 2026-04-03 11:04:24,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:04:24,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9a16000c489abc2c181e HTTP/1.1" 200 None 2026-04-03 11:04:24,049 - lava_callback - DEBUG - Processing node 69cf9a16000c489abc2c181e: baseline-nfs-arm64 2026-04-03 11:04:25,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 11:04:25,256 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 11:04:25,259 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf9a16000c489abc2c181e/log.txt.gz 2026-04-03 11:04:25,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 11:04:26,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 11:04:26,180 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf9a16000c489abc2c181e/lava-logs.yaml 2026-04-03 11:04:26,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 11:04:26,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 11:04:26,408 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf9a16000c489abc2c181e/lava_callback.json.gz DEBUG: Setup failed for 69cf9a16000c489abc2c181e. Transitting job node result: incomplete -> fail 2026-04-03 11:04:26,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:04:26,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9a16000c489abc2c181e HTTP/1.1" 200 None INFO: 172.21.0.1:58530 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:04:26,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:04:26,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77bf000c489abc2bf938 HTTP/1.1" 200 None 2026-04-03 11:04:26,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:04:26,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf9a16000c489abc2c181e HTTP/1.1" 200 None 2026-04-03 11:04:26,663 - lava_callback - INFO - Completed processing callback for node 69cf9a16000c489abc2c181e INFO: 172.21.0.1:50452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40670 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:04:52,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:04:52,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46370 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:05:12,605 - lava_callback - INFO - Received callback for node 69cf9a6b000c489abc2c1870 INFO: 172.21.0.1:46894 - "POST /node/69cf9a6b000c489abc2c1870 HTTP/1.0" 202 Accepted 2026-04-03 11:05:12,657 - lava_callback - INFO - Starting async job submit for node 69cf9a6b000c489abc2c1870 2026-04-03 11:05:12,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:05:12,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9a6b000c489abc2c1870 HTTP/1.1" 200 None 2026-04-03 11:05:12,707 - lava_callback - DEBUG - Processing node 69cf9a6b000c489abc2c1870: baseline-arm-pengutronix 2026-04-03 11:05:12,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 11:05:12,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 11:05:12,999 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf9a6b000c489abc2c1870/log.txt.gz 2026-04-03 11:05:13,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 11:05:13,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 11:05:13,225 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf9a6b000c489abc2c1870/lava-logs.yaml 2026-04-03 11:05:13,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 11:05:13,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 11:05:13,350 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf9a6b000c489abc2c1870/lava_callback.json.gz DEBUG: Setup failed for 69cf9a6b000c489abc2c1870. Transitting job node result: incomplete -> fail 2026-04-03 11:05:13,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:05:13,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf9a6b000c489abc2c1870 HTTP/1.1" 200 None 2026-04-03 11:05:13,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:05:13,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77b3000c489abc2bf929 HTTP/1.1" 200 None 2026-04-03 11:05:13,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:05:13,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf9a6b000c489abc2c1870 HTTP/1.1" 200 None 2026-04-03 11:05:13,520 - lava_callback - INFO - Completed processing callback for node 69cf9a6b000c489abc2c1870 INFO: 172.21.0.1:46896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46902 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:05:22,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:05:22,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39346 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56060 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:13:59,205 - lava_callback - INFO - Received callback for node 69ccfde674bda3ec93a4248d INFO: 172.21.0.1:56064 - "POST /node/69ccfde674bda3ec93a4248d HTTP/1.0" 202 Accepted 2026-04-03 11:13:59,211 - lava_callback - INFO - Starting async job submit for node 69ccfde674bda3ec93a4248d 2026-04-03 11:13:59,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:13:59,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccfde674bda3ec93a4248d HTTP/1.1" 200 None 2026-04-03 11:13:59,234 - lava_callback - DEBUG - Processing node 69ccfde674bda3ec93a4248d: baseline-arm-clabbe 2026-04-03 11:13:59,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccfde674bda3ec93a4248d 2026-04-03 11:13:59,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:13:59,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccfde674bda3ec93a4248d HTTP/1.1" 200 None 2026-04-03 11:13:59,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:13:59,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 11:13:59,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:13:59,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccfde674bda3ec93a4248d HTTP/1.1" 200 None 2026-04-03 11:13:59,307 - lava_callback - ERROR - Error processing callback for node 69ccfde674bda3ec93a4248d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57858 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:15:59,299 - lava_callback - INFO - Received callback for node 69ccfe5f74bda3ec93a4249d INFO: 172.21.0.1:57868 - "POST /node/69ccfe5f74bda3ec93a4249d HTTP/1.0" 202 Accepted 2026-04-03 11:15:59,305 - lava_callback - INFO - Starting async job submit for node 69ccfe5f74bda3ec93a4249d 2026-04-03 11:15:59,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:15:59,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccfe5f74bda3ec93a4249d HTTP/1.1" 200 None 2026-04-03 11:15:59,324 - lava_callback - DEBUG - Processing node 69ccfe5f74bda3ec93a4249d: baseline-arm-clabbe 2026-04-03 11:15:59,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccfe5f74bda3ec93a4249d 2026-04-03 11:15:59,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:15:59,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccfe5f74bda3ec93a4249d HTTP/1.1" 200 None 2026-04-03 11:15:59,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:15:59,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 11:15:59,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:15:59,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccfe5f74bda3ec93a4249d HTTP/1.1" 200 None 2026-04-03 11:15:59,399 - lava_callback - ERROR - Error processing callback for node 69ccfe5f74bda3ec93a4249d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59944 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:16:39,273 - lava_callback - INFO - Received callback for node 69ccfe8674bda3ec93a424ab INFO: 172.21.0.1:59954 - "POST /node/69ccfe8674bda3ec93a424ab HTTP/1.0" 202 Accepted 2026-04-03 11:16:39,281 - lava_callback - INFO - Starting async job submit for node 69ccfe8674bda3ec93a424ab 2026-04-03 11:16:39,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:16:39,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccfe8674bda3ec93a424ab HTTP/1.1" 200 None 2026-04-03 11:16:39,303 - lava_callback - DEBUG - Processing node 69ccfe8674bda3ec93a424ab: baseline-arm-clabbe 2026-04-03 11:16:39,304 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccfe8674bda3ec93a424ab 2026-04-03 11:16:39,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:16:39,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccfe8674bda3ec93a424ab HTTP/1.1" 200 None 2026-04-03 11:16:39,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:16:39,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 11:16:39,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:16:39,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccfe8674bda3ec93a424ab HTTP/1.1" 200 None 2026-04-03 11:16:39,365 - lava_callback - ERROR - Error processing callback for node 69ccfe8674bda3ec93a424ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33448 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:17:59,346 - lava_callback - INFO - Received callback for node 69ccfed774bda3ec93a424bc INFO: 172.21.0.1:33464 - "POST /node/69ccfed774bda3ec93a424bc HTTP/1.0" 202 Accepted 2026-04-03 11:17:59,351 - lava_callback - INFO - Starting async job submit for node 69ccfed774bda3ec93a424bc 2026-04-03 11:17:59,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:17:59,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccfed774bda3ec93a424bc HTTP/1.1" 200 None 2026-04-03 11:17:59,368 - lava_callback - DEBUG - Processing node 69ccfed774bda3ec93a424bc: baseline-arm-clabbe 2026-04-03 11:17:59,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccfed774bda3ec93a424bc 2026-04-03 11:17:59,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:17:59,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccfed774bda3ec93a424bc HTTP/1.1" 200 None 2026-04-03 11:17:59,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:17:59,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 11:17:59,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:17:59,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccfed774bda3ec93a424bc HTTP/1.1" 200 None 2026-04-03 11:17:59,431 - lava_callback - ERROR - Error processing callback for node 69ccfed774bda3ec93a424bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54542 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:18:59,402 - lava_callback - INFO - Received callback for node 69ccff1374bda3ec93a424c9 INFO: 172.21.0.1:54544 - "POST /node/69ccff1374bda3ec93a424c9 HTTP/1.0" 202 Accepted 2026-04-03 11:18:59,408 - lava_callback - INFO - Starting async job submit for node 69ccff1374bda3ec93a424c9 2026-04-03 11:18:59,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:18:59,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccff1374bda3ec93a424c9 HTTP/1.1" 200 None 2026-04-03 11:18:59,432 - lava_callback - DEBUG - Processing node 69ccff1374bda3ec93a424c9: baseline-arm-clabbe 2026-04-03 11:18:59,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccff1374bda3ec93a424c9 2026-04-03 11:18:59,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:18:59,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccff1374bda3ec93a424c9 HTTP/1.1" 200 None 2026-04-03 11:18:59,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:18:59,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 11:18:59,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:18:59,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccff1374bda3ec93a424c9 HTTP/1.1" 200 None 2026-04-03 11:18:59,483 - lava_callback - ERROR - Error processing callback for node 69ccff1374bda3ec93a424c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38490 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:19:59,481 - lava_callback - INFO - Received callback for node 69ccff4f74bda3ec93a424d8 INFO: 172.21.0.1:38506 - "POST /node/69ccff4f74bda3ec93a424d8 HTTP/1.0" 202 Accepted 2026-04-03 11:19:59,488 - lava_callback - INFO - Starting async job submit for node 69ccff4f74bda3ec93a424d8 2026-04-03 11:19:59,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:19:59,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccff4f74bda3ec93a424d8 HTTP/1.1" 200 None 2026-04-03 11:19:59,511 - lava_callback - DEBUG - Processing node 69ccff4f74bda3ec93a424d8: baseline-arm-clabbe 2026-04-03 11:19:59,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccff4f74bda3ec93a424d8 2026-04-03 11:19:59,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:19:59,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccff4f74bda3ec93a424d8 HTTP/1.1" 200 None 2026-04-03 11:19:59,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:19:59,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 11:19:59,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:19:59,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccff4f74bda3ec93a424d8 HTTP/1.1" 200 None 2026-04-03 11:19:59,569 - lava_callback - ERROR - Error processing callback for node 69ccff4f74bda3ec93a424d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42292 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:34:08,740 - lava_callback - INFO - Received callback for node 69cd029e74bda3ec93a4254d INFO: 172.21.0.1:42302 - "POST /node/69cd029e74bda3ec93a4254d HTTP/1.0" 202 Accepted 2026-04-03 11:34:08,745 - lava_callback - INFO - Starting async job submit for node 69cd029e74bda3ec93a4254d 2026-04-03 11:34:08,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:34:08,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd029e74bda3ec93a4254d HTTP/1.1" 200 None 2026-04-03 11:34:08,782 - lava_callback - DEBUG - Processing node 69cd029e74bda3ec93a4254d: baseline-arm64-baylibre 2026-04-03 11:34:08,783 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd029e74bda3ec93a4254d 2026-04-03 11:34:08,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:34:08,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd029e74bda3ec93a4254d HTTP/1.1" 200 None 2026-04-03 11:34:08,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:34:08,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:34:08,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:34:08,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd029e74bda3ec93a4254d HTTP/1.1" 200 None 2026-04-03 11:34:08,857 - lava_callback - ERROR - Error processing callback for node 69cd029e74bda3ec93a4254d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39934 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:37:08,898 - lava_callback - INFO - Received callback for node 69cd035374bda3ec93a42567 INFO: 172.21.0.1:39948 - "POST /node/69cd035374bda3ec93a42567 HTTP/1.0" 202 Accepted 2026-04-03 11:37:08,904 - lava_callback - INFO - Starting async job submit for node 69cd035374bda3ec93a42567 2026-04-03 11:37:08,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:37:08,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd035374bda3ec93a42567 HTTP/1.1" 200 None 2026-04-03 11:37:08,920 - lava_callback - DEBUG - Processing node 69cd035374bda3ec93a42567: baseline-arm64-baylibre 2026-04-03 11:37:08,920 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd035374bda3ec93a42567 2026-04-03 11:37:08,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:37:08,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd035374bda3ec93a42567 HTTP/1.1" 200 None 2026-04-03 11:37:08,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:37:08,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:37:08,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:37:08,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd035374bda3ec93a42567 HTTP/1.1" 200 None 2026-04-03 11:37:08,995 - lava_callback - ERROR - Error processing callback for node 69cd035374bda3ec93a42567: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59878 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:38:28,969 - lava_callback - INFO - Received callback for node 69cd03a774bda3ec93a42581 INFO: 172.21.0.1:59892 - "POST /node/69cd03a774bda3ec93a42581 HTTP/1.0" 202 Accepted 2026-04-03 11:38:28,975 - lava_callback - INFO - Starting async job submit for node 69cd03a774bda3ec93a42581 2026-04-03 11:38:28,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:38:28,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd03a774bda3ec93a42581 HTTP/1.1" 200 None 2026-04-03 11:38:28,998 - lava_callback - DEBUG - Processing node 69cd03a774bda3ec93a42581: baseline-arm64-baylibre 2026-04-03 11:38:28,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd03a774bda3ec93a42581 2026-04-03 11:38:28,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:38:29,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd03a774bda3ec93a42581 HTTP/1.1" 200 None 2026-04-03 11:38:29,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:38:29,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:38:29,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:38:29,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd03a774bda3ec93a42581 HTTP/1.1" 200 None 2026-04-03 11:38:29,070 - lava_callback - ERROR - Error processing callback for node 69cd03a774bda3ec93a42581: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47894 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:40:09,070 - lava_callback - INFO - Received callback for node 69cd040974bda3ec93a4259a INFO: 172.21.0.1:47906 - "POST /node/69cd040974bda3ec93a4259a HTTP/1.0" 202 Accepted 2026-04-03 11:40:09,076 - lava_callback - INFO - Starting async job submit for node 69cd040974bda3ec93a4259a 2026-04-03 11:40:09,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:40:09,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd040974bda3ec93a4259a HTTP/1.1" 200 None 2026-04-03 11:40:09,092 - lava_callback - DEBUG - Processing node 69cd040974bda3ec93a4259a: baseline-arm64-baylibre 2026-04-03 11:40:09,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd040974bda3ec93a4259a 2026-04-03 11:40:09,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:40:09,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd040974bda3ec93a4259a HTTP/1.1" 200 None 2026-04-03 11:40:09,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:40:09,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:40:09,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:40:09,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd040974bda3ec93a4259a HTTP/1.1" 200 None 2026-04-03 11:40:09,145 - lava_callback - ERROR - Error processing callback for node 69cd040974bda3ec93a4259a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52438 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:41:29,288 - lava_callback - INFO - Received callback for node 69cd045f74bda3ec93a425b3 INFO: 172.21.0.1:52440 - "POST /node/69cd045f74bda3ec93a425b3 HTTP/1.0" 202 Accepted 2026-04-03 11:41:29,294 - lava_callback - INFO - Starting async job submit for node 69cd045f74bda3ec93a425b3 2026-04-03 11:41:29,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:41:29,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd045f74bda3ec93a425b3 HTTP/1.1" 200 None 2026-04-03 11:41:29,317 - lava_callback - DEBUG - Processing node 69cd045f74bda3ec93a425b3: baseline-arm64-baylibre 2026-04-03 11:41:29,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd045f74bda3ec93a425b3 2026-04-03 11:41:29,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:41:29,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd045f74bda3ec93a425b3 HTTP/1.1" 200 None 2026-04-03 11:41:29,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:41:29,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:41:29,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:41:29,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd045f74bda3ec93a425b3 HTTP/1.1" 200 None 2026-04-03 11:41:29,386 - lava_callback - ERROR - Error processing callback for node 69cd045f74bda3ec93a425b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45216 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:47:20,592 - lava_callback - INFO - Received callback for node 69cd05b674bda3ec93a425d1 INFO: 172.21.0.1:57302 - "POST /node/69cd05b674bda3ec93a425d1 HTTP/1.0" 202 Accepted 2026-04-03 11:47:20,598 - lava_callback - INFO - Starting async job submit for node 69cd05b674bda3ec93a425d1 2026-04-03 11:47:20,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:47:20,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd05b674bda3ec93a425d1 HTTP/1.1" 200 None 2026-04-03 11:47:20,614 - lava_callback - DEBUG - Processing node 69cd05b674bda3ec93a425d1: baseline-arm64-clabbe 2026-04-03 11:47:20,614 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd05b674bda3ec93a425d1 2026-04-03 11:47:20,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:47:20,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd05b674bda3ec93a425d1 HTTP/1.1" 200 None 2026-04-03 11:47:20,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:47:20,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:47:20,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:47:20,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd05b674bda3ec93a425d1 HTTP/1.1" 200 None 2026-04-03 11:47:20,672 - lava_callback - ERROR - Error processing callback for node 69cd05b674bda3ec93a425d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56768 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:49:20,930 - lava_callback - INFO - Received callback for node 69cd062d74bda3ec93a425ea INFO: 172.21.0.1:49118 - "POST /node/69cd062d74bda3ec93a425ea HTTP/1.0" 202 Accepted 2026-04-03 11:49:20,935 - lava_callback - INFO - Starting async job submit for node 69cd062d74bda3ec93a425ea 2026-04-03 11:49:20,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:49:20,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd062d74bda3ec93a425ea HTTP/1.1" 200 None 2026-04-03 11:49:20,952 - lava_callback - DEBUG - Processing node 69cd062d74bda3ec93a425ea: baseline-arm64-clabbe 2026-04-03 11:49:20,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd062d74bda3ec93a425ea 2026-04-03 11:49:20,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:49:20,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd062d74bda3ec93a425ea HTTP/1.1" 200 None 2026-04-03 11:49:20,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:49:20,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:49:20,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:49:21,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd062d74bda3ec93a425ea HTTP/1.1" 200 None 2026-04-03 11:49:21,018 - lava_callback - ERROR - Error processing callback for node 69cd062d74bda3ec93a425ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45654 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:50:40,712 - lava_callback - INFO - Received callback for node 69cd068174bda3ec93a425fe INFO: 172.21.0.1:50278 - "POST /node/69cd068174bda3ec93a425fe HTTP/1.0" 202 Accepted 2026-04-03 11:50:40,719 - lava_callback - INFO - Starting async job submit for node 69cd068174bda3ec93a425fe 2026-04-03 11:50:40,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:50:40,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd068174bda3ec93a425fe HTTP/1.1" 200 None 2026-04-03 11:50:40,736 - lava_callback - DEBUG - Processing node 69cd068174bda3ec93a425fe: baseline-arm64-clabbe 2026-04-03 11:50:40,736 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd068174bda3ec93a425fe 2026-04-03 11:50:40,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:50:40,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd068174bda3ec93a425fe HTTP/1.1" 200 None 2026-04-03 11:50:40,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:50:40,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:50:40,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:50:40,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd068174bda3ec93a425fe HTTP/1.1" 200 None 2026-04-03 11:50:40,793 - lava_callback - ERROR - Error processing callback for node 69cd068174bda3ec93a425fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42580 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:52:20,776 - lava_callback - INFO - Received callback for node 69cd06e174bda3ec93a4261c INFO: 172.21.0.1:47208 - "POST /node/69cd06e174bda3ec93a4261c HTTP/1.0" 202 Accepted 2026-04-03 11:52:20,782 - lava_callback - INFO - Starting async job submit for node 69cd06e174bda3ec93a4261c 2026-04-03 11:52:20,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:52:20,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd06e174bda3ec93a4261c HTTP/1.1" 200 None 2026-04-03 11:52:20,797 - lava_callback - DEBUG - Processing node 69cd06e174bda3ec93a4261c: baseline-arm64-clabbe 2026-04-03 11:52:20,798 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd06e174bda3ec93a4261c 2026-04-03 11:52:20,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:52:20,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd06e174bda3ec93a4261c HTTP/1.1" 200 None 2026-04-03 11:52:20,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:52:20,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:52:20,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:52:20,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd06e174bda3ec93a4261c HTTP/1.1" 200 None 2026-04-03 11:52:20,854 - lava_callback - ERROR - Error processing callback for node 69cd06e174bda3ec93a4261c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53578 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:53:40,814 - lava_callback - INFO - Received callback for node 69cd073974bda3ec93a4262f INFO: 172.21.0.1:41240 - "POST /node/69cd073974bda3ec93a4262f HTTP/1.0" 202 Accepted 2026-04-03 11:53:40,821 - lava_callback - INFO - Starting async job submit for node 69cd073974bda3ec93a4262f 2026-04-03 11:53:40,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:53:40,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd073974bda3ec93a4262f HTTP/1.1" 200 None 2026-04-03 11:53:40,837 - lava_callback - DEBUG - Processing node 69cd073974bda3ec93a4262f: baseline-arm64-clabbe 2026-04-03 11:53:40,837 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd073974bda3ec93a4262f 2026-04-03 11:53:40,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:53:40,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd073974bda3ec93a4262f HTTP/1.1" 200 None 2026-04-03 11:53:40,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:53:40,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:53:40,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:53:40,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd073974bda3ec93a4262f HTTP/1.1" 200 None 2026-04-03 11:53:40,897 - lava_callback - ERROR - Error processing callback for node 69cd073974bda3ec93a4262f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43846 - "GET / HTTP/1.0" 200 OK 2026-04-03 11:55:20,855 - lava_callback - INFO - Received callback for node 69cd079674bda3ec93a4264e INFO: 172.21.0.1:49350 - "POST /node/69cd079674bda3ec93a4264e HTTP/1.0" 202 Accepted 2026-04-03 11:55:20,861 - lava_callback - INFO - Starting async job submit for node 69cd079674bda3ec93a4264e 2026-04-03 11:55:20,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:55:20,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd079674bda3ec93a4264e HTTP/1.1" 200 None 2026-04-03 11:55:20,877 - lava_callback - DEBUG - Processing node 69cd079674bda3ec93a4264e: baseline-arm64-clabbe 2026-04-03 11:55:20,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd079674bda3ec93a4264e 2026-04-03 11:55:20,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:55:20,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd079674bda3ec93a4264e HTTP/1.1" 200 None 2026-04-03 11:55:20,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:55:20,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 11:55:20,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 11:55:20,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd079674bda3ec93a4264e HTTP/1.1" 200 None 2026-04-03 11:55:20,928 - lava_callback - ERROR - Error processing callback for node 69cd079674bda3ec93a4264e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49760 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53920 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:20:54,829 - lava_callback - INFO - Received callback for node 69cf7da8000c489abc2bfe7d INFO: 172.21.0.1:48768 - "POST /node/69cf7da8000c489abc2bfe7d HTTP/1.0" 202 Accepted 2026-04-03 12:20:54,835 - lava_callback - INFO - Starting async job submit for node 69cf7da8000c489abc2bfe7d 2026-04-03 12:20:54,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:20:54,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7da8000c489abc2bfe7d HTTP/1.1" 200 None 2026-04-03 12:20:54,867 - lava_callback - DEBUG - Processing node 69cf7da8000c489abc2bfe7d: baseline-x86 INFO: 172.21.0.1:48782 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:20:55,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 12:20:55,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 12:20:55,376 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da8000c489abc2bfe7d/log.txt.gz 2026-04-03 12:20:55,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 12:20:55,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 12:20:55,858 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da8000c489abc2bfe7d/lava-logs.yaml 2026-04-03 12:20:55,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 12:20:56,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 12:20:56,023 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf7da8000c489abc2bfe7d/lava_callback.json.gz 2026-04-03 12:20:56,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:20:56,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf7da8000c489abc2bfe7d HTTP/1.1" 200 None 2026-04-03 12:20:56,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:20:56,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf77d1000c489abc2bf955 HTTP/1.1" 200 None 2026-04-03 12:20:56,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:20:56,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf7da8000c489abc2bfe7d HTTP/1.1" 200 None 2026-04-03 12:20:56,096 - lava_callback - INFO - Completed processing callback for node 69cf7da8000c489abc2bfe7d INFO: 172.21.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:21:22,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:21:22,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45658 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:38:32,473 - lava_callback - INFO - Received callback for node 69cd11bf74bda3ec93a426a9 INFO: 172.21.0.1:52982 - "POST /node/69cd11bf74bda3ec93a426a9 HTTP/1.0" 202 Accepted 2026-04-03 12:38:32,479 - lava_callback - INFO - Starting async job submit for node 69cd11bf74bda3ec93a426a9 2026-04-03 12:38:32,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:38:32,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd11bf74bda3ec93a426a9 HTTP/1.1" 200 None 2026-04-03 12:38:32,496 - lava_callback - DEBUG - Processing node 69cd11bf74bda3ec93a426a9: baseline-arm-baylibre 2026-04-03 12:38:32,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd11bf74bda3ec93a426a9 2026-04-03 12:38:32,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:38:32,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd11bf74bda3ec93a426a9 HTTP/1.1" 200 None 2026-04-03 12:38:32,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:38:32,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 12:38:32,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:38:32,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd11bf74bda3ec93a426a9 HTTP/1.1" 200 None 2026-04-03 12:38:32,555 - lava_callback - ERROR - Error processing callback for node 69cd11bf74bda3ec93a426a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53928 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:39:12,337 - lava_callback - INFO - Received callback for node 69cd11ec74bda3ec93a426b7 INFO: 172.21.0.1:46186 - "POST /node/69cd11ec74bda3ec93a426b7 HTTP/1.0" 202 Accepted 2026-04-03 12:39:12,342 - lava_callback - INFO - Starting async job submit for node 69cd11ec74bda3ec93a426b7 2026-04-03 12:39:12,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:39:12,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd11ec74bda3ec93a426b7 HTTP/1.1" 200 None 2026-04-03 12:39:12,358 - lava_callback - DEBUG - Processing node 69cd11ec74bda3ec93a426b7: baseline-arm-baylibre 2026-04-03 12:39:12,358 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd11ec74bda3ec93a426b7 2026-04-03 12:39:12,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:39:12,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd11ec74bda3ec93a426b7 HTTP/1.1" 200 None 2026-04-03 12:39:12,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:39:12,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 12:39:12,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:39:12,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd11ec74bda3ec93a426b7 HTTP/1.1" 200 None 2026-04-03 12:39:12,434 - lava_callback - ERROR - Error processing callback for node 69cd11ec74bda3ec93a426b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36446 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:41:42,856 - lava_callback - INFO - Received callback for node 69cd127a74bda3ec93a426db INFO: 172.21.0.1:47526 - "POST /node/69cd127a74bda3ec93a426db HTTP/1.0" 202 Accepted 2026-04-03 12:41:42,862 - lava_callback - INFO - Starting async job submit for node 69cd127a74bda3ec93a426db 2026-04-03 12:41:42,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:41:42,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd127a74bda3ec93a426db HTTP/1.1" 200 None 2026-04-03 12:41:42,885 - lava_callback - DEBUG - Processing node 69cd127a74bda3ec93a426db: baseline-arm-clabbe 2026-04-03 12:41:42,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd127a74bda3ec93a426db 2026-04-03 12:41:42,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:41:42,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd127a74bda3ec93a426db HTTP/1.1" 200 None 2026-04-03 12:41:42,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:41:42,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 12:41:42,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:41:42,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd127a74bda3ec93a426db HTTP/1.1" 200 None 2026-04-03 12:41:42,958 - lava_callback - ERROR - Error processing callback for node 69cd127a74bda3ec93a426db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54086 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:42:22,829 - lava_callback - INFO - Received callback for node 69cd12a374bda3ec93a426e5 INFO: 172.21.0.1:60292 - "POST /node/69cd12a374bda3ec93a426e5 HTTP/1.0" 202 Accepted 2026-04-03 12:42:22,834 - lava_callback - INFO - Starting async job submit for node 69cd12a374bda3ec93a426e5 2026-04-03 12:42:22,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:42:22,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd12a374bda3ec93a426e5 HTTP/1.1" 200 None 2026-04-03 12:42:22,850 - lava_callback - DEBUG - Processing node 69cd12a374bda3ec93a426e5: baseline-arm-clabbe 2026-04-03 12:42:22,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd12a374bda3ec93a426e5 2026-04-03 12:42:22,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:42:22,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd12a374bda3ec93a426e5 HTTP/1.1" 200 None 2026-04-03 12:42:22,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:42:22,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 12:42:22,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:42:22,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd12a374bda3ec93a426e5 HTTP/1.1" 200 None 2026-04-03 12:42:22,908 - lava_callback - ERROR - Error processing callback for node 69cd12a374bda3ec93a426e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38842 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:43:02,868 - lava_callback - INFO - Received callback for node 69cd12cb74bda3ec93a426f2 INFO: 172.21.0.1:55502 - "POST /node/69cd12cb74bda3ec93a426f2 HTTP/1.0" 202 Accepted 2026-04-03 12:43:02,873 - lava_callback - INFO - Starting async job submit for node 69cd12cb74bda3ec93a426f2 2026-04-03 12:43:02,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:43:02,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd12cb74bda3ec93a426f2 HTTP/1.1" 200 None 2026-04-03 12:43:02,889 - lava_callback - DEBUG - Processing node 69cd12cb74bda3ec93a426f2: baseline-arm-clabbe 2026-04-03 12:43:02,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd12cb74bda3ec93a426f2 2026-04-03 12:43:02,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:43:02,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd12cb74bda3ec93a426f2 HTTP/1.1" 200 None 2026-04-03 12:43:02,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:43:02,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 12:43:02,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:43:02,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd12cb74bda3ec93a426f2 HTTP/1.1" 200 None 2026-04-03 12:43:02,951 - lava_callback - ERROR - Error processing callback for node 69cd12cb74bda3ec93a426f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46780 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:43:42,908 - lava_callback - INFO - Received callback for node 69cd12f574bda3ec93a426ff INFO: 172.21.0.1:46500 - "POST /node/69cd12f574bda3ec93a426ff HTTP/1.0" 202 Accepted 2026-04-03 12:43:42,913 - lava_callback - INFO - Starting async job submit for node 69cd12f574bda3ec93a426ff 2026-04-03 12:43:42,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:43:42,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd12f574bda3ec93a426ff HTTP/1.1" 200 None 2026-04-03 12:43:42,929 - lava_callback - DEBUG - Processing node 69cd12f574bda3ec93a426ff: baseline-arm-clabbe 2026-04-03 12:43:42,930 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd12f574bda3ec93a426ff 2026-04-03 12:43:42,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:43:42,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd12f574bda3ec93a426ff HTTP/1.1" 200 None 2026-04-03 12:43:42,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:43:42,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 12:43:42,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:43:42,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd12f574bda3ec93a426ff HTTP/1.1" 200 None 2026-04-03 12:43:42,982 - lava_callback - ERROR - Error processing callback for node 69cd12f574bda3ec93a426ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53426 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:44:22,958 - lava_callback - INFO - Received callback for node 69cd131e74bda3ec93a4270c INFO: 172.21.0.1:45800 - "POST /node/69cd131e74bda3ec93a4270c HTTP/1.0" 202 Accepted 2026-04-03 12:44:22,964 - lava_callback - INFO - Starting async job submit for node 69cd131e74bda3ec93a4270c 2026-04-03 12:44:22,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:44:22,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd131e74bda3ec93a4270c HTTP/1.1" 200 None 2026-04-03 12:44:22,982 - lava_callback - DEBUG - Processing node 69cd131e74bda3ec93a4270c: baseline-arm-clabbe 2026-04-03 12:44:22,982 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd131e74bda3ec93a4270c 2026-04-03 12:44:22,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:44:22,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd131e74bda3ec93a4270c HTTP/1.1" 200 None 2026-04-03 12:44:22,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:44:23,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 12:44:23,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:44:23,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd131e74bda3ec93a4270c HTTP/1.1" 200 None 2026-04-03 12:44:23,063 - lava_callback - ERROR - Error processing callback for node 69cd131e74bda3ec93a4270c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59440 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:45:02,931 - lava_callback - INFO - Received callback for node 69cd134774bda3ec93a42718 INFO: 172.21.0.1:47756 - "POST /node/69cd134774bda3ec93a42718 HTTP/1.0" 202 Accepted 2026-04-03 12:45:02,936 - lava_callback - INFO - Starting async job submit for node 69cd134774bda3ec93a42718 2026-04-03 12:45:02,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:45:02,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd134774bda3ec93a42718 HTTP/1.1" 200 None 2026-04-03 12:45:02,952 - lava_callback - DEBUG - Processing node 69cd134774bda3ec93a42718: baseline-arm-clabbe 2026-04-03 12:45:02,953 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd134774bda3ec93a42718 2026-04-03 12:45:02,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:45:02,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd134774bda3ec93a42718 HTTP/1.1" 200 None 2026-04-03 12:45:02,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:45:02,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 12:45:02,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:45:03,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd134774bda3ec93a42718 HTTP/1.1" 200 None 2026-04-03 12:45:03,010 - lava_callback - ERROR - Error processing callback for node 69cd134774bda3ec93a42718: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55350 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:45:32,793 - lava_callback - INFO - Received callback for node 69cd136a74bda3ec93a42723 INFO: 172.21.0.1:49330 - "POST /node/69cd136a74bda3ec93a42723 HTTP/1.0" 202 Accepted 2026-04-03 12:45:32,800 - lava_callback - INFO - Starting async job submit for node 69cd136a74bda3ec93a42723 2026-04-03 12:45:32,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:45:32,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd136a74bda3ec93a42723 HTTP/1.1" 200 None 2026-04-03 12:45:32,816 - lava_callback - DEBUG - Processing node 69cd136a74bda3ec93a42723: baseline-arm-baylibre 2026-04-03 12:45:32,816 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd136a74bda3ec93a42723 2026-04-03 12:45:32,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:45:32,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd136a74bda3ec93a42723 HTTP/1.1" 200 None 2026-04-03 12:45:32,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:45:32,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:45:32,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:45:32,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd136a74bda3ec93a42723 HTTP/1.1" 200 None 2026-04-03 12:45:32,873 - lava_callback - ERROR - Error processing callback for node 69cd136a74bda3ec93a42723: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39008 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:46:32,810 - lava_callback - INFO - Received callback for node 69cd139374bda3ec93a42730 INFO: 172.21.0.1:45878 - "POST /node/69cd139374bda3ec93a42730 HTTP/1.0" 202 Accepted 2026-04-03 12:46:32,816 - lava_callback - INFO - Starting async job submit for node 69cd139374bda3ec93a42730 2026-04-03 12:46:32,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:46:32,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd139374bda3ec93a42730 HTTP/1.1" 200 None 2026-04-03 12:46:32,831 - lava_callback - DEBUG - Processing node 69cd139374bda3ec93a42730: baseline-arm-baylibre 2026-04-03 12:46:32,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd139374bda3ec93a42730 2026-04-03 12:46:32,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:46:32,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd139374bda3ec93a42730 HTTP/1.1" 200 None 2026-04-03 12:46:32,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:46:32,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:46:32,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:46:32,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd139374bda3ec93a42730 HTTP/1.1" 200 None 2026-04-03 12:46:32,882 - lava_callback - ERROR - Error processing callback for node 69cd139374bda3ec93a42730: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49010 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:47:12,908 - lava_callback - INFO - Received callback for node 69cd13bc74bda3ec93a42740 INFO: 172.21.0.1:50010 - "POST /node/69cd13bc74bda3ec93a42740 HTTP/1.0" 202 Accepted 2026-04-03 12:47:12,913 - lava_callback - INFO - Starting async job submit for node 69cd13bc74bda3ec93a42740 2026-04-03 12:47:12,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:47:12,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd13bc74bda3ec93a42740 HTTP/1.1" 200 None 2026-04-03 12:47:12,936 - lava_callback - DEBUG - Processing node 69cd13bc74bda3ec93a42740: baseline-arm-baylibre 2026-04-03 12:47:12,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd13bc74bda3ec93a42740 2026-04-03 12:47:12,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:47:12,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd13bc74bda3ec93a42740 HTTP/1.1" 200 None 2026-04-03 12:47:12,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:47:12,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:47:12,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:47:12,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd13bc74bda3ec93a42740 HTTP/1.1" 200 None 2026-04-03 12:47:12,993 - lava_callback - ERROR - Error processing callback for node 69cd13bc74bda3ec93a42740: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53966 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:47:52,856 - lava_callback - INFO - Received callback for node 69cd13e574bda3ec93a4274f INFO: 172.21.0.1:39536 - "POST /node/69cd13e574bda3ec93a4274f HTTP/1.0" 202 Accepted 2026-04-03 12:47:52,862 - lava_callback - INFO - Starting async job submit for node 69cd13e574bda3ec93a4274f 2026-04-03 12:47:52,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:47:52,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd13e574bda3ec93a4274f HTTP/1.1" 200 None 2026-04-03 12:47:52,878 - lava_callback - DEBUG - Processing node 69cd13e574bda3ec93a4274f: baseline-arm-baylibre 2026-04-03 12:47:52,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd13e574bda3ec93a4274f 2026-04-03 12:47:52,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:47:52,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd13e574bda3ec93a4274f HTTP/1.1" 200 None 2026-04-03 12:47:52,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:47:52,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:47:52,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:47:52,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd13e574bda3ec93a4274f HTTP/1.1" 200 None 2026-04-03 12:47:52,936 - lava_callback - ERROR - Error processing callback for node 69cd13e574bda3ec93a4274f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41026 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:48:32,981 - lava_callback - INFO - Received callback for node 69cd140c74bda3ec93a4275d INFO: 172.21.0.1:60130 - "POST /node/69cd140c74bda3ec93a4275d HTTP/1.0" 202 Accepted 2026-04-03 12:48:32,986 - lava_callback - INFO - Starting async job submit for node 69cd140c74bda3ec93a4275d 2026-04-03 12:48:32,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:48:33,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd140c74bda3ec93a4275d HTTP/1.1" 200 None 2026-04-03 12:48:33,002 - lava_callback - DEBUG - Processing node 69cd140c74bda3ec93a4275d: baseline-arm-baylibre 2026-04-03 12:48:33,002 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd140c74bda3ec93a4275d 2026-04-03 12:48:33,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:48:33,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd140c74bda3ec93a4275d HTTP/1.1" 200 None 2026-04-03 12:48:33,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:48:33,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:48:33,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:48:33,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd140c74bda3ec93a4275d HTTP/1.1" 200 None 2026-04-03 12:48:33,054 - lava_callback - ERROR - Error processing callback for node 69cd140c74bda3ec93a4275d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35418 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:49:12,951 - lava_callback - INFO - Received callback for node 69cd143574bda3ec93a4276d INFO: 172.21.0.1:42462 - "POST /node/69cd143574bda3ec93a4276d HTTP/1.0" 202 Accepted 2026-04-03 12:49:12,956 - lava_callback - INFO - Starting async job submit for node 69cd143574bda3ec93a4276d 2026-04-03 12:49:12,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:49:12,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd143574bda3ec93a4276d HTTP/1.1" 200 None 2026-04-03 12:49:12,973 - lava_callback - DEBUG - Processing node 69cd143574bda3ec93a4276d: baseline-arm-baylibre 2026-04-03 12:49:12,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd143574bda3ec93a4276d 2026-04-03 12:49:12,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:49:12,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd143574bda3ec93a4276d HTTP/1.1" 200 None 2026-04-03 12:49:12,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:49:13,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:49:13,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:49:13,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd143574bda3ec93a4276d HTTP/1.1" 200 None 2026-04-03 12:49:13,035 - lava_callback - ERROR - Error processing callback for node 69cd143574bda3ec93a4276d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39228 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:51:03,271 - lava_callback - INFO - Received callback for node 69cd14a474bda3ec93a4278b INFO: 172.21.0.1:34760 - "POST /node/69cd14a474bda3ec93a4278b HTTP/1.0" 202 Accepted 2026-04-03 12:51:03,276 - lava_callback - INFO - Starting async job submit for node 69cd14a474bda3ec93a4278b 2026-04-03 12:51:03,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:51:03,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd14a474bda3ec93a4278b HTTP/1.1" 200 None 2026-04-03 12:51:03,291 - lava_callback - DEBUG - Processing node 69cd14a474bda3ec93a4278b: baseline-arm-clabbe 2026-04-03 12:51:03,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd14a474bda3ec93a4278b 2026-04-03 12:51:03,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:51:03,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd14a474bda3ec93a4278b HTTP/1.1" 200 None 2026-04-03 12:51:03,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:51:03,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:51:03,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:51:03,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd14a474bda3ec93a4278b HTTP/1.1" 200 None 2026-04-03 12:51:03,351 - lava_callback - ERROR - Error processing callback for node 69cd14a474bda3ec93a4278b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54456 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:52:23,302 - lava_callback - INFO - Received callback for node 69cd14f374bda3ec93a427b2 INFO: 172.21.0.1:53300 - "POST /node/69cd14f374bda3ec93a427b2 HTTP/1.0" 202 Accepted 2026-04-03 12:52:23,308 - lava_callback - INFO - Starting async job submit for node 69cd14f374bda3ec93a427b2 2026-04-03 12:52:23,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:52:23,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd14f374bda3ec93a427b2 HTTP/1.1" 200 None 2026-04-03 12:52:23,327 - lava_callback - DEBUG - Processing node 69cd14f374bda3ec93a427b2: baseline-arm-clabbe 2026-04-03 12:52:23,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd14f374bda3ec93a427b2 2026-04-03 12:52:23,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:52:23,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd14f374bda3ec93a427b2 HTTP/1.1" 200 None 2026-04-03 12:52:23,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:52:23,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:52:23,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:52:23,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd14f374bda3ec93a427b2 HTTP/1.1" 200 None 2026-04-03 12:52:23,389 - lava_callback - ERROR - Error processing callback for node 69cd14f374bda3ec93a427b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36070 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:53:03,317 - lava_callback - INFO - Received callback for node 69cd151d74bda3ec93a427ba INFO: 172.21.0.1:44146 - "POST /node/69cd151d74bda3ec93a427ba HTTP/1.0" 202 Accepted 2026-04-03 12:53:03,322 - lava_callback - INFO - Starting async job submit for node 69cd151d74bda3ec93a427ba 2026-04-03 12:53:03,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:53:03,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd151d74bda3ec93a427ba HTTP/1.1" 200 None 2026-04-03 12:53:03,339 - lava_callback - DEBUG - Processing node 69cd151d74bda3ec93a427ba: baseline-arm-clabbe 2026-04-03 12:53:03,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd151d74bda3ec93a427ba 2026-04-03 12:53:03,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:53:03,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd151d74bda3ec93a427ba HTTP/1.1" 200 None 2026-04-03 12:53:03,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:53:03,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:53:03,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:53:03,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd151d74bda3ec93a427ba HTTP/1.1" 200 None 2026-04-03 12:53:03,390 - lava_callback - ERROR - Error processing callback for node 69cd151d74bda3ec93a427ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36302 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:53:43,390 - lava_callback - INFO - Received callback for node 69cd154674bda3ec93a427c8 INFO: 172.21.0.1:50636 - "POST /node/69cd154674bda3ec93a427c8 HTTP/1.0" 202 Accepted 2026-04-03 12:53:43,399 - lava_callback - INFO - Starting async job submit for node 69cd154674bda3ec93a427c8 2026-04-03 12:53:43,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:53:43,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd154674bda3ec93a427c8 HTTP/1.1" 200 None 2026-04-03 12:53:43,417 - lava_callback - DEBUG - Processing node 69cd154674bda3ec93a427c8: baseline-arm-clabbe 2026-04-03 12:53:43,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd154674bda3ec93a427c8 2026-04-03 12:53:43,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:53:43,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd154674bda3ec93a427c8 HTTP/1.1" 200 None 2026-04-03 12:53:43,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:53:43,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:53:43,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:53:43,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd154674bda3ec93a427c8 HTTP/1.1" 200 None 2026-04-03 12:53:43,480 - lava_callback - ERROR - Error processing callback for node 69cd154674bda3ec93a427c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44864 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:54:23,381 - lava_callback - INFO - Received callback for node 69cd156f74bda3ec93a427db INFO: 172.21.0.1:46322 - "POST /node/69cd156f74bda3ec93a427db HTTP/1.0" 202 Accepted 2026-04-03 12:54:23,386 - lava_callback - INFO - Starting async job submit for node 69cd156f74bda3ec93a427db 2026-04-03 12:54:23,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:54:23,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd156f74bda3ec93a427db HTTP/1.1" 200 None 2026-04-03 12:54:23,412 - lava_callback - DEBUG - Processing node 69cd156f74bda3ec93a427db: baseline-arm-clabbe 2026-04-03 12:54:23,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd156f74bda3ec93a427db 2026-04-03 12:54:23,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:54:23,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd156f74bda3ec93a427db HTTP/1.1" 200 None 2026-04-03 12:54:23,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:54:23,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:54:23,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:54:23,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd156f74bda3ec93a427db HTTP/1.1" 200 None 2026-04-03 12:54:23,465 - lava_callback - ERROR - Error processing callback for node 69cd156f74bda3ec93a427db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53996 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:55:03,380 - lava_callback - INFO - Received callback for node 69cd159874bda3ec93a427ea INFO: 172.21.0.1:59580 - "POST /node/69cd159874bda3ec93a427ea HTTP/1.0" 202 Accepted 2026-04-03 12:55:03,385 - lava_callback - INFO - Starting async job submit for node 69cd159874bda3ec93a427ea 2026-04-03 12:55:03,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:55:03,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd159874bda3ec93a427ea HTTP/1.1" 200 None 2026-04-03 12:55:03,402 - lava_callback - DEBUG - Processing node 69cd159874bda3ec93a427ea: baseline-arm-clabbe 2026-04-03 12:55:03,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd159874bda3ec93a427ea 2026-04-03 12:55:03,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:55:03,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd159874bda3ec93a427ea HTTP/1.1" 200 None 2026-04-03 12:55:03,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:55:03,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:55:03,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:55:03,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd159874bda3ec93a427ea HTTP/1.1" 200 None 2026-04-03 12:55:03,452 - lava_callback - ERROR - Error processing callback for node 69cd159874bda3ec93a427ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45762 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:55:43,421 - lava_callback - INFO - Received callback for node 69cd15c174bda3ec93a427f8 INFO: 172.21.0.1:55350 - "POST /node/69cd15c174bda3ec93a427f8 HTTP/1.0" 202 Accepted 2026-04-03 12:55:43,427 - lava_callback - INFO - Starting async job submit for node 69cd15c174bda3ec93a427f8 2026-04-03 12:55:43,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:55:43,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd15c174bda3ec93a427f8 HTTP/1.1" 200 None 2026-04-03 12:55:43,443 - lava_callback - DEBUG - Processing node 69cd15c174bda3ec93a427f8: baseline-arm-clabbe 2026-04-03 12:55:43,443 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd15c174bda3ec93a427f8 2026-04-03 12:55:43,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:55:43,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd15c174bda3ec93a427f8 HTTP/1.1" 200 None 2026-04-03 12:55:43,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:55:43,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:55:43,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:55:43,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd15c174bda3ec93a427f8 HTTP/1.1" 200 None 2026-04-03 12:55:43,507 - lava_callback - ERROR - Error processing callback for node 69cd15c174bda3ec93a427f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55174 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:56:23,494 - lava_callback - INFO - Received callback for node 69cd15ec74bda3ec93a42807 INFO: 172.21.0.1:58166 - "POST /node/69cd15ec74bda3ec93a42807 HTTP/1.0" 202 Accepted 2026-04-03 12:56:23,500 - lava_callback - INFO - Starting async job submit for node 69cd15ec74bda3ec93a42807 2026-04-03 12:56:23,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:56:23,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd15ec74bda3ec93a42807 HTTP/1.1" 200 None 2026-04-03 12:56:23,522 - lava_callback - DEBUG - Processing node 69cd15ec74bda3ec93a42807: baseline-arm-clabbe 2026-04-03 12:56:23,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd15ec74bda3ec93a42807 2026-04-03 12:56:23,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:56:23,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd15ec74bda3ec93a42807 HTTP/1.1" 200 None 2026-04-03 12:56:23,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:56:23,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:56:23,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:56:23,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd15ec74bda3ec93a42807 HTTP/1.1" 200 None 2026-04-03 12:56:23,578 - lava_callback - ERROR - Error processing callback for node 69cd15ec74bda3ec93a42807: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36128 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:57:03,409 - lava_callback - INFO - Received callback for node 69cd161874bda3ec93a42819 INFO: 172.21.0.1:57640 - "POST /node/69cd161874bda3ec93a42819 HTTP/1.0" 202 Accepted 2026-04-03 12:57:03,414 - lava_callback - INFO - Starting async job submit for node 69cd161874bda3ec93a42819 2026-04-03 12:57:03,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:57:03,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd161874bda3ec93a42819 HTTP/1.1" 200 None 2026-04-03 12:57:03,430 - lava_callback - DEBUG - Processing node 69cd161874bda3ec93a42819: baseline-arm-clabbe 2026-04-03 12:57:03,430 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd161874bda3ec93a42819 2026-04-03 12:57:03,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:57:03,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd161874bda3ec93a42819 HTTP/1.1" 200 None 2026-04-03 12:57:03,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:57:03,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:57:03,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:57:03,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd161874bda3ec93a42819 HTTP/1.1" 200 None 2026-04-03 12:57:03,490 - lava_callback - ERROR - Error processing callback for node 69cd161874bda3ec93a42819: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52628 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:57:43,517 - lava_callback - INFO - Received callback for node 69cd164074bda3ec93a42827 INFO: 172.21.0.1:39996 - "POST /node/69cd164074bda3ec93a42827 HTTP/1.0" 202 Accepted 2026-04-03 12:57:43,522 - lava_callback - INFO - Starting async job submit for node 69cd164074bda3ec93a42827 2026-04-03 12:57:43,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:57:43,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd164074bda3ec93a42827 HTTP/1.1" 200 None 2026-04-03 12:57:43,539 - lava_callback - DEBUG - Processing node 69cd164074bda3ec93a42827: baseline-arm-clabbe 2026-04-03 12:57:43,539 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd164074bda3ec93a42827 2026-04-03 12:57:43,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:57:43,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd164074bda3ec93a42827 HTTP/1.1" 200 None 2026-04-03 12:57:43,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:57:43,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:57:43,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:57:43,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd164074bda3ec93a42827 HTTP/1.1" 200 None 2026-04-03 12:57:43,594 - lava_callback - ERROR - Error processing callback for node 69cd164074bda3ec93a42827: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32920 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:59:03,628 - lava_callback - INFO - Received callback for node 69cd169074bda3ec93a42842 INFO: 172.21.0.1:41822 - "POST /node/69cd169074bda3ec93a42842 HTTP/1.0" 202 Accepted 2026-04-03 12:59:03,635 - lava_callback - INFO - Starting async job submit for node 69cd169074bda3ec93a42842 2026-04-03 12:59:03,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:59:03,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd169074bda3ec93a42842 HTTP/1.1" 200 None 2026-04-03 12:59:03,652 - lava_callback - DEBUG - Processing node 69cd169074bda3ec93a42842: baseline-arm-clabbe 2026-04-03 12:59:03,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd169074bda3ec93a42842 2026-04-03 12:59:03,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:59:03,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd169074bda3ec93a42842 HTTP/1.1" 200 None 2026-04-03 12:59:03,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:59:03,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:59:03,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:59:03,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd169074bda3ec93a42842 HTTP/1.1" 200 None 2026-04-03 12:59:03,711 - lava_callback - ERROR - Error processing callback for node 69cd169074bda3ec93a42842: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET / HTTP/1.0" 200 OK 2026-04-03 12:59:43,631 - lava_callback - INFO - Received callback for node 69cd16b874bda3ec93a42851 INFO: 172.21.0.1:55794 - "POST /node/69cd16b874bda3ec93a42851 HTTP/1.0" 202 Accepted 2026-04-03 12:59:43,637 - lava_callback - INFO - Starting async job submit for node 69cd16b874bda3ec93a42851 2026-04-03 12:59:43,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:59:43,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd16b874bda3ec93a42851 HTTP/1.1" 200 None 2026-04-03 12:59:43,652 - lava_callback - DEBUG - Processing node 69cd16b874bda3ec93a42851: baseline-arm-clabbe 2026-04-03 12:59:43,652 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd16b874bda3ec93a42851 2026-04-03 12:59:43,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:59:43,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd16b874bda3ec93a42851 HTTP/1.1" 200 None 2026-04-03 12:59:43,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:59:43,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 12:59:43,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 12:59:43,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd16b874bda3ec93a42851 HTTP/1.1" 200 None 2026-04-03 12:59:43,708 - lava_callback - ERROR - Error processing callback for node 69cd16b874bda3ec93a42851: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52754 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:00:23,702 - lava_callback - INFO - Received callback for node 69cd16e374bda3ec93a42860 INFO: 172.21.0.1:50152 - "POST /node/69cd16e374bda3ec93a42860 HTTP/1.0" 202 Accepted 2026-04-03 13:00:23,708 - lava_callback - INFO - Starting async job submit for node 69cd16e374bda3ec93a42860 2026-04-03 13:00:23,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:00:23,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd16e374bda3ec93a42860 HTTP/1.1" 200 None 2026-04-03 13:00:23,723 - lava_callback - DEBUG - Processing node 69cd16e374bda3ec93a42860: baseline-arm-clabbe 2026-04-03 13:00:23,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd16e374bda3ec93a42860 2026-04-03 13:00:23,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:00:23,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd16e374bda3ec93a42860 HTTP/1.1" 200 None 2026-04-03 13:00:23,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:00:23,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 13:00:23,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:00:23,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd16e374bda3ec93a42860 HTTP/1.1" 200 None 2026-04-03 13:00:23,782 - lava_callback - ERROR - Error processing callback for node 69cd16e374bda3ec93a42860: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40888 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:01:03,657 - lava_callback - INFO - Received callback for node 69cd170b74bda3ec93a42870 INFO: 172.21.0.1:54826 - "POST /node/69cd170b74bda3ec93a42870 HTTP/1.0" 202 Accepted 2026-04-03 13:01:03,663 - lava_callback - INFO - Starting async job submit for node 69cd170b74bda3ec93a42870 2026-04-03 13:01:03,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:01:03,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd170b74bda3ec93a42870 HTTP/1.1" 200 None 2026-04-03 13:01:03,682 - lava_callback - DEBUG - Processing node 69cd170b74bda3ec93a42870: baseline-arm-clabbe 2026-04-03 13:01:03,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd170b74bda3ec93a42870 2026-04-03 13:01:03,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:01:03,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd170b74bda3ec93a42870 HTTP/1.1" 200 None 2026-04-03 13:01:03,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:01:03,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 13:01:03,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:01:03,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd170b74bda3ec93a42870 HTTP/1.1" 200 None 2026-04-03 13:01:03,734 - lava_callback - ERROR - Error processing callback for node 69cd170b74bda3ec93a42870: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37072 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:02:03,693 - lava_callback - INFO - Received callback for node 69cd173674bda3ec93a4287d INFO: 172.21.0.1:57330 - "POST /node/69cd173674bda3ec93a4287d HTTP/1.0" 202 Accepted 2026-04-03 13:02:03,699 - lava_callback - INFO - Starting async job submit for node 69cd173674bda3ec93a4287d 2026-04-03 13:02:03,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:02:03,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd173674bda3ec93a4287d HTTP/1.1" 200 None 2026-04-03 13:02:03,715 - lava_callback - DEBUG - Processing node 69cd173674bda3ec93a4287d: baseline-arm-clabbe 2026-04-03 13:02:03,716 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd173674bda3ec93a4287d 2026-04-03 13:02:03,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:02:03,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd173674bda3ec93a4287d HTTP/1.1" 200 None 2026-04-03 13:02:03,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:02:03,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 13:02:03,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:02:03,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd173674bda3ec93a4287d HTTP/1.1" 200 None 2026-04-03 13:02:03,771 - lava_callback - ERROR - Error processing callback for node 69cd173674bda3ec93a4287d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42250 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:18:14,617 - lava_callback - INFO - Received callback for node 69cd1b0a74bda3ec93a4294a INFO: 172.21.0.1:57138 - "POST /node/69cd1b0a74bda3ec93a4294a HTTP/1.0" 202 Accepted 2026-04-03 13:18:14,624 - lava_callback - INFO - Starting async job submit for node 69cd1b0a74bda3ec93a4294a 2026-04-03 13:18:14,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:18:14,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1b0a74bda3ec93a4294a HTTP/1.1" 200 None 2026-04-03 13:18:14,639 - lava_callback - DEBUG - Processing node 69cd1b0a74bda3ec93a4294a: baseline-arm64-baylibre 2026-04-03 13:18:14,640 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1b0a74bda3ec93a4294a 2026-04-03 13:18:14,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:18:14,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1b0a74bda3ec93a4294a HTTP/1.1" 200 None 2026-04-03 13:18:14,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:18:14,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:18:14,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:18:14,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1b0a74bda3ec93a4294a HTTP/1.1" 200 None 2026-04-03 13:18:14,705 - lava_callback - ERROR - Error processing callback for node 69cd1b0a74bda3ec93a4294a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41260 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:20:14,692 - lava_callback - INFO - Received callback for node 69cd1b8274bda3ec93a4296f INFO: 172.21.0.1:59266 - "POST /node/69cd1b8274bda3ec93a4296f HTTP/1.0" 202 Accepted 2026-04-03 13:20:14,698 - lava_callback - INFO - Starting async job submit for node 69cd1b8274bda3ec93a4296f 2026-04-03 13:20:14,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:20:14,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1b8274bda3ec93a4296f HTTP/1.1" 200 None 2026-04-03 13:20:14,714 - lava_callback - DEBUG - Processing node 69cd1b8274bda3ec93a4296f: baseline-arm64-baylibre 2026-04-03 13:20:14,714 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1b8274bda3ec93a4296f 2026-04-03 13:20:14,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:20:14,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1b8274bda3ec93a4296f HTTP/1.1" 200 None 2026-04-03 13:20:14,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:20:14,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:20:14,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:20:14,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1b8274bda3ec93a4296f HTTP/1.1" 200 None 2026-04-03 13:20:14,772 - lava_callback - ERROR - Error processing callback for node 69cd1b8274bda3ec93a4296f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51752 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:21:34,769 - lava_callback - INFO - Received callback for node 69cd1bda74bda3ec93a4298b INFO: 172.21.0.1:37506 - "POST /node/69cd1bda74bda3ec93a4298b HTTP/1.0" 202 Accepted 2026-04-03 13:21:34,774 - lava_callback - INFO - Starting async job submit for node 69cd1bda74bda3ec93a4298b 2026-04-03 13:21:34,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:21:34,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1bda74bda3ec93a4298b HTTP/1.1" 200 None 2026-04-03 13:21:34,789 - lava_callback - DEBUG - Processing node 69cd1bda74bda3ec93a4298b: baseline-arm64-baylibre 2026-04-03 13:21:34,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1bda74bda3ec93a4298b 2026-04-03 13:21:34,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:21:34,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1bda74bda3ec93a4298b HTTP/1.1" 200 None 2026-04-03 13:21:34,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:21:34,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:21:34,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:21:34,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1bda74bda3ec93a4298b HTTP/1.1" 200 None 2026-04-03 13:21:34,855 - lava_callback - ERROR - Error processing callback for node 69cd1bda74bda3ec93a4298b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49300 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:23:14,868 - lava_callback - INFO - Received callback for node 69cd1c3874bda3ec93a429b8 INFO: 172.21.0.1:33504 - "POST /node/69cd1c3874bda3ec93a429b8 HTTP/1.0" 202 Accepted 2026-04-03 13:23:14,874 - lava_callback - INFO - Starting async job submit for node 69cd1c3874bda3ec93a429b8 2026-04-03 13:23:14,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:23:14,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1c3874bda3ec93a429b8 HTTP/1.1" 200 None 2026-04-03 13:23:14,890 - lava_callback - DEBUG - Processing node 69cd1c3874bda3ec93a429b8: baseline-arm64-baylibre 2026-04-03 13:23:14,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1c3874bda3ec93a429b8 2026-04-03 13:23:14,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:23:14,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1c3874bda3ec93a429b8 HTTP/1.1" 200 None 2026-04-03 13:23:14,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:23:14,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:23:14,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:23:14,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1c3874bda3ec93a429b8 HTTP/1.1" 200 None 2026-04-03 13:23:14,952 - lava_callback - ERROR - Error processing callback for node 69cd1c3874bda3ec93a429b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60424 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:24:34,942 - lava_callback - INFO - Received callback for node 69cd1c8f74bda3ec93a429d5 INFO: 172.21.0.1:49230 - "POST /node/69cd1c8f74bda3ec93a429d5 HTTP/1.0" 202 Accepted 2026-04-03 13:24:34,947 - lava_callback - INFO - Starting async job submit for node 69cd1c8f74bda3ec93a429d5 2026-04-03 13:24:34,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:24:34,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1c8f74bda3ec93a429d5 HTTP/1.1" 200 None 2026-04-03 13:24:34,963 - lava_callback - DEBUG - Processing node 69cd1c8f74bda3ec93a429d5: baseline-arm64-baylibre 2026-04-03 13:24:34,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1c8f74bda3ec93a429d5 2026-04-03 13:24:34,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:24:34,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1c8f74bda3ec93a429d5 HTTP/1.1" 200 None 2026-04-03 13:24:34,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:24:34,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:24:34,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:24:35,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1c8f74bda3ec93a429d5 HTTP/1.1" 200 None 2026-04-03 13:24:35,026 - lava_callback - ERROR - Error processing callback for node 69cd1c8f74bda3ec93a429d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35956 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:25:55,000 - lava_callback - INFO - Received callback for node 69cd1cdf74bda3ec93a429fd INFO: 172.21.0.1:45600 - "POST /node/69cd1cdf74bda3ec93a429fd HTTP/1.0" 202 Accepted 2026-04-03 13:25:55,007 - lava_callback - INFO - Starting async job submit for node 69cd1cdf74bda3ec93a429fd 2026-04-03 13:25:55,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:25:55,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1cdf74bda3ec93a429fd HTTP/1.1" 200 None 2026-04-03 13:25:55,026 - lava_callback - DEBUG - Processing node 69cd1cdf74bda3ec93a429fd: baseline-arm64-baylibre 2026-04-03 13:25:55,026 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1cdf74bda3ec93a429fd 2026-04-03 13:25:55,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:25:55,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1cdf74bda3ec93a429fd HTTP/1.1" 200 None 2026-04-03 13:25:55,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:25:55,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:25:55,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:25:55,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1cdf74bda3ec93a429fd HTTP/1.1" 200 None 2026-04-03 13:25:55,090 - lava_callback - ERROR - Error processing callback for node 69cd1cdf74bda3ec93a429fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48398 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:27:44,928 - lava_callback - INFO - Received callback for node 69cd1d3e74bda3ec93a42a28 INFO: 172.21.0.1:33760 - "POST /node/69cd1d3e74bda3ec93a42a28 HTTP/1.0" 202 Accepted 2026-04-03 13:27:44,933 - lava_callback - INFO - Starting async job submit for node 69cd1d3e74bda3ec93a42a28 2026-04-03 13:27:44,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:27:44,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1d3e74bda3ec93a42a28 HTTP/1.1" 200 None 2026-04-03 13:27:44,948 - lava_callback - DEBUG - Processing node 69cd1d3e74bda3ec93a42a28: baseline-arm64-clabbe 2026-04-03 13:27:44,949 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1d3e74bda3ec93a42a28 2026-04-03 13:27:44,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:27:44,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1d3e74bda3ec93a42a28 HTTP/1.1" 200 None 2026-04-03 13:27:44,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:27:44,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:27:44,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:27:45,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1d3e74bda3ec93a42a28 HTTP/1.1" 200 None 2026-04-03 13:27:45,008 - lava_callback - ERROR - Error processing callback for node 69cd1d3e74bda3ec93a42a28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34282 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:28:55,125 - lava_callback - INFO - Received callback for node 69cd1d8c74bda3ec93a42a4f INFO: 172.21.0.1:50270 - "POST /node/69cd1d8c74bda3ec93a42a4f HTTP/1.0" 202 Accepted 2026-04-03 13:28:55,131 - lava_callback - INFO - Starting async job submit for node 69cd1d8c74bda3ec93a42a4f 2026-04-03 13:28:55,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:28:55,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1d8c74bda3ec93a42a4f HTTP/1.1" 200 None 2026-04-03 13:28:55,146 - lava_callback - DEBUG - Processing node 69cd1d8c74bda3ec93a42a4f: baseline-arm64-baylibre 2026-04-03 13:28:55,146 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1d8c74bda3ec93a42a4f 2026-04-03 13:28:55,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:28:55,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1d8c74bda3ec93a42a4f HTTP/1.1" 200 None 2026-04-03 13:28:55,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:28:55,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:28:55,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:28:55,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1d8c74bda3ec93a42a4f HTTP/1.1" 200 None 2026-04-03 13:28:55,205 - lava_callback - ERROR - Error processing callback for node 69cd1d8c74bda3ec93a42a4f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48694 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:30:25,217 - lava_callback - INFO - Received callback for node 69cd1de974bda3ec93a42a78 INFO: 172.21.0.1:44096 - "POST /node/69cd1de974bda3ec93a42a78 HTTP/1.0" 202 Accepted 2026-04-03 13:30:25,222 - lava_callback - INFO - Starting async job submit for node 69cd1de974bda3ec93a42a78 2026-04-03 13:30:25,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:30:25,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1de974bda3ec93a42a78 HTTP/1.1" 200 None 2026-04-03 13:30:25,238 - lava_callback - DEBUG - Processing node 69cd1de974bda3ec93a42a78: baseline-arm64-clabbe 2026-04-03 13:30:25,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1de974bda3ec93a42a78 2026-04-03 13:30:25,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:30:25,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1de974bda3ec93a42a78 HTTP/1.1" 200 None 2026-04-03 13:30:25,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:30:25,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:30:25,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:30:25,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1de974bda3ec93a42a78 HTTP/1.1" 200 None 2026-04-03 13:30:25,291 - lava_callback - ERROR - Error processing callback for node 69cd1de974bda3ec93a42a78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59482 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:32:05,077 - lava_callback - INFO - Received callback for node 69cd1e4074bda3ec93a42aa0 INFO: 172.21.0.1:59932 - "POST /node/69cd1e4074bda3ec93a42aa0 HTTP/1.0" 202 Accepted 2026-04-03 13:32:05,083 - lava_callback - INFO - Starting async job submit for node 69cd1e4074bda3ec93a42aa0 2026-04-03 13:32:05,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:32:05,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1e4074bda3ec93a42aa0 HTTP/1.1" 200 None 2026-04-03 13:32:05,110 - lava_callback - DEBUG - Processing node 69cd1e4074bda3ec93a42aa0: baseline-arm64-clabbe 2026-04-03 13:32:05,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1e4074bda3ec93a42aa0 2026-04-03 13:32:05,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:32:05,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1e4074bda3ec93a42aa0 HTTP/1.1" 200 None 2026-04-03 13:32:05,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:32:05,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:32:05,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:32:05,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1e4074bda3ec93a42aa0 HTTP/1.1" 200 None 2026-04-03 13:32:05,162 - lava_callback - ERROR - Error processing callback for node 69cd1e4074bda3ec93a42aa0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38958 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 13:33:15,410 - lava_callback - INFO - Received callback for node 69cd1e8f74bda3ec93a42ac7 INFO: 172.21.0.1:38974 - "POST /node/69cd1e8f74bda3ec93a42ac7 HTTP/1.0" 202 Accepted 2026-04-03 13:33:15,416 - lava_callback - INFO - Starting async job submit for node 69cd1e8f74bda3ec93a42ac7 2026-04-03 13:33:15,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:33:15,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1e8f74bda3ec93a42ac7 HTTP/1.1" 200 None 2026-04-03 13:33:15,432 - lava_callback - DEBUG - Processing node 69cd1e8f74bda3ec93a42ac7: baseline-arm64-baylibre 2026-04-03 13:33:15,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1e8f74bda3ec93a42ac7 2026-04-03 13:33:15,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:33:15,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1e8f74bda3ec93a42ac7 HTTP/1.1" 200 None 2026-04-03 13:33:15,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:33:15,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:33:15,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:33:15,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1e8f74bda3ec93a42ac7 HTTP/1.1" 200 None 2026-04-03 13:33:15,495 - lava_callback - ERROR - Error processing callback for node 69cd1e8f74bda3ec93a42ac7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37370 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:34:45,177 - lava_callback - INFO - Received callback for node 69cd1eec74bda3ec93a42aef INFO: 172.21.0.1:54682 - "POST /node/69cd1eec74bda3ec93a42aef HTTP/1.0" 202 Accepted 2026-04-03 13:34:45,182 - lava_callback - INFO - Starting async job submit for node 69cd1eec74bda3ec93a42aef 2026-04-03 13:34:45,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:34:45,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1eec74bda3ec93a42aef HTTP/1.1" 200 None 2026-04-03 13:34:45,201 - lava_callback - DEBUG - Processing node 69cd1eec74bda3ec93a42aef: baseline-arm64-clabbe 2026-04-03 13:34:45,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1eec74bda3ec93a42aef 2026-04-03 13:34:45,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:34:45,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1eec74bda3ec93a42aef HTTP/1.1" 200 None 2026-04-03 13:34:45,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:34:45,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:34:45,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:34:45,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1eec74bda3ec93a42aef HTTP/1.1" 200 None 2026-04-03 13:34:45,263 - lava_callback - ERROR - Error processing callback for node 69cd1eec74bda3ec93a42aef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56858 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 13:36:15,555 - lava_callback - INFO - Received callback for node 69cd1f4274bda3ec93a42b0c INFO: 172.21.0.1:56874 - "POST /node/69cd1f4274bda3ec93a42b0c HTTP/1.0" 202 Accepted 2026-04-03 13:36:15,560 - lava_callback - INFO - Starting async job submit for node 69cd1f4274bda3ec93a42b0c 2026-04-03 13:36:15,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:36:15,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1f4274bda3ec93a42b0c HTTP/1.1" 200 None 2026-04-03 13:36:15,577 - lava_callback - DEBUG - Processing node 69cd1f4274bda3ec93a42b0c: baseline-arm64-baylibre 2026-04-03 13:36:15,577 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1f4274bda3ec93a42b0c 2026-04-03 13:36:15,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:36:15,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1f4274bda3ec93a42b0c HTTP/1.1" 200 None 2026-04-03 13:36:15,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:36:15,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:36:15,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:36:15,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1f4274bda3ec93a42b0c HTTP/1.1" 200 None 2026-04-03 13:36:15,646 - lava_callback - ERROR - Error processing callback for node 69cd1f4274bda3ec93a42b0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56828 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:37:45,246 - lava_callback - INFO - Received callback for node 69cd1fa074bda3ec93a42b2a INFO: 172.21.0.1:34010 - "POST /node/69cd1fa074bda3ec93a42b2a HTTP/1.0" 202 Accepted 2026-04-03 13:37:45,252 - lava_callback - INFO - Starting async job submit for node 69cd1fa074bda3ec93a42b2a 2026-04-03 13:37:45,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:37:45,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1fa074bda3ec93a42b2a HTTP/1.1" 200 None 2026-04-03 13:37:45,268 - lava_callback - DEBUG - Processing node 69cd1fa074bda3ec93a42b2a: baseline-arm64-clabbe 2026-04-03 13:37:45,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1fa074bda3ec93a42b2a 2026-04-03 13:37:45,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:37:45,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1fa074bda3ec93a42b2a HTTP/1.1" 200 None 2026-04-03 13:37:45,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:37:45,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:37:45,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:37:45,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1fa074bda3ec93a42b2a HTTP/1.1" 200 None 2026-04-03 13:37:45,339 - lava_callback - ERROR - Error processing callback for node 69cd1fa074bda3ec93a42b2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43692 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:39:25,380 - lava_callback - INFO - Received callback for node 69cd1ff774bda3ec93a42b45 INFO: 172.21.0.1:35332 - "POST /node/69cd1ff774bda3ec93a42b45 HTTP/1.0" 202 Accepted 2026-04-03 13:39:25,386 - lava_callback - INFO - Starting async job submit for node 69cd1ff774bda3ec93a42b45 2026-04-03 13:39:25,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:39:25,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1ff774bda3ec93a42b45 HTTP/1.1" 200 None 2026-04-03 13:39:25,402 - lava_callback - DEBUG - Processing node 69cd1ff774bda3ec93a42b45: baseline-arm64-clabbe 2026-04-03 13:39:25,402 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd1ff774bda3ec93a42b45 2026-04-03 13:39:25,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:39:25,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd1ff774bda3ec93a42b45 HTTP/1.1" 200 None 2026-04-03 13:39:25,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:39:25,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:39:25,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:39:25,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd1ff774bda3ec93a42b45 HTTP/1.1" 200 None 2026-04-03 13:39:25,454 - lava_callback - ERROR - Error processing callback for node 69cd1ff774bda3ec93a42b45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60456 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:40:45,454 - lava_callback - INFO - Received callback for node 69cd204d74bda3ec93a42b5a INFO: 172.21.0.1:42432 - "POST /node/69cd204d74bda3ec93a42b5a HTTP/1.0" 202 Accepted 2026-04-03 13:40:45,459 - lava_callback - INFO - Starting async job submit for node 69cd204d74bda3ec93a42b5a 2026-04-03 13:40:45,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:40:45,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd204d74bda3ec93a42b5a HTTP/1.1" 200 None 2026-04-03 13:40:45,476 - lava_callback - DEBUG - Processing node 69cd204d74bda3ec93a42b5a: baseline-arm64-clabbe 2026-04-03 13:40:45,476 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd204d74bda3ec93a42b5a 2026-04-03 13:40:45,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:40:45,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd204d74bda3ec93a42b5a HTTP/1.1" 200 None 2026-04-03 13:40:45,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:40:45,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 13:40:45,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:40:45,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd204d74bda3ec93a42b5a HTTP/1.1" 200 None 2026-04-03 13:40:45,533 - lava_callback - ERROR - Error processing callback for node 69cd204d74bda3ec93a42b5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41162 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:42:05,525 - lava_callback - INFO - Received callback for node 69cd20a474bda3ec93a42b75 INFO: 172.21.0.1:58210 - "POST /node/69cd20a474bda3ec93a42b75 HTTP/1.0" 202 Accepted 2026-04-03 13:42:05,531 - lava_callback - INFO - Starting async job submit for node 69cd20a474bda3ec93a42b75 2026-04-03 13:42:05,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:42:05,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd20a474bda3ec93a42b75 HTTP/1.1" 200 None 2026-04-03 13:42:05,546 - lava_callback - DEBUG - Processing node 69cd20a474bda3ec93a42b75: baseline-arm64-clabbe 2026-04-03 13:42:05,547 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd20a474bda3ec93a42b75 2026-04-03 13:42:05,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:42:05,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd20a474bda3ec93a42b75 HTTP/1.1" 200 None 2026-04-03 13:42:05,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:42:05,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:42:05,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:42:05,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd20a474bda3ec93a42b75 HTTP/1.1" 200 None 2026-04-03 13:42:05,607 - lava_callback - ERROR - Error processing callback for node 69cd20a474bda3ec93a42b75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41388 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 13:43:15,974 - lava_callback - INFO - Received callback for node 69cd20ee74bda3ec93a42b91 INFO: 172.21.0.1:41402 - "POST /node/69cd20ee74bda3ec93a42b91 HTTP/1.0" 202 Accepted 2026-04-03 13:43:15,979 - lava_callback - INFO - Starting async job submit for node 69cd20ee74bda3ec93a42b91 2026-04-03 13:43:15,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:43:15,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd20ee74bda3ec93a42b91 HTTP/1.1" 200 None 2026-04-03 13:43:15,996 - lava_callback - DEBUG - Processing node 69cd20ee74bda3ec93a42b91: baseline-arm64-baylibre 2026-04-03 13:43:15,996 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd20ee74bda3ec93a42b91 2026-04-03 13:43:15,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:43:16,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd20ee74bda3ec93a42b91 HTTP/1.1" 200 None 2026-04-03 13:43:16,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:43:16,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:43:16,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:43:16,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd20ee74bda3ec93a42b91 HTTP/1.1" 200 None 2026-04-03 13:43:16,051 - lava_callback - ERROR - Error processing callback for node 69cd20ee74bda3ec93a42b91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44206 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:45:05,642 - lava_callback - INFO - Received callback for node 69cd214c74bda3ec93a42bac INFO: 172.21.0.1:49554 - "POST /node/69cd214c74bda3ec93a42bac HTTP/1.0" 202 Accepted 2026-04-03 13:45:05,647 - lava_callback - INFO - Starting async job submit for node 69cd214c74bda3ec93a42bac 2026-04-03 13:45:05,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:45:05,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd214c74bda3ec93a42bac HTTP/1.1" 200 None 2026-04-03 13:45:05,663 - lava_callback - DEBUG - Processing node 69cd214c74bda3ec93a42bac: baseline-arm64-clabbe 2026-04-03 13:45:05,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd214c74bda3ec93a42bac 2026-04-03 13:45:05,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:45:05,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd214c74bda3ec93a42bac HTTP/1.1" 200 None 2026-04-03 13:45:05,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:45:05,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:45:05,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:45:05,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd214c74bda3ec93a42bac HTTP/1.1" 200 None 2026-04-03 13:45:05,725 - lava_callback - ERROR - Error processing callback for node 69cd214c74bda3ec93a42bac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43652 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 13:46:16,098 - lava_callback - INFO - Received callback for node 69cd219874bda3ec93a42bbf INFO: 172.21.0.1:43656 - "POST /node/69cd219874bda3ec93a42bbf HTTP/1.0" 202 Accepted 2026-04-03 13:46:16,104 - lava_callback - INFO - Starting async job submit for node 69cd219874bda3ec93a42bbf 2026-04-03 13:46:16,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:46:16,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd219874bda3ec93a42bbf HTTP/1.1" 200 None 2026-04-03 13:46:16,121 - lava_callback - DEBUG - Processing node 69cd219874bda3ec93a42bbf: baseline-arm64-baylibre 2026-04-03 13:46:16,121 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd219874bda3ec93a42bbf 2026-04-03 13:46:16,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:46:16,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd219874bda3ec93a42bbf HTTP/1.1" 200 None 2026-04-03 13:46:16,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:46:16,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:46:16,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:46:16,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd219874bda3ec93a42bbf HTTP/1.1" 200 None 2026-04-03 13:46:16,185 - lava_callback - ERROR - Error processing callback for node 69cd219874bda3ec93a42bbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:47:36,142 - lava_callback - INFO - Received callback for node 69cd21f074bda3ec93a42bda INFO: 172.21.0.1:41100 - "POST /node/69cd21f074bda3ec93a42bda HTTP/1.0" 202 Accepted 2026-04-03 13:47:36,148 - lava_callback - INFO - Starting async job submit for node 69cd21f074bda3ec93a42bda 2026-04-03 13:47:36,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:47:36,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd21f074bda3ec93a42bda HTTP/1.1" 200 None 2026-04-03 13:47:36,163 - lava_callback - DEBUG - Processing node 69cd21f074bda3ec93a42bda: baseline-arm64-baylibre 2026-04-03 13:47:36,164 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd21f074bda3ec93a42bda 2026-04-03 13:47:36,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:47:36,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd21f074bda3ec93a42bda HTTP/1.1" 200 None 2026-04-03 13:47:36,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:47:36,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:47:36,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:47:36,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd21f074bda3ec93a42bda HTTP/1.1" 200 None 2026-04-03 13:47:36,222 - lava_callback - ERROR - Error processing callback for node 69cd21f074bda3ec93a42bda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45410 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:49:05,776 - lava_callback - INFO - Received callback for node 69cd224e74bda3ec93a42bfe INFO: 172.21.0.1:36628 - "POST /node/69cd224e74bda3ec93a42bfe HTTP/1.0" 202 Accepted 2026-04-03 13:49:05,782 - lava_callback - INFO - Starting async job submit for node 69cd224e74bda3ec93a42bfe 2026-04-03 13:49:05,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:49:05,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd224e74bda3ec93a42bfe HTTP/1.1" 200 None 2026-04-03 13:49:05,798 - lava_callback - DEBUG - Processing node 69cd224e74bda3ec93a42bfe: baseline-arm64-clabbe 2026-04-03 13:49:05,798 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd224e74bda3ec93a42bfe 2026-04-03 13:49:05,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:49:05,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd224e74bda3ec93a42bfe HTTP/1.1" 200 None 2026-04-03 13:49:05,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:49:05,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:49:05,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:49:05,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd224e74bda3ec93a42bfe HTTP/1.1" 200 None 2026-04-03 13:49:05,855 - lava_callback - ERROR - Error processing callback for node 69cd224e74bda3ec93a42bfe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33768 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:50:36,658 - lava_callback - INFO - Received callback for node 69cd229d74bda3ec93a42c12 INFO: 172.21.0.1:33782 - "POST /node/69cd229d74bda3ec93a42c12 HTTP/1.0" 202 Accepted 2026-04-03 13:50:36,663 - lava_callback - INFO - Starting async job submit for node 69cd229d74bda3ec93a42c12 2026-04-03 13:50:36,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:50:36,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd229d74bda3ec93a42c12 HTTP/1.1" 200 None 2026-04-03 13:50:36,680 - lava_callback - DEBUG - Processing node 69cd229d74bda3ec93a42c12: baseline-arm64-baylibre 2026-04-03 13:50:36,680 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd229d74bda3ec93a42c12 2026-04-03 13:50:36,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:50:36,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd229d74bda3ec93a42c12 HTTP/1.1" 200 None 2026-04-03 13:50:36,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:50:36,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:50:36,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:50:36,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd229d74bda3ec93a42c12 HTTP/1.1" 200 None 2026-04-03 13:50:36,750 - lava_callback - ERROR - Error processing callback for node 69cd229d74bda3ec93a42c12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59324 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:52:05,892 - lava_callback - INFO - Received callback for node 69cd22f974bda3ec93a42c2e INFO: 172.21.0.1:43812 - "POST /node/69cd22f974bda3ec93a42c2e HTTP/1.0" 202 Accepted 2026-04-03 13:52:05,898 - lava_callback - INFO - Starting async job submit for node 69cd22f974bda3ec93a42c2e 2026-04-03 13:52:05,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:52:05,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd22f974bda3ec93a42c2e HTTP/1.1" 200 None 2026-04-03 13:52:05,914 - lava_callback - DEBUG - Processing node 69cd22f974bda3ec93a42c2e: baseline-arm64-clabbe 2026-04-03 13:52:05,914 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd22f974bda3ec93a42c2e 2026-04-03 13:52:05,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:52:05,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd22f974bda3ec93a42c2e HTTP/1.1" 200 None 2026-04-03 13:52:05,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:52:05,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:52:05,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:52:05,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd22f974bda3ec93a42c2e HTTP/1.1" 200 None 2026-04-03 13:52:05,980 - lava_callback - ERROR - Error processing callback for node 69cd22f974bda3ec93a42c2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50496 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 13:53:16,517 - lava_callback - INFO - Received callback for node 69cd234774bda3ec93a42c4a INFO: 172.21.0.1:50502 - "POST /node/69cd234774bda3ec93a42c4a HTTP/1.0" 202 Accepted 2026-04-03 13:53:16,523 - lava_callback - INFO - Starting async job submit for node 69cd234774bda3ec93a42c4a 2026-04-03 13:53:16,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:50512 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:53:16,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd234774bda3ec93a42c4a HTTP/1.1" 200 None 2026-04-03 13:53:16,539 - lava_callback - DEBUG - Processing node 69cd234774bda3ec93a42c4a: baseline-arm64-baylibre 2026-04-03 13:53:16,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd234774bda3ec93a42c4a 2026-04-03 13:53:16,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:53:16,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd234774bda3ec93a42c4a HTTP/1.1" 200 None 2026-04-03 13:53:16,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:53:16,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:53:16,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:53:16,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd234774bda3ec93a42c4a HTTP/1.1" 200 None 2026-04-03 13:53:16,600 - lava_callback - ERROR - Error processing callback for node 69cd234774bda3ec93a42c4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44828 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:54:56,539 - lava_callback - INFO - Received callback for node 69cd239d74bda3ec93a42c66 INFO: 172.21.0.1:44834 - "POST /node/69cd239d74bda3ec93a42c66 HTTP/1.0" 202 Accepted 2026-04-03 13:54:56,544 - lava_callback - INFO - Starting async job submit for node 69cd239d74bda3ec93a42c66 2026-04-03 13:54:56,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:54:56,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd239d74bda3ec93a42c66 HTTP/1.1" 200 None 2026-04-03 13:54:56,580 - lava_callback - DEBUG - Processing node 69cd239d74bda3ec93a42c66: baseline-arm64-baylibre 2026-04-03 13:54:56,580 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd239d74bda3ec93a42c66 2026-04-03 13:54:56,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:54:56,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd239d74bda3ec93a42c66 HTTP/1.1" 200 None 2026-04-03 13:54:56,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:54:56,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:54:56,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:54:56,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd239d74bda3ec93a42c66 HTTP/1.1" 200 None 2026-04-03 13:54:56,637 - lava_callback - ERROR - Error processing callback for node 69cd239d74bda3ec93a42c66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41748 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:56:26,051 - lava_callback - INFO - Received callback for node 69cd23f674bda3ec93a42c85 INFO: 172.21.0.1:39892 - "POST /node/69cd23f674bda3ec93a42c85 HTTP/1.0" 202 Accepted 2026-04-03 13:56:26,057 - lava_callback - INFO - Starting async job submit for node 69cd23f674bda3ec93a42c85 2026-04-03 13:56:26,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:56:26,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd23f674bda3ec93a42c85 HTTP/1.1" 200 None 2026-04-03 13:56:26,073 - lava_callback - DEBUG - Processing node 69cd23f674bda3ec93a42c85: baseline-arm64-clabbe 2026-04-03 13:56:26,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd23f674bda3ec93a42c85 2026-04-03 13:56:26,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:56:26,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd23f674bda3ec93a42c85 HTTP/1.1" 200 None 2026-04-03 13:56:26,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:56:26,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:56:26,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:56:26,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd23f674bda3ec93a42c85 HTTP/1.1" 200 None 2026-04-03 13:56:26,130 - lava_callback - ERROR - Error processing callback for node 69cd23f674bda3ec93a42c85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33124 - "GET / HTTP/1.0" 200 OK 2026-04-03 13:59:06,183 - lava_callback - INFO - Received callback for node 69cd24a474bda3ec93a42cb1 INFO: 172.21.0.1:56390 - "POST /node/69cd24a474bda3ec93a42cb1 HTTP/1.0" 202 Accepted 2026-04-03 13:59:06,189 - lava_callback - INFO - Starting async job submit for node 69cd24a474bda3ec93a42cb1 2026-04-03 13:59:06,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:59:06,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd24a474bda3ec93a42cb1 HTTP/1.1" 200 None 2026-04-03 13:59:06,205 - lava_callback - DEBUG - Processing node 69cd24a474bda3ec93a42cb1: baseline-arm64-clabbe 2026-04-03 13:59:06,205 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd24a474bda3ec93a42cb1 2026-04-03 13:59:06,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:59:06,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd24a474bda3ec93a42cb1 HTTP/1.1" 200 None 2026-04-03 13:59:06,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:59:06,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 13:59:06,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 13:59:06,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd24a474bda3ec93a42cb1 HTTP/1.1" 200 None 2026-04-03 13:59:06,268 - lava_callback - ERROR - Error processing callback for node 69cd24a474bda3ec93a42cb1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40142 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:00:46,269 - lava_callback - INFO - Received callback for node 69cd24fa74bda3ec93a42cd0 INFO: 172.21.0.1:48216 - "POST /node/69cd24fa74bda3ec93a42cd0 HTTP/1.0" 202 Accepted 2026-04-03 14:00:46,275 - lava_callback - INFO - Starting async job submit for node 69cd24fa74bda3ec93a42cd0 2026-04-03 14:00:46,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:00:46,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd24fa74bda3ec93a42cd0 HTTP/1.1" 200 None 2026-04-03 14:00:46,296 - lava_callback - DEBUG - Processing node 69cd24fa74bda3ec93a42cd0: baseline-arm64-clabbe 2026-04-03 14:00:46,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd24fa74bda3ec93a42cd0 2026-04-03 14:00:46,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:00:46,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd24fa74bda3ec93a42cd0 HTTP/1.1" 200 None 2026-04-03 14:00:46,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:00:46,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:00:46,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:00:46,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd24fa74bda3ec93a42cd0 HTTP/1.1" 200 None 2026-04-03 14:00:46,399 - lava_callback - ERROR - Error processing callback for node 69cd24fa74bda3ec93a42cd0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36598 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:03:26,310 - lava_callback - INFO - Received callback for node 69cd25a974bda3ec93a42d02 INFO: 172.21.0.1:57814 - "POST /node/69cd25a974bda3ec93a42d02 HTTP/1.0" 202 Accepted 2026-04-03 14:03:26,316 - lava_callback - INFO - Starting async job submit for node 69cd25a974bda3ec93a42d02 2026-04-03 14:03:26,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:03:26,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd25a974bda3ec93a42d02 HTTP/1.1" 200 None 2026-04-03 14:03:26,334 - lava_callback - DEBUG - Processing node 69cd25a974bda3ec93a42d02: baseline-arm64-clabbe 2026-04-03 14:03:26,334 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd25a974bda3ec93a42d02 2026-04-03 14:03:26,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:03:26,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd25a974bda3ec93a42d02 HTTP/1.1" 200 None 2026-04-03 14:03:26,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:03:26,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:03:26,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:03:26,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd25a974bda3ec93a42d02 HTTP/1.1" 200 None 2026-04-03 14:03:26,407 - lava_callback - ERROR - Error processing callback for node 69cd25a974bda3ec93a42d02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38680 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:04:57,156 - lava_callback - INFO - Received callback for node 69cd25f674bda3ec93a42d1f INFO: 172.21.0.1:38690 - "POST /node/69cd25f674bda3ec93a42d1f HTTP/1.0" 202 Accepted 2026-04-03 14:04:57,161 - lava_callback - INFO - Starting async job submit for node 69cd25f674bda3ec93a42d1f 2026-04-03 14:04:57,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:04:57,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd25f674bda3ec93a42d1f HTTP/1.1" 200 None 2026-04-03 14:04:57,196 - lava_callback - DEBUG - Processing node 69cd25f674bda3ec93a42d1f: baseline-arm64-baylibre 2026-04-03 14:04:57,197 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd25f674bda3ec93a42d1f 2026-04-03 14:04:57,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:04:57,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd25f674bda3ec93a42d1f HTTP/1.1" 200 None 2026-04-03 14:04:57,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:04:57,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:04:57,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:04:57,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd25f674bda3ec93a42d1f HTTP/1.1" 200 None 2026-04-03 14:04:57,282 - lava_callback - ERROR - Error processing callback for node 69cd25f674bda3ec93a42d1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39128 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:06:26,450 - lava_callback - INFO - Received callback for node 69cd265474bda3ec93a42d3e INFO: 172.21.0.1:42952 - "POST /node/69cd265474bda3ec93a42d3e HTTP/1.0" 202 Accepted 2026-04-03 14:06:26,456 - lava_callback - INFO - Starting async job submit for node 69cd265474bda3ec93a42d3e 2026-04-03 14:06:26,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:06:26,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd265474bda3ec93a42d3e HTTP/1.1" 200 None 2026-04-03 14:06:26,471 - lava_callback - DEBUG - Processing node 69cd265474bda3ec93a42d3e: baseline-arm64-clabbe 2026-04-03 14:06:26,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd265474bda3ec93a42d3e 2026-04-03 14:06:26,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:06:26,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd265474bda3ec93a42d3e HTTP/1.1" 200 None 2026-04-03 14:06:26,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:06:26,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None INFO: 172.21.0.1:42960 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:06:26,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:06:26,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd265474bda3ec93a42d3e HTTP/1.1" 200 None 2026-04-03 14:06:26,539 - lava_callback - ERROR - Error processing callback for node 69cd265474bda3ec93a42d3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39262 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:07:37,223 - lava_callback - INFO - Received callback for node 69cd26a474bda3ec93a42d4d INFO: 172.21.0.1:39268 - "POST /node/69cd26a474bda3ec93a42d4d HTTP/1.0" 202 Accepted 2026-04-03 14:07:37,229 - lava_callback - INFO - Starting async job submit for node 69cd26a474bda3ec93a42d4d 2026-04-03 14:07:37,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:07:37,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd26a474bda3ec93a42d4d HTTP/1.1" 200 None 2026-04-03 14:07:37,245 - lava_callback - DEBUG - Processing node 69cd26a474bda3ec93a42d4d: baseline-arm64-baylibre 2026-04-03 14:07:37,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd26a474bda3ec93a42d4d 2026-04-03 14:07:37,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:07:37,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd26a474bda3ec93a42d4d HTTP/1.1" 200 None 2026-04-03 14:07:37,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:07:37,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:07:37,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:07:37,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd26a474bda3ec93a42d4d HTTP/1.1" 200 None 2026-04-03 14:07:37,317 - lava_callback - ERROR - Error processing callback for node 69cd26a474bda3ec93a42d4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35266 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:09:06,592 - lava_callback - INFO - Received callback for node 69cd26ff74bda3ec93a42d75 INFO: 172.21.0.1:35268 - "POST /node/69cd26ff74bda3ec93a42d75 HTTP/1.0" 202 Accepted 2026-04-03 14:09:06,598 - lava_callback - INFO - Starting async job submit for node 69cd26ff74bda3ec93a42d75 2026-04-03 14:09:06,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:09:06,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd26ff74bda3ec93a42d75 HTTP/1.1" 200 None 2026-04-03 14:09:06,615 - lava_callback - DEBUG - Processing node 69cd26ff74bda3ec93a42d75: baseline-arm64-clabbe 2026-04-03 14:09:06,615 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd26ff74bda3ec93a42d75 2026-04-03 14:09:06,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:09:06,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd26ff74bda3ec93a42d75 HTTP/1.1" 200 None 2026-04-03 14:09:06,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:09:06,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:09:06,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:09:06,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd26ff74bda3ec93a42d75 HTTP/1.1" 200 None 2026-04-03 14:09:06,686 - lava_callback - ERROR - Error processing callback for node 69cd26ff74bda3ec93a42d75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55802 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:12:06,758 - lava_callback - INFO - Received callback for node 69cd27ab74bda3ec93a42da1 INFO: 172.21.0.1:55808 - "POST /node/69cd27ab74bda3ec93a42da1 HTTP/1.0" 202 Accepted 2026-04-03 14:12:06,764 - lava_callback - INFO - Starting async job submit for node 69cd27ab74bda3ec93a42da1 2026-04-03 14:12:06,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:12:06,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd27ab74bda3ec93a42da1 HTTP/1.1" 200 None 2026-04-03 14:12:06,787 - lava_callback - DEBUG - Processing node 69cd27ab74bda3ec93a42da1: baseline-arm64-clabbe 2026-04-03 14:12:06,788 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd27ab74bda3ec93a42da1 2026-04-03 14:12:06,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:12:06,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd27ab74bda3ec93a42da1 HTTP/1.1" 200 None 2026-04-03 14:12:06,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:12:06,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:12:06,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:12:06,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd27ab74bda3ec93a42da1 HTTP/1.1" 200 None 2026-04-03 14:12:06,846 - lava_callback - ERROR - Error processing callback for node 69cd27ab74bda3ec93a42da1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41104 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:13:17,534 - lava_callback - INFO - Received callback for node 69cd27f774bda3ec93a42dbf INFO: 172.21.0.1:41110 - "POST /node/69cd27f774bda3ec93a42dbf HTTP/1.0" 202 Accepted 2026-04-03 14:13:17,540 - lava_callback - INFO - Starting async job submit for node 69cd27f774bda3ec93a42dbf 2026-04-03 14:13:17,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:13:17,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd27f774bda3ec93a42dbf HTTP/1.1" 200 None 2026-04-03 14:13:17,562 - lava_callback - DEBUG - Processing node 69cd27f774bda3ec93a42dbf: baseline-arm64-baylibre 2026-04-03 14:13:17,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd27f774bda3ec93a42dbf 2026-04-03 14:13:17,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:13:17,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd27f774bda3ec93a42dbf HTTP/1.1" 200 None 2026-04-03 14:13:17,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:13:17,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:13:17,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:13:17,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd27f774bda3ec93a42dbf HTTP/1.1" 200 None 2026-04-03 14:13:17,652 - lava_callback - ERROR - Error processing callback for node 69cd27f774bda3ec93a42dbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41390 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:14:46,869 - lava_callback - INFO - Received callback for node 69cd285374bda3ec93a42dd3 INFO: 172.21.0.1:41406 - "POST /node/69cd285374bda3ec93a42dd3 HTTP/1.0" 202 Accepted 2026-04-03 14:14:46,874 - lava_callback - INFO - Starting async job submit for node 69cd285374bda3ec93a42dd3 2026-04-03 14:14:46,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:14:47,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd285374bda3ec93a42dd3 HTTP/1.1" 200 None 2026-04-03 14:14:47,061 - lava_callback - DEBUG - Processing node 69cd285374bda3ec93a42dd3: baseline-arm64-clabbe 2026-04-03 14:14:47,061 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd285374bda3ec93a42dd3 2026-04-03 14:14:47,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:14:47,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd285374bda3ec93a42dd3 HTTP/1.1" 200 None 2026-04-03 14:14:47,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:14:47,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:14:47,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:14:47,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd285374bda3ec93a42dd3 HTTP/1.1" 200 None 2026-04-03 14:14:47,123 - lava_callback - ERROR - Error processing callback for node 69cd285374bda3ec93a42dd3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60648 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:16:27,148 - lava_callback - INFO - Received callback for node 69cd28a774bda3ec93a42de2 INFO: 172.21.0.1:60658 - "POST /node/69cd28a774bda3ec93a42de2 HTTP/1.0" 202 Accepted 2026-04-03 14:16:27,153 - lava_callback - INFO - Starting async job submit for node 69cd28a774bda3ec93a42de2 2026-04-03 14:16:27,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:16:27,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd28a774bda3ec93a42de2 HTTP/1.1" 200 None 2026-04-03 14:16:27,178 - lava_callback - DEBUG - Processing node 69cd28a774bda3ec93a42de2: baseline-arm64-clabbe 2026-04-03 14:16:27,178 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd28a774bda3ec93a42de2 2026-04-03 14:16:27,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:16:27,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd28a774bda3ec93a42de2 HTTP/1.1" 200 None 2026-04-03 14:16:27,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:16:27,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:16:27,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:16:27,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd28a774bda3ec93a42de2 HTTP/1.1" 200 None 2026-04-03 14:16:27,238 - lava_callback - ERROR - Error processing callback for node 69cd28a774bda3ec93a42de2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57836 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:17:46,984 - lava_callback - INFO - Received callback for node 69cd28fb74bda3ec93a42dea INFO: 172.21.0.1:57846 - "POST /node/69cd28fb74bda3ec93a42dea HTTP/1.0" 202 Accepted 2026-04-03 14:17:46,990 - lava_callback - INFO - Starting async job submit for node 69cd28fb74bda3ec93a42dea 2026-04-03 14:17:46,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:17:47,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd28fb74bda3ec93a42dea HTTP/1.1" 200 None 2026-04-03 14:17:47,042 - lava_callback - DEBUG - Processing node 69cd28fb74bda3ec93a42dea: baseline-arm64-clabbe 2026-04-03 14:17:47,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd28fb74bda3ec93a42dea 2026-04-03 14:17:47,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:17:47,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd28fb74bda3ec93a42dea HTTP/1.1" 200 None 2026-04-03 14:17:47,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:17:47,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:17:47,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:17:47,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd28fb74bda3ec93a42dea HTTP/1.1" 200 None 2026-04-03 14:17:47,290 - lava_callback - ERROR - Error processing callback for node 69cd28fb74bda3ec93a42dea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57668 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:19:07,052 - lava_callback - INFO - Received callback for node 69cd294f74bda3ec93a42e01 INFO: 172.21.0.1:57680 - "POST /node/69cd294f74bda3ec93a42e01 HTTP/1.0" 202 Accepted 2026-04-03 14:19:07,058 - lava_callback - INFO - Starting async job submit for node 69cd294f74bda3ec93a42e01 2026-04-03 14:19:07,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:19:07,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd294f74bda3ec93a42e01 HTTP/1.1" 200 None 2026-04-03 14:19:07,080 - lava_callback - DEBUG - Processing node 69cd294f74bda3ec93a42e01: baseline-arm64-clabbe 2026-04-03 14:19:07,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd294f74bda3ec93a42e01 2026-04-03 14:19:07,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:19:07,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd294f74bda3ec93a42e01 HTTP/1.1" 200 None 2026-04-03 14:19:07,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:19:07,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 14:19:07,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:19:07,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd294f74bda3ec93a42e01 HTTP/1.1" 200 None 2026-04-03 14:19:07,132 - lava_callback - ERROR - Error processing callback for node 69cd294f74bda3ec93a42e01: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51632 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:39:59,097 - lava_callback - INFO - Received callback for node 69cd2e3174bda3ec93a42e55 INFO: 172.21.0.1:51640 - "POST /node/69cd2e3174bda3ec93a42e55 HTTP/1.0" 202 Accepted 2026-04-03 14:39:59,102 - lava_callback - INFO - Starting async job submit for node 69cd2e3174bda3ec93a42e55 2026-04-03 14:39:59,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:39:59,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2e3174bda3ec93a42e55 HTTP/1.1" 200 None 2026-04-03 14:39:59,118 - lava_callback - DEBUG - Processing node 69cd2e3174bda3ec93a42e55: baseline-arm-baylibre 2026-04-03 14:39:59,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2e3174bda3ec93a42e55 2026-04-03 14:39:59,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:39:59,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2e3174bda3ec93a42e55 HTTP/1.1" 200 None 2026-04-03 14:39:59,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:39:59,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 14:39:59,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:39:59,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2e3174bda3ec93a42e55 HTTP/1.1" 200 None 2026-04-03 14:39:59,182 - lava_callback - ERROR - Error processing callback for node 69cd2e3174bda3ec93a42e55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39266 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:40:39,002 - lava_callback - INFO - Received callback for node 69cd2e5574bda3ec93a42e60 INFO: 172.21.0.1:39268 - "POST /node/69cd2e5574bda3ec93a42e60 HTTP/1.0" 202 Accepted 2026-04-03 14:40:39,007 - lava_callback - INFO - Starting async job submit for node 69cd2e5574bda3ec93a42e60 2026-04-03 14:40:39,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:40:39,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2e5574bda3ec93a42e60 HTTP/1.1" 200 None 2026-04-03 14:40:39,022 - lava_callback - DEBUG - Processing node 69cd2e5574bda3ec93a42e60: baseline-arm-baylibre 2026-04-03 14:40:39,023 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2e5574bda3ec93a42e60 2026-04-03 14:40:39,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:40:39,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2e5574bda3ec93a42e60 HTTP/1.1" 200 None 2026-04-03 14:40:39,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:40:39,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 14:40:39,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:40:39,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2e5574bda3ec93a42e60 HTTP/1.1" 200 None 2026-04-03 14:40:39,086 - lava_callback - ERROR - Error processing callback for node 69cd2e5574bda3ec93a42e60: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51426 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:42:28,188 - lava_callback - INFO - Received callback for node 69cd2ec774bda3ec93a42e8b INFO: 172.21.0.1:51434 - "POST /node/69cd2ec774bda3ec93a42e8b HTTP/1.0" 202 Accepted 2026-04-03 14:42:28,193 - lava_callback - INFO - Starting async job submit for node 69cd2ec774bda3ec93a42e8b 2026-04-03 14:42:28,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:42:28,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2ec774bda3ec93a42e8b HTTP/1.1" 200 None 2026-04-03 14:42:28,208 - lava_callback - DEBUG - Processing node 69cd2ec774bda3ec93a42e8b: baseline-arm-clabbe 2026-04-03 14:42:28,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2ec774bda3ec93a42e8b 2026-04-03 14:42:28,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:42:28,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2ec774bda3ec93a42e8b HTTP/1.1" 200 None 2026-04-03 14:42:28,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:42:28,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 14:42:28,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:42:28,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2ec774bda3ec93a42e8b HTTP/1.1" 200 None 2026-04-03 14:42:28,284 - lava_callback - ERROR - Error processing callback for node 69cd2ec774bda3ec93a42e8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48182 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:43:08,137 - lava_callback - INFO - Received callback for node 69cd2eeb74bda3ec93a42e9a INFO: 172.21.0.1:48198 - "POST /node/69cd2eeb74bda3ec93a42e9a HTTP/1.0" 202 Accepted 2026-04-03 14:43:08,143 - lava_callback - INFO - Starting async job submit for node 69cd2eeb74bda3ec93a42e9a 2026-04-03 14:43:08,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:43:08,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2eeb74bda3ec93a42e9a HTTP/1.1" 200 None 2026-04-03 14:43:08,162 - lava_callback - DEBUG - Processing node 69cd2eeb74bda3ec93a42e9a: baseline-arm-clabbe 2026-04-03 14:43:08,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2eeb74bda3ec93a42e9a 2026-04-03 14:43:08,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:43:08,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2eeb74bda3ec93a42e9a HTTP/1.1" 200 None 2026-04-03 14:43:08,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:43:08,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 14:43:08,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:43:08,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2eeb74bda3ec93a42e9a HTTP/1.1" 200 None 2026-04-03 14:43:08,219 - lava_callback - ERROR - Error processing callback for node 69cd2eeb74bda3ec93a42e9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55470 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:43:48,214 - lava_callback - INFO - Received callback for node 69cd2f0f74bda3ec93a42ea9 INFO: 172.21.0.1:55480 - "POST /node/69cd2f0f74bda3ec93a42ea9 HTTP/1.0" 202 Accepted 2026-04-03 14:43:48,220 - lava_callback - INFO - Starting async job submit for node 69cd2f0f74bda3ec93a42ea9 2026-04-03 14:43:48,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:43:48,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2f0f74bda3ec93a42ea9 HTTP/1.1" 200 None 2026-04-03 14:43:48,239 - lava_callback - DEBUG - Processing node 69cd2f0f74bda3ec93a42ea9: baseline-arm-clabbe 2026-04-03 14:43:48,239 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2f0f74bda3ec93a42ea9 2026-04-03 14:43:48,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:43:48,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2f0f74bda3ec93a42ea9 HTTP/1.1" 200 None 2026-04-03 14:43:48,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:43:48,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 14:43:48,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:43:48,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2f0f74bda3ec93a42ea9 HTTP/1.1" 200 None 2026-04-03 14:43:48,290 - lava_callback - ERROR - Error processing callback for node 69cd2f0f74bda3ec93a42ea9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37718 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:44:08,199 - lava_callback - INFO - Received callback for node 69cd2f3574bda3ec93a42ebc INFO: 172.21.0.1:37734 - "POST /node/69cd2f3574bda3ec93a42ebc HTTP/1.0" 202 Accepted 2026-04-03 14:44:08,205 - lava_callback - INFO - Starting async job submit for node 69cd2f3574bda3ec93a42ebc 2026-04-03 14:44:08,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:44:08,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2f3574bda3ec93a42ebc HTTP/1.1" 200 None 2026-04-03 14:44:08,229 - lava_callback - DEBUG - Processing node 69cd2f3574bda3ec93a42ebc: baseline-arm-clabbe 2026-04-03 14:44:08,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2f3574bda3ec93a42ebc 2026-04-03 14:44:08,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:44:08,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2f3574bda3ec93a42ebc HTTP/1.1" 200 None 2026-04-03 14:44:08,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:44:08,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 14:44:08,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:44:08,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2f3574bda3ec93a42ebc HTTP/1.1" 200 None 2026-04-03 14:44:08,310 - lava_callback - ERROR - Error processing callback for node 69cd2f3574bda3ec93a42ebc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53034 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:44:48,248 - lava_callback - INFO - Received callback for node 69cd2f5974bda3ec93a42ec3 INFO: 172.21.0.1:53042 - "POST /node/69cd2f5974bda3ec93a42ec3 HTTP/1.0" 202 Accepted 2026-04-03 14:44:48,254 - lava_callback - INFO - Starting async job submit for node 69cd2f5974bda3ec93a42ec3 2026-04-03 14:44:48,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:44:48,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2f5974bda3ec93a42ec3 HTTP/1.1" 200 None 2026-04-03 14:44:48,269 - lava_callback - DEBUG - Processing node 69cd2f5974bda3ec93a42ec3: baseline-arm-clabbe 2026-04-03 14:44:48,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2f5974bda3ec93a42ec3 2026-04-03 14:44:48,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:44:48,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2f5974bda3ec93a42ec3 HTTP/1.1" 200 None 2026-04-03 14:44:48,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:44:48,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 14:44:48,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:44:48,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2f5974bda3ec93a42ec3 HTTP/1.1" 200 None 2026-04-03 14:44:48,325 - lava_callback - ERROR - Error processing callback for node 69cd2f5974bda3ec93a42ec3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60640 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:45:28,285 - lava_callback - INFO - Received callback for node 69cd2f7e74bda3ec93a42ed5 INFO: 172.21.0.1:60650 - "POST /node/69cd2f7e74bda3ec93a42ed5 HTTP/1.0" 202 Accepted 2026-04-03 14:45:28,291 - lava_callback - INFO - Starting async job submit for node 69cd2f7e74bda3ec93a42ed5 2026-04-03 14:45:28,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:45:28,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2f7e74bda3ec93a42ed5 HTTP/1.1" 200 None 2026-04-03 14:45:28,306 - lava_callback - DEBUG - Processing node 69cd2f7e74bda3ec93a42ed5: baseline-arm-clabbe 2026-04-03 14:45:28,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2f7e74bda3ec93a42ed5 2026-04-03 14:45:28,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:45:28,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2f7e74bda3ec93a42ed5 HTTP/1.1" 200 None 2026-04-03 14:45:28,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:45:28,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-03 14:45:28,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:45:28,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2f7e74bda3ec93a42ed5 HTTP/1.1" 200 None 2026-04-03 14:45:28,362 - lava_callback - ERROR - Error processing callback for node 69cd2f7e74bda3ec93a42ed5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53004 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:46:08,372 - lava_callback - INFO - Received callback for node 69cd2fa274bda3ec93a42ee4 INFO: 172.21.0.1:53020 - "POST /node/69cd2fa274bda3ec93a42ee4 HTTP/1.0" 202 Accepted 2026-04-03 14:46:08,378 - lava_callback - INFO - Starting async job submit for node 69cd2fa274bda3ec93a42ee4 2026-04-03 14:46:08,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:46:08,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2fa274bda3ec93a42ee4 HTTP/1.1" 200 None 2026-04-03 14:46:08,394 - lava_callback - DEBUG - Processing node 69cd2fa274bda3ec93a42ee4: baseline-arm-clabbe 2026-04-03 14:46:08,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2fa274bda3ec93a42ee4 2026-04-03 14:46:08,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:46:08,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2fa274bda3ec93a42ee4 HTTP/1.1" 200 None 2026-04-03 14:46:08,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:46:08,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:46:08,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:46:08,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2fa274bda3ec93a42ee4 HTTP/1.1" 200 None 2026-04-03 14:46:08,503 - lava_callback - ERROR - Error processing callback for node 69cd2fa274bda3ec93a42ee4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59270 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:46:48,321 - lava_callback - INFO - Received callback for node 69cd2fc774bda3ec93a42ef4 INFO: 172.21.0.1:59280 - "POST /node/69cd2fc774bda3ec93a42ef4 HTTP/1.0" 202 Accepted 2026-04-03 14:46:48,330 - lava_callback - INFO - Starting async job submit for node 69cd2fc774bda3ec93a42ef4 2026-04-03 14:46:48,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:46:48,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2fc774bda3ec93a42ef4 HTTP/1.1" 200 None 2026-04-03 14:46:48,346 - lava_callback - DEBUG - Processing node 69cd2fc774bda3ec93a42ef4: baseline-arm-clabbe 2026-04-03 14:46:48,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2fc774bda3ec93a42ef4 2026-04-03 14:46:48,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:46:48,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2fc774bda3ec93a42ef4 HTTP/1.1" 200 None 2026-04-03 14:46:48,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:46:48,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:46:48,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:46:48,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2fc774bda3ec93a42ef4 HTTP/1.1" 200 None 2026-04-03 14:46:48,403 - lava_callback - ERROR - Error processing callback for node 69cd2fc774bda3ec93a42ef4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41840 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:47:28,417 - lava_callback - INFO - Received callback for node 69cd2feb74bda3ec93a42f0c INFO: 172.21.0.1:41842 - "POST /node/69cd2feb74bda3ec93a42f0c HTTP/1.0" 202 Accepted 2026-04-03 14:47:28,423 - lava_callback - INFO - Starting async job submit for node 69cd2feb74bda3ec93a42f0c 2026-04-03 14:47:28,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:47:28,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2feb74bda3ec93a42f0c HTTP/1.1" 200 None 2026-04-03 14:47:28,471 - lava_callback - DEBUG - Processing node 69cd2feb74bda3ec93a42f0c: baseline-arm-clabbe 2026-04-03 14:47:28,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd2feb74bda3ec93a42f0c 2026-04-03 14:47:28,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:47:28,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd2feb74bda3ec93a42f0c HTTP/1.1" 200 None 2026-04-03 14:47:28,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:47:28,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:47:28,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:47:28,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd2feb74bda3ec93a42f0c HTTP/1.1" 200 None 2026-04-03 14:47:28,529 - lava_callback - ERROR - Error processing callback for node 69cd2feb74bda3ec93a42f0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45824 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:47:48,378 - lava_callback - INFO - Received callback for node 69cd301074bda3ec93a42f14 INFO: 172.21.0.1:45830 - "POST /node/69cd301074bda3ec93a42f14 HTTP/1.0" 202 Accepted 2026-04-03 14:47:48,384 - lava_callback - INFO - Starting async job submit for node 69cd301074bda3ec93a42f14 2026-04-03 14:47:48,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:47:48,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd301074bda3ec93a42f14 HTTP/1.1" 200 None 2026-04-03 14:47:48,399 - lava_callback - DEBUG - Processing node 69cd301074bda3ec93a42f14: baseline-arm-clabbe 2026-04-03 14:47:48,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd301074bda3ec93a42f14 2026-04-03 14:47:48,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:47:48,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd301074bda3ec93a42f14 HTTP/1.1" 200 None 2026-04-03 14:47:48,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:47:48,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:47:48,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:47:48,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd301074bda3ec93a42f14 HTTP/1.1" 200 None 2026-04-03 14:47:48,456 - lava_callback - ERROR - Error processing callback for node 69cd301074bda3ec93a42f14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38778 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:48:28,400 - lava_callback - INFO - Received callback for node 69cd303474bda3ec93a42f27 INFO: 172.21.0.1:38790 - "POST /node/69cd303474bda3ec93a42f27 HTTP/1.0" 202 Accepted 2026-04-03 14:48:28,406 - lava_callback - INFO - Starting async job submit for node 69cd303474bda3ec93a42f27 2026-04-03 14:48:28,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:48:28,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd303474bda3ec93a42f27 HTTP/1.1" 200 None 2026-04-03 14:48:28,422 - lava_callback - DEBUG - Processing node 69cd303474bda3ec93a42f27: baseline-arm-clabbe 2026-04-03 14:48:28,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd303474bda3ec93a42f27 2026-04-03 14:48:28,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:48:28,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd303474bda3ec93a42f27 HTTP/1.1" 200 None 2026-04-03 14:48:28,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:48:28,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:48:28,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:48:28,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd303474bda3ec93a42f27 HTTP/1.1" 200 None 2026-04-03 14:48:28,480 - lava_callback - ERROR - Error processing callback for node 69cd303474bda3ec93a42f27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32802 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:49:08,409 - lava_callback - INFO - Received callback for node 69cd305b74bda3ec93a42f38 INFO: 172.21.0.1:32814 - "POST /node/69cd305b74bda3ec93a42f38 HTTP/1.0" 202 Accepted 2026-04-03 14:49:08,415 - lava_callback - INFO - Starting async job submit for node 69cd305b74bda3ec93a42f38 2026-04-03 14:49:08,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:49:08,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd305b74bda3ec93a42f38 HTTP/1.1" 200 None 2026-04-03 14:49:08,430 - lava_callback - DEBUG - Processing node 69cd305b74bda3ec93a42f38: baseline-arm-clabbe 2026-04-03 14:49:08,431 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd305b74bda3ec93a42f38 2026-04-03 14:49:08,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:49:08,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd305b74bda3ec93a42f38 HTTP/1.1" 200 None 2026-04-03 14:49:08,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:49:08,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:49:08,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:49:08,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd305b74bda3ec93a42f38 HTTP/1.1" 200 None 2026-04-03 14:49:08,488 - lava_callback - ERROR - Error processing callback for node 69cd305b74bda3ec93a42f38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56986 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:50:48,455 - lava_callback - INFO - Received callback for node 69cd30c174bda3ec93a42f54 INFO: 172.21.0.1:56988 - "POST /node/69cd30c174bda3ec93a42f54 HTTP/1.0" 202 Accepted 2026-04-03 14:50:48,461 - lava_callback - INFO - Starting async job submit for node 69cd30c174bda3ec93a42f54 2026-04-03 14:50:48,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:50:48,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd30c174bda3ec93a42f54 HTTP/1.1" 200 None 2026-04-03 14:50:48,476 - lava_callback - DEBUG - Processing node 69cd30c174bda3ec93a42f54: baseline-arm-clabbe 2026-04-03 14:50:48,476 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd30c174bda3ec93a42f54 2026-04-03 14:50:48,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:50:48,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd30c174bda3ec93a42f54 HTTP/1.1" 200 None 2026-04-03 14:50:48,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:50:48,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:50:48,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:50:48,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd30c174bda3ec93a42f54 HTTP/1.1" 200 None 2026-04-03 14:50:48,529 - lava_callback - ERROR - Error processing callback for node 69cd30c174bda3ec93a42f54: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60480 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:52:28,539 - lava_callback - INFO - Received callback for node 69cd312974bda3ec93a42f73 INFO: 172.21.0.1:60492 - "POST /node/69cd312974bda3ec93a42f73 HTTP/1.0" 202 Accepted 2026-04-03 14:52:28,544 - lava_callback - INFO - Starting async job submit for node 69cd312974bda3ec93a42f73 2026-04-03 14:52:28,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:52:28,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd312974bda3ec93a42f73 HTTP/1.1" 200 None 2026-04-03 14:52:28,560 - lava_callback - DEBUG - Processing node 69cd312974bda3ec93a42f73: baseline-arm-clabbe 2026-04-03 14:52:28,561 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd312974bda3ec93a42f73 2026-04-03 14:52:28,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:52:28,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd312974bda3ec93a42f73 HTTP/1.1" 200 None 2026-04-03 14:52:28,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:52:28,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:52:28,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:52:28,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd312974bda3ec93a42f73 HTTP/1.1" 200 None 2026-04-03 14:52:28,612 - lava_callback - ERROR - Error processing callback for node 69cd312974bda3ec93a42f73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58160 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:53:08,551 - lava_callback - INFO - Received callback for node 69cd314d74bda3ec93a42f8a INFO: 172.21.0.1:58164 - "POST /node/69cd314d74bda3ec93a42f8a HTTP/1.0" 202 Accepted 2026-04-03 14:53:08,556 - lava_callback - INFO - Starting async job submit for node 69cd314d74bda3ec93a42f8a 2026-04-03 14:53:08,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:53:08,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd314d74bda3ec93a42f8a HTTP/1.1" 200 None 2026-04-03 14:53:08,576 - lava_callback - DEBUG - Processing node 69cd314d74bda3ec93a42f8a: baseline-arm-clabbe 2026-04-03 14:53:08,576 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd314d74bda3ec93a42f8a 2026-04-03 14:53:08,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:53:08,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd314d74bda3ec93a42f8a HTTP/1.1" 200 None 2026-04-03 14:53:08,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:53:08,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:53:08,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:53:08,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd314d74bda3ec93a42f8a HTTP/1.1" 200 None 2026-04-03 14:53:08,630 - lava_callback - ERROR - Error processing callback for node 69cd314d74bda3ec93a42f8a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35770 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:53:48,556 - lava_callback - INFO - Received callback for node 69cd317274bda3ec93a42f9e INFO: 172.21.0.1:35774 - "POST /node/69cd317274bda3ec93a42f9e HTTP/1.0" 202 Accepted 2026-04-03 14:53:48,563 - lava_callback - INFO - Starting async job submit for node 69cd317274bda3ec93a42f9e 2026-04-03 14:53:48,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:53:48,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd317274bda3ec93a42f9e HTTP/1.1" 200 None 2026-04-03 14:53:48,587 - lava_callback - DEBUG - Processing node 69cd317274bda3ec93a42f9e: baseline-arm-clabbe 2026-04-03 14:53:48,587 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd317274bda3ec93a42f9e 2026-04-03 14:53:48,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:53:48,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd317274bda3ec93a42f9e HTTP/1.1" 200 None 2026-04-03 14:53:48,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:53:48,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 14:53:48,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:53:48,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd317274bda3ec93a42f9e HTTP/1.1" 200 None 2026-04-03 14:53:48,669 - lava_callback - ERROR - Error processing callback for node 69cd317274bda3ec93a42f9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54914 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:55:08,609 - lava_callback - INFO - Received callback for node 69cd31bb74bda3ec93a42fb9 INFO: 172.21.0.1:54916 - "POST /node/69cd31bb74bda3ec93a42fb9 HTTP/1.0" 202 Accepted 2026-04-03 14:55:08,614 - lava_callback - INFO - Starting async job submit for node 69cd31bb74bda3ec93a42fb9 2026-04-03 14:55:08,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:55:08,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd31bb74bda3ec93a42fb9 HTTP/1.1" 200 None 2026-04-03 14:55:08,653 - lava_callback - DEBUG - Processing node 69cd31bb74bda3ec93a42fb9: baseline-arm-clabbe 2026-04-03 14:55:08,653 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd31bb74bda3ec93a42fb9 2026-04-03 14:55:08,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:55:08,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd31bb74bda3ec93a42fb9 HTTP/1.1" 200 None 2026-04-03 14:55:08,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:55:08,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 14:55:08,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:55:08,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd31bb74bda3ec93a42fb9 HTTP/1.1" 200 None 2026-04-03 14:55:08,718 - lava_callback - ERROR - Error processing callback for node 69cd31bb74bda3ec93a42fb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45826 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:55:48,664 - lava_callback - INFO - Received callback for node 69cd31e074bda3ec93a42fca INFO: 172.21.0.1:45834 - "POST /node/69cd31e074bda3ec93a42fca HTTP/1.0" 202 Accepted 2026-04-03 14:55:48,669 - lava_callback - INFO - Starting async job submit for node 69cd31e074bda3ec93a42fca 2026-04-03 14:55:48,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:55:48,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd31e074bda3ec93a42fca HTTP/1.1" 200 None 2026-04-03 14:55:48,686 - lava_callback - DEBUG - Processing node 69cd31e074bda3ec93a42fca: baseline-arm-clabbe 2026-04-03 14:55:48,687 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd31e074bda3ec93a42fca 2026-04-03 14:55:48,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:55:48,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd31e074bda3ec93a42fca HTTP/1.1" 200 None 2026-04-03 14:55:48,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:55:48,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 14:55:48,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:55:48,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd31e074bda3ec93a42fca HTTP/1.1" 200 None 2026-04-03 14:55:48,739 - lava_callback - ERROR - Error processing callback for node 69cd31e074bda3ec93a42fca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46006 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:56:28,626 - lava_callback - INFO - Received callback for node 69cd320974bda3ec93a42fdf INFO: 172.21.0.1:46008 - "POST /node/69cd320974bda3ec93a42fdf HTTP/1.0" 202 Accepted 2026-04-03 14:56:28,632 - lava_callback - INFO - Starting async job submit for node 69cd320974bda3ec93a42fdf 2026-04-03 14:56:28,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:56:28,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd320974bda3ec93a42fdf HTTP/1.1" 200 None 2026-04-03 14:56:28,654 - lava_callback - DEBUG - Processing node 69cd320974bda3ec93a42fdf: baseline-arm-clabbe 2026-04-03 14:56:28,654 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd320974bda3ec93a42fdf 2026-04-03 14:56:28,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:56:28,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd320974bda3ec93a42fdf HTTP/1.1" 200 None 2026-04-03 14:56:28,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:56:28,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:56:28,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:56:28,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd320974bda3ec93a42fdf HTTP/1.1" 200 None 2026-04-03 14:56:28,709 - lava_callback - ERROR - Error processing callback for node 69cd320974bda3ec93a42fdf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56888 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:56:48,680 - lava_callback - INFO - Received callback for node 69cd322d74bda3ec93a42fe9 INFO: 172.21.0.1:56890 - "POST /node/69cd322d74bda3ec93a42fe9 HTTP/1.0" 202 Accepted 2026-04-03 14:56:48,686 - lava_callback - INFO - Starting async job submit for node 69cd322d74bda3ec93a42fe9 2026-04-03 14:56:48,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:56:48,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd322d74bda3ec93a42fe9 HTTP/1.1" 200 None 2026-04-03 14:56:48,701 - lava_callback - DEBUG - Processing node 69cd322d74bda3ec93a42fe9: baseline-arm-clabbe 2026-04-03 14:56:48,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd322d74bda3ec93a42fe9 2026-04-03 14:56:48,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:56:48,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd322d74bda3ec93a42fe9 HTTP/1.1" 200 None 2026-04-03 14:56:48,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:56:48,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 14:56:48,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:56:48,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd322d74bda3ec93a42fe9 HTTP/1.1" 200 None 2026-04-03 14:56:48,757 - lava_callback - ERROR - Error processing callback for node 69cd322d74bda3ec93a42fe9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58640 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:57:39,938 - lava_callback - INFO - Received callback for node 69cd324f74bda3ec93a42ffa INFO: 172.21.0.1:33586 - "POST /node/69cd324f74bda3ec93a42ffa HTTP/1.0" 202 Accepted 2026-04-03 14:57:39,943 - lava_callback - INFO - Starting async job submit for node 69cd324f74bda3ec93a42ffa 2026-04-03 14:57:39,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:57:39,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd324f74bda3ec93a42ffa HTTP/1.1" 200 None 2026-04-03 14:57:39,959 - lava_callback - DEBUG - Processing node 69cd324f74bda3ec93a42ffa: baseline-arm-baylibre 2026-04-03 14:57:39,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd324f74bda3ec93a42ffa 2026-04-03 14:57:39,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:57:39,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd324f74bda3ec93a42ffa HTTP/1.1" 200 None 2026-04-03 14:57:39,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:57:39,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 14:57:39,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:57:40,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd324f74bda3ec93a42ffa HTTP/1.1" 200 None 2026-04-03 14:57:40,024 - lava_callback - ERROR - Error processing callback for node 69cd324f74bda3ec93a42ffa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33770 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:57:59,954 - lava_callback - INFO - Received callback for node 69cd327474bda3ec93a43008 INFO: 172.21.0.1:56130 - "POST /node/69cd327474bda3ec93a43008 HTTP/1.0" 202 Accepted 2026-04-03 14:57:59,960 - lava_callback - INFO - Starting async job submit for node 69cd327474bda3ec93a43008 2026-04-03 14:57:59,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:57:59,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd327474bda3ec93a43008 HTTP/1.1" 200 None 2026-04-03 14:57:59,978 - lava_callback - DEBUG - Processing node 69cd327474bda3ec93a43008: baseline-arm-baylibre 2026-04-03 14:57:59,978 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd327474bda3ec93a43008 2026-04-03 14:57:59,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:57:59,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd327474bda3ec93a43008 HTTP/1.1" 200 None 2026-04-03 14:57:59,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:58:00,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 14:58:00,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:58:00,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd327474bda3ec93a43008 HTTP/1.1" 200 None 2026-04-03 14:58:00,056 - lava_callback - ERROR - Error processing callback for node 69cd327474bda3ec93a43008: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55174 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:58:48,772 - lava_callback - INFO - Received callback for node 69cd329974bda3ec93a43017 INFO: 172.21.0.1:55178 - "POST /node/69cd329974bda3ec93a43017 HTTP/1.0" 202 Accepted 2026-04-03 14:58:48,777 - lava_callback - INFO - Starting async job submit for node 69cd329974bda3ec93a43017 2026-04-03 14:58:48,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:58:48,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd329974bda3ec93a43017 HTTP/1.1" 200 None 2026-04-03 14:58:48,794 - lava_callback - DEBUG - Processing node 69cd329974bda3ec93a43017: baseline-arm-clabbe 2026-04-03 14:58:48,794 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd329974bda3ec93a43017 2026-04-03 14:58:48,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:58:48,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd329974bda3ec93a43017 HTTP/1.1" 200 None 2026-04-03 14:58:48,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:58:48,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 14:58:48,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:58:48,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd329974bda3ec93a43017 HTTP/1.1" 200 None 2026-04-03 14:58:48,846 - lava_callback - ERROR - Error processing callback for node 69cd329974bda3ec93a43017: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42586 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:59:28,769 - lava_callback - INFO - Received callback for node 69cd32bd74bda3ec93a43028 INFO: 172.21.0.1:42596 - "POST /node/69cd32bd74bda3ec93a43028 HTTP/1.0" 202 Accepted 2026-04-03 14:59:28,776 - lava_callback - INFO - Starting async job submit for node 69cd32bd74bda3ec93a43028 2026-04-03 14:59:28,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:59:28,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd32bd74bda3ec93a43028 HTTP/1.1" 200 None 2026-04-03 14:59:28,790 - lava_callback - DEBUG - Processing node 69cd32bd74bda3ec93a43028: baseline-arm-clabbe 2026-04-03 14:59:28,791 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd32bd74bda3ec93a43028 2026-04-03 14:59:28,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:59:28,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd32bd74bda3ec93a43028 HTTP/1.1" 200 None 2026-04-03 14:59:28,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:59:28,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 14:59:28,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:59:28,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd32bd74bda3ec93a43028 HTTP/1.1" 200 None 2026-04-03 14:59:28,845 - lava_callback - ERROR - Error processing callback for node 69cd32bd74bda3ec93a43028: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58848 - "GET / HTTP/1.0" 200 OK 2026-04-03 14:59:48,800 - lava_callback - INFO - Received callback for node 69cd32e174bda3ec93a43034 INFO: 172.21.0.1:58854 - "POST /node/69cd32e174bda3ec93a43034 HTTP/1.0" 202 Accepted 2026-04-03 14:59:48,806 - lava_callback - INFO - Starting async job submit for node 69cd32e174bda3ec93a43034 2026-04-03 14:59:48,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:59:48,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd32e174bda3ec93a43034 HTTP/1.1" 200 None 2026-04-03 14:59:48,820 - lava_callback - DEBUG - Processing node 69cd32e174bda3ec93a43034: baseline-arm-clabbe 2026-04-03 14:59:48,821 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd32e174bda3ec93a43034 2026-04-03 14:59:48,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:59:48,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd32e174bda3ec93a43034 HTTP/1.1" 200 None 2026-04-03 14:59:48,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:59:48,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 14:59:48,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 14:59:48,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd32e174bda3ec93a43034 HTTP/1.1" 200 None 2026-04-03 14:59:48,877 - lava_callback - ERROR - Error processing callback for node 69cd32e174bda3ec93a43034: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53364 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:00:28,811 - lava_callback - INFO - Received callback for node 69cd330674bda3ec93a43042 INFO: 172.21.0.1:53368 - "POST /node/69cd330674bda3ec93a43042 HTTP/1.0" 202 Accepted 2026-04-03 15:00:28,817 - lava_callback - INFO - Starting async job submit for node 69cd330674bda3ec93a43042 2026-04-03 15:00:28,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:00:28,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd330674bda3ec93a43042 HTTP/1.1" 200 None 2026-04-03 15:00:28,833 - lava_callback - DEBUG - Processing node 69cd330674bda3ec93a43042: baseline-arm-clabbe 2026-04-03 15:00:28,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd330674bda3ec93a43042 2026-04-03 15:00:28,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:00:28,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd330674bda3ec93a43042 HTTP/1.1" 200 None 2026-04-03 15:00:28,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:00:28,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 15:00:28,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:00:28,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd330674bda3ec93a43042 HTTP/1.1" 200 None 2026-04-03 15:00:28,889 - lava_callback - ERROR - Error processing callback for node 69cd330674bda3ec93a43042: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41096 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:01:08,857 - lava_callback - INFO - Received callback for node 69cd332b74bda3ec93a43058 INFO: 172.21.0.1:41104 - "POST /node/69cd332b74bda3ec93a43058 HTTP/1.0" 202 Accepted 2026-04-03 15:01:08,862 - lava_callback - INFO - Starting async job submit for node 69cd332b74bda3ec93a43058 2026-04-03 15:01:08,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:01:08,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd332b74bda3ec93a43058 HTTP/1.1" 200 None 2026-04-03 15:01:08,880 - lava_callback - DEBUG - Processing node 69cd332b74bda3ec93a43058: baseline-arm-clabbe 2026-04-03 15:01:08,881 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd332b74bda3ec93a43058 2026-04-03 15:01:08,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:01:08,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd332b74bda3ec93a43058 HTTP/1.1" 200 None 2026-04-03 15:01:08,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:01:08,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 15:01:08,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:01:08,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd332b74bda3ec93a43058 HTTP/1.1" 200 None 2026-04-03 15:01:08,935 - lava_callback - ERROR - Error processing callback for node 69cd332b74bda3ec93a43058: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44852 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:01:48,833 - lava_callback - INFO - Received callback for node 69cd335074bda3ec93a43071 INFO: 172.21.0.1:44862 - "POST /node/69cd335074bda3ec93a43071 HTTP/1.0" 202 Accepted 2026-04-03 15:01:48,839 - lava_callback - INFO - Starting async job submit for node 69cd335074bda3ec93a43071 2026-04-03 15:01:48,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:01:48,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd335074bda3ec93a43071 HTTP/1.1" 200 None 2026-04-03 15:01:48,855 - lava_callback - DEBUG - Processing node 69cd335074bda3ec93a43071: baseline-arm-clabbe 2026-04-03 15:01:48,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd335074bda3ec93a43071 2026-04-03 15:01:48,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:01:48,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd335074bda3ec93a43071 HTTP/1.1" 200 None 2026-04-03 15:01:48,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:01:48,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 15:01:48,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:01:48,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd335074bda3ec93a43071 HTTP/1.1" 200 None 2026-04-03 15:01:48,941 - lava_callback - ERROR - Error processing callback for node 69cd335074bda3ec93a43071: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36496 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:02:28,925 - lava_callback - INFO - Received callback for node 69cd337574bda3ec93a43092 INFO: 172.21.0.1:36508 - "POST /node/69cd337574bda3ec93a43092 HTTP/1.0" 202 Accepted 2026-04-03 15:02:28,931 - lava_callback - INFO - Starting async job submit for node 69cd337574bda3ec93a43092 2026-04-03 15:02:28,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:02:28,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd337574bda3ec93a43092 HTTP/1.1" 200 None 2026-04-03 15:02:28,947 - lava_callback - DEBUG - Processing node 69cd337574bda3ec93a43092: baseline-arm-clabbe 2026-04-03 15:02:28,947 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd337574bda3ec93a43092 2026-04-03 15:02:28,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:02:28,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd337574bda3ec93a43092 HTTP/1.1" 200 None 2026-04-03 15:02:28,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:02:28,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 15:02:28,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:02:29,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd337574bda3ec93a43092 HTTP/1.1" 200 None 2026-04-03 15:02:29,004 - lava_callback - ERROR - Error processing callback for node 69cd337574bda3ec93a43092: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35878 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:03:08,948 - lava_callback - INFO - Received callback for node 69cd339974bda3ec93a430ad INFO: 172.21.0.1:35890 - "POST /node/69cd339974bda3ec93a430ad HTTP/1.0" 202 Accepted 2026-04-03 15:03:08,954 - lava_callback - INFO - Starting async job submit for node 69cd339974bda3ec93a430ad 2026-04-03 15:03:08,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:03:08,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd339974bda3ec93a430ad HTTP/1.1" 200 None 2026-04-03 15:03:08,971 - lava_callback - DEBUG - Processing node 69cd339974bda3ec93a430ad: baseline-arm-clabbe 2026-04-03 15:03:08,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd339974bda3ec93a430ad 2026-04-03 15:03:08,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:03:08,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd339974bda3ec93a430ad HTTP/1.1" 200 None 2026-04-03 15:03:08,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:03:09,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 15:03:09,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:03:09,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd339974bda3ec93a430ad HTTP/1.1" 200 None 2026-04-03 15:03:09,067 - lava_callback - ERROR - Error processing callback for node 69cd339974bda3ec93a430ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54642 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:03:29,015 - lava_callback - INFO - Received callback for node 69cd33be74bda3ec93a430bb INFO: 172.21.0.1:54650 - "POST /node/69cd33be74bda3ec93a430bb HTTP/1.0" 202 Accepted 2026-04-03 15:03:29,020 - lava_callback - INFO - Starting async job submit for node 69cd33be74bda3ec93a430bb 2026-04-03 15:03:29,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:03:29,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd33be74bda3ec93a430bb HTTP/1.1" 200 None 2026-04-03 15:03:29,036 - lava_callback - DEBUG - Processing node 69cd33be74bda3ec93a430bb: baseline-arm-clabbe 2026-04-03 15:03:29,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd33be74bda3ec93a430bb 2026-04-03 15:03:29,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:03:29,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd33be74bda3ec93a430bb HTTP/1.1" 200 None 2026-04-03 15:03:29,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:03:29,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 15:03:29,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:03:29,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd33be74bda3ec93a430bb HTTP/1.1" 200 None 2026-04-03 15:03:29,091 - lava_callback - ERROR - Error processing callback for node 69cd33be74bda3ec93a430bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35766 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:04:08,950 - lava_callback - INFO - Received callback for node 69cd33e374bda3ec93a430c9 INFO: 172.21.0.1:35774 - "POST /node/69cd33e374bda3ec93a430c9 HTTP/1.0" 202 Accepted 2026-04-03 15:04:08,960 - lava_callback - INFO - Starting async job submit for node 69cd33e374bda3ec93a430c9 2026-04-03 15:04:08,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:04:08,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd33e374bda3ec93a430c9 HTTP/1.1" 200 None 2026-04-03 15:04:08,976 - lava_callback - DEBUG - Processing node 69cd33e374bda3ec93a430c9: baseline-arm-clabbe 2026-04-03 15:04:08,976 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd33e374bda3ec93a430c9 2026-04-03 15:04:08,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:04:08,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd33e374bda3ec93a430c9 HTTP/1.1" 200 None 2026-04-03 15:04:08,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:04:09,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ef0e33d030878a87ef HTTP/1.1" 200 None 2026-04-03 15:04:09,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:04:09,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd33e374bda3ec93a430c9 HTTP/1.1" 200 None 2026-04-03 15:04:09,039 - lava_callback - ERROR - Error processing callback for node 69cd33e374bda3ec93a430c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44784 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:04:49,071 - lava_callback - INFO - Received callback for node 69cd340974bda3ec93a430d8 INFO: 172.21.0.1:44792 - "POST /node/69cd340974bda3ec93a430d8 HTTP/1.0" 202 Accepted 2026-04-03 15:04:49,078 - lava_callback - INFO - Starting async job submit for node 69cd340974bda3ec93a430d8 2026-04-03 15:04:49,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:04:49,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd340974bda3ec93a430d8 HTTP/1.1" 200 None 2026-04-03 15:04:49,094 - lava_callback - DEBUG - Processing node 69cd340974bda3ec93a430d8: baseline-arm-clabbe 2026-04-03 15:04:49,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd340974bda3ec93a430d8 2026-04-03 15:04:49,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:04:49,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd340974bda3ec93a430d8 HTTP/1.1" 200 None 2026-04-03 15:04:49,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:04:49,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eacbe430e18c5d159df HTTP/1.1" 200 None 2026-04-03 15:04:49,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:04:49,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd340974bda3ec93a430d8 HTTP/1.1" 200 None 2026-04-03 15:04:49,154 - lava_callback - ERROR - Error processing callback for node 69cd340974bda3ec93a430d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44032 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:11:20,809 - lava_callback - INFO - Received callback for node 69cd358374bda3ec93a43121 INFO: 172.21.0.1:51682 - "POST /node/69cd358374bda3ec93a43121 HTTP/1.0" 202 Accepted 2026-04-03 15:11:20,814 - lava_callback - INFO - Starting async job submit for node 69cd358374bda3ec93a43121 2026-04-03 15:11:20,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:11:20,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd358374bda3ec93a43121 HTTP/1.1" 200 None 2026-04-03 15:11:20,833 - lava_callback - DEBUG - Processing node 69cd358374bda3ec93a43121: baseline-arm64-baylibre 2026-04-03 15:11:20,833 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd358374bda3ec93a43121 2026-04-03 15:11:20,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:11:20,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd358374bda3ec93a43121 HTTP/1.1" 200 None 2026-04-03 15:11:20,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:11:20,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:11:20,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:11:20,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd358374bda3ec93a43121 HTTP/1.1" 200 None 2026-04-03 15:11:20,893 - lava_callback - ERROR - Error processing callback for node 69cd358374bda3ec93a43121: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53278 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:12:40,833 - lava_callback - INFO - Received callback for node 69cd35d574bda3ec93a4312d INFO: 172.21.0.1:39236 - "POST /node/69cd35d574bda3ec93a4312d HTTP/1.0" 202 Accepted 2026-04-03 15:12:40,839 - lava_callback - INFO - Starting async job submit for node 69cd35d574bda3ec93a4312d 2026-04-03 15:12:40,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:12:40,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd35d574bda3ec93a4312d HTTP/1.1" 200 None 2026-04-03 15:12:40,854 - lava_callback - DEBUG - Processing node 69cd35d574bda3ec93a4312d: baseline-arm64-baylibre 2026-04-03 15:12:40,854 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd35d574bda3ec93a4312d 2026-04-03 15:12:40,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:12:40,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd35d574bda3ec93a4312d HTTP/1.1" 200 None 2026-04-03 15:12:40,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:12:40,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:12:40,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:12:40,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd35d574bda3ec93a4312d HTTP/1.1" 200 None 2026-04-03 15:12:40,906 - lava_callback - ERROR - Error processing callback for node 69cd35d574bda3ec93a4312d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56012 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:14:00,831 - lava_callback - INFO - Received callback for node 69cd362974bda3ec93a4313e INFO: 172.21.0.1:37304 - "POST /node/69cd362974bda3ec93a4313e HTTP/1.0" 202 Accepted 2026-04-03 15:14:00,836 - lava_callback - INFO - Starting async job submit for node 69cd362974bda3ec93a4313e 2026-04-03 15:14:00,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:14:00,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd362974bda3ec93a4313e HTTP/1.1" 200 None 2026-04-03 15:14:00,859 - lava_callback - DEBUG - Processing node 69cd362974bda3ec93a4313e: baseline-arm64-baylibre 2026-04-03 15:14:00,860 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd362974bda3ec93a4313e 2026-04-03 15:14:00,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:14:00,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd362974bda3ec93a4313e HTTP/1.1" 200 None 2026-04-03 15:14:00,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:14:00,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:14:00,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:14:00,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd362974bda3ec93a4313e HTTP/1.1" 200 None 2026-04-03 15:14:00,914 - lava_callback - ERROR - Error processing callback for node 69cd362974bda3ec93a4313e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42452 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:15:20,920 - lava_callback - INFO - Received callback for node 69cd367d74bda3ec93a4314e INFO: 172.21.0.1:46806 - "POST /node/69cd367d74bda3ec93a4314e HTTP/1.0" 202 Accepted 2026-04-03 15:15:20,925 - lava_callback - INFO - Starting async job submit for node 69cd367d74bda3ec93a4314e 2026-04-03 15:15:20,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:15:20,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd367d74bda3ec93a4314e HTTP/1.1" 200 None 2026-04-03 15:15:20,941 - lava_callback - DEBUG - Processing node 69cd367d74bda3ec93a4314e: baseline-arm64-baylibre 2026-04-03 15:15:20,942 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd367d74bda3ec93a4314e 2026-04-03 15:15:20,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:15:20,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd367d74bda3ec93a4314e HTTP/1.1" 200 None 2026-04-03 15:15:20,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:15:20,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:15:20,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:15:20,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd367d74bda3ec93a4314e HTTP/1.1" 200 None 2026-04-03 15:15:20,998 - lava_callback - ERROR - Error processing callback for node 69cd367d74bda3ec93a4314e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43334 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:16:41,007 - lava_callback - INFO - Received callback for node 69cd36d274bda3ec93a4315a INFO: 172.21.0.1:48884 - "POST /node/69cd36d274bda3ec93a4315a HTTP/1.0" 202 Accepted 2026-04-03 15:16:41,012 - lava_callback - INFO - Starting async job submit for node 69cd36d274bda3ec93a4315a 2026-04-03 15:16:41,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:16:41,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd36d274bda3ec93a4315a HTTP/1.1" 200 None 2026-04-03 15:16:41,031 - lava_callback - DEBUG - Processing node 69cd36d274bda3ec93a4315a: baseline-arm64-baylibre 2026-04-03 15:16:41,031 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd36d274bda3ec93a4315a 2026-04-03 15:16:41,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:16:41,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd36d274bda3ec93a4315a HTTP/1.1" 200 None 2026-04-03 15:16:41,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:16:41,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:16:41,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:16:41,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd36d274bda3ec93a4315a HTTP/1.1" 200 None 2026-04-03 15:16:41,111 - lava_callback - ERROR - Error processing callback for node 69cd36d274bda3ec93a4315a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49022 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:18:21,116 - lava_callback - INFO - Received callback for node 69cd372874bda3ec93a4316a INFO: 172.21.0.1:43548 - "POST /node/69cd372874bda3ec93a4316a HTTP/1.0" 202 Accepted 2026-04-03 15:18:21,122 - lava_callback - INFO - Starting async job submit for node 69cd372874bda3ec93a4316a 2026-04-03 15:18:21,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:18:21,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd372874bda3ec93a4316a HTTP/1.1" 200 None 2026-04-03 15:18:21,143 - lava_callback - DEBUG - Processing node 69cd372874bda3ec93a4316a: baseline-arm64-baylibre 2026-04-03 15:18:21,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd372874bda3ec93a4316a 2026-04-03 15:18:21,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:18:21,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd372874bda3ec93a4316a HTTP/1.1" 200 None 2026-04-03 15:18:21,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:18:21,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:18:21,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:18:21,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd372874bda3ec93a4316a HTTP/1.1" 200 None 2026-04-03 15:18:21,200 - lava_callback - ERROR - Error processing callback for node 69cd372874bda3ec93a4316a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60278 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:21:09,757 - lava_callback - INFO - Received callback for node 69cd37d674bda3ec93a4318c INFO: 172.21.0.1:47154 - "POST /node/69cd37d674bda3ec93a4318c HTTP/1.0" 202 Accepted 2026-04-03 15:21:09,764 - lava_callback - INFO - Starting async job submit for node 69cd37d674bda3ec93a4318c 2026-04-03 15:21:09,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:21:09,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd37d674bda3ec93a4318c HTTP/1.1" 200 None 2026-04-03 15:21:09,779 - lava_callback - DEBUG - Processing node 69cd37d674bda3ec93a4318c: baseline-arm64-clabbe 2026-04-03 15:21:09,780 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd37d674bda3ec93a4318c 2026-04-03 15:21:09,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:21:09,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd37d674bda3ec93a4318c HTTP/1.1" 200 None 2026-04-03 15:21:09,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:21:09,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:21:09,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:21:09,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd37d674bda3ec93a4318c HTTP/1.1" 200 None 2026-04-03 15:21:09,831 - lava_callback - ERROR - Error processing callback for node 69cd37d674bda3ec93a4318c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40590 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:22:29,818 - lava_callback - INFO - Received callback for node 69cd382b74bda3ec93a43199 INFO: 172.21.0.1:34602 - "POST /node/69cd382b74bda3ec93a43199 HTTP/1.0" 202 Accepted 2026-04-03 15:22:29,823 - lava_callback - INFO - Starting async job submit for node 69cd382b74bda3ec93a43199 2026-04-03 15:22:29,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:22:29,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd382b74bda3ec93a43199 HTTP/1.1" 200 None 2026-04-03 15:22:29,840 - lava_callback - DEBUG - Processing node 69cd382b74bda3ec93a43199: baseline-arm64-clabbe 2026-04-03 15:22:29,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd382b74bda3ec93a43199 2026-04-03 15:22:29,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:22:29,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd382b74bda3ec93a43199 HTTP/1.1" 200 None 2026-04-03 15:22:29,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:22:29,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:22:29,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:22:29,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd382b74bda3ec93a43199 HTTP/1.1" 200 None 2026-04-03 15:22:29,906 - lava_callback - ERROR - Error processing callback for node 69cd382b74bda3ec93a43199: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59136 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:23:49,861 - lava_callback - INFO - Received callback for node 69cd388174bda3ec93a431a8 INFO: 172.21.0.1:42560 - "POST /node/69cd388174bda3ec93a431a8 HTTP/1.0" 202 Accepted 2026-04-03 15:23:49,867 - lava_callback - INFO - Starting async job submit for node 69cd388174bda3ec93a431a8 2026-04-03 15:23:49,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:23:49,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd388174bda3ec93a431a8 HTTP/1.1" 200 None 2026-04-03 15:23:49,882 - lava_callback - DEBUG - Processing node 69cd388174bda3ec93a431a8: baseline-arm64-clabbe 2026-04-03 15:23:49,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd388174bda3ec93a431a8 2026-04-03 15:23:49,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:23:49,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd388174bda3ec93a431a8 HTTP/1.1" 200 None 2026-04-03 15:23:49,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:23:49,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:23:49,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:23:49,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd388174bda3ec93a431a8 HTTP/1.1" 200 None 2026-04-03 15:23:49,970 - lava_callback - ERROR - Error processing callback for node 69cd388174bda3ec93a431a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51222 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:25:29,927 - lava_callback - INFO - Received callback for node 69cd38d674bda3ec93a431b9 INFO: 172.21.0.1:55840 - "POST /node/69cd38d674bda3ec93a431b9 HTTP/1.0" 202 Accepted 2026-04-03 15:25:29,933 - lava_callback - INFO - Starting async job submit for node 69cd38d674bda3ec93a431b9 2026-04-03 15:25:29,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:25:29,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd38d674bda3ec93a431b9 HTTP/1.1" 200 None 2026-04-03 15:25:29,958 - lava_callback - DEBUG - Processing node 69cd38d674bda3ec93a431b9: baseline-arm64-clabbe 2026-04-03 15:25:29,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd38d674bda3ec93a431b9 2026-04-03 15:25:29,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:25:29,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd38d674bda3ec93a431b9 HTTP/1.1" 200 None 2026-04-03 15:25:29,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:25:29,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:25:29,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:25:30,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd38d674bda3ec93a431b9 HTTP/1.1" 200 None 2026-04-03 15:25:30,024 - lava_callback - ERROR - Error processing callback for node 69cd38d674bda3ec93a431b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51224 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:26:50,012 - lava_callback - INFO - Received callback for node 69cd392974bda3ec93a431c0 INFO: 172.21.0.1:49056 - "POST /node/69cd392974bda3ec93a431c0 HTTP/1.0" 202 Accepted 2026-04-03 15:26:50,018 - lava_callback - INFO - Starting async job submit for node 69cd392974bda3ec93a431c0 2026-04-03 15:26:50,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:26:50,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd392974bda3ec93a431c0 HTTP/1.1" 200 None 2026-04-03 15:26:50,042 - lava_callback - DEBUG - Processing node 69cd392974bda3ec93a431c0: baseline-arm64-clabbe 2026-04-03 15:26:50,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd392974bda3ec93a431c0 2026-04-03 15:26:50,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:26:50,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd392974bda3ec93a431c0 HTTP/1.1" 200 None 2026-04-03 15:26:50,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:26:50,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:26:50,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:26:50,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd392974bda3ec93a431c0 HTTP/1.1" 200 None 2026-04-03 15:26:50,106 - lava_callback - ERROR - Error processing callback for node 69cd392974bda3ec93a431c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33784 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:28:10,051 - lava_callback - INFO - Received callback for node 69cd397e74bda3ec93a431cd INFO: 172.21.0.1:36564 - "POST /node/69cd397e74bda3ec93a431cd HTTP/1.0" 202 Accepted 2026-04-03 15:28:10,056 - lava_callback - INFO - Starting async job submit for node 69cd397e74bda3ec93a431cd 2026-04-03 15:28:10,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:28:10,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd397e74bda3ec93a431cd HTTP/1.1" 200 None 2026-04-03 15:28:10,072 - lava_callback - DEBUG - Processing node 69cd397e74bda3ec93a431cd: baseline-arm64-clabbe 2026-04-03 15:28:10,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd397e74bda3ec93a431cd 2026-04-03 15:28:10,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:28:10,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd397e74bda3ec93a431cd HTTP/1.1" 200 None 2026-04-03 15:28:10,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:28:10,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-03 15:28:10,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:28:10,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd397e74bda3ec93a431cd HTTP/1.1" 200 None 2026-04-03 15:28:10,133 - lava_callback - ERROR - Error processing callback for node 69cd397e74bda3ec93a431cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39352 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:29:21,657 - lava_callback - INFO - Received callback for node 69cd39c174bda3ec93a431dd INFO: 172.21.0.1:49096 - "POST /node/69cd39c174bda3ec93a431dd HTTP/1.0" 202 Accepted 2026-04-03 15:29:21,663 - lava_callback - INFO - Starting async job submit for node 69cd39c174bda3ec93a431dd 2026-04-03 15:29:21,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:29:21,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd39c174bda3ec93a431dd HTTP/1.1" 200 None 2026-04-03 15:29:21,682 - lava_callback - DEBUG - Processing node 69cd39c174bda3ec93a431dd: baseline-arm64-baylibre 2026-04-03 15:29:21,682 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd39c174bda3ec93a431dd 2026-04-03 15:29:21,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:29:21,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd39c174bda3ec93a431dd HTTP/1.1" 200 None 2026-04-03 15:29:21,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:29:21,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:29:21,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:29:21,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd39c174bda3ec93a431dd HTTP/1.1" 200 None 2026-04-03 15:29:21,734 - lava_callback - ERROR - Error processing callback for node 69cd39c174bda3ec93a431dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55876 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:34:42,058 - lava_callback - INFO - Received callback for node 69cd3b0b74bda3ec93a431fe INFO: 172.21.0.1:50794 - "POST /node/69cd3b0b74bda3ec93a431fe HTTP/1.0" 202 Accepted 2026-04-03 15:34:42,063 - lava_callback - INFO - Starting async job submit for node 69cd3b0b74bda3ec93a431fe 2026-04-03 15:34:42,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:34:42,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3b0b74bda3ec93a431fe HTTP/1.1" 200 None 2026-04-03 15:34:42,080 - lava_callback - DEBUG - Processing node 69cd3b0b74bda3ec93a431fe: baseline-arm64-baylibre 2026-04-03 15:34:42,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3b0b74bda3ec93a431fe 2026-04-03 15:34:42,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:34:42,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3b0b74bda3ec93a431fe HTTP/1.1" 200 None 2026-04-03 15:34:42,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:34:42,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:34:42,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:34:42,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3b0b74bda3ec93a431fe HTTP/1.1" 200 None 2026-04-03 15:34:42,164 - lava_callback - ERROR - Error processing callback for node 69cd3b0b74bda3ec93a431fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38590 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:36:02,013 - lava_callback - INFO - Received callback for node 69cd3b5f74bda3ec93a4320c INFO: 172.21.0.1:56906 - "POST /node/69cd3b5f74bda3ec93a4320c HTTP/1.0" 202 Accepted 2026-04-03 15:36:02,019 - lava_callback - INFO - Starting async job submit for node 69cd3b5f74bda3ec93a4320c 2026-04-03 15:36:02,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:36:02,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3b5f74bda3ec93a4320c HTTP/1.1" 200 None 2026-04-03 15:36:02,035 - lava_callback - DEBUG - Processing node 69cd3b5f74bda3ec93a4320c: baseline-arm64-baylibre 2026-04-03 15:36:02,035 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3b5f74bda3ec93a4320c 2026-04-03 15:36:02,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:36:02,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3b5f74bda3ec93a4320c HTTP/1.1" 200 None 2026-04-03 15:36:02,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:36:02,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:36:02,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:36:02,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3b5f74bda3ec93a4320c HTTP/1.1" 200 None 2026-04-03 15:36:02,097 - lava_callback - ERROR - Error processing callback for node 69cd3b5f74bda3ec93a4320c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55718 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:37:30,374 - lava_callback - INFO - Received callback for node 69cd3bb474bda3ec93a4321a INFO: 172.21.0.1:36788 - "POST /node/69cd3bb474bda3ec93a4321a HTTP/1.0" 202 Accepted 2026-04-03 15:37:30,380 - lava_callback - INFO - Starting async job submit for node 69cd3bb474bda3ec93a4321a 2026-04-03 15:37:30,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:37:30,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3bb474bda3ec93a4321a HTTP/1.1" 200 None 2026-04-03 15:37:30,397 - lava_callback - DEBUG - Processing node 69cd3bb474bda3ec93a4321a: baseline-arm64-clabbe 2026-04-03 15:37:30,397 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3bb474bda3ec93a4321a 2026-04-03 15:37:30,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:37:30,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3bb474bda3ec93a4321a HTTP/1.1" 200 None 2026-04-03 15:37:30,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:37:30,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:37:30,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:37:30,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3bb474bda3ec93a4321a HTTP/1.1" 200 None 2026-04-03 15:37:30,448 - lava_callback - ERROR - Error processing callback for node 69cd3bb474bda3ec93a4321a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59146 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:40:10,431 - lava_callback - INFO - Received callback for node 69cd3c5574bda3ec93a43233 INFO: 172.21.0.1:40956 - "POST /node/69cd3c5574bda3ec93a43233 HTTP/1.0" 202 Accepted 2026-04-03 15:40:10,436 - lava_callback - INFO - Starting async job submit for node 69cd3c5574bda3ec93a43233 2026-04-03 15:40:10,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:40:10,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3c5574bda3ec93a43233 HTTP/1.1" 200 None 2026-04-03 15:40:10,457 - lava_callback - DEBUG - Processing node 69cd3c5574bda3ec93a43233: baseline-arm64-clabbe 2026-04-03 15:40:10,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3c5574bda3ec93a43233 2026-04-03 15:40:10,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:40:10,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3c5574bda3ec93a43233 HTTP/1.1" 200 None 2026-04-03 15:40:10,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:40:10,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:40:10,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:40:10,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3c5574bda3ec93a43233 HTTP/1.1" 200 None 2026-04-03 15:40:10,526 - lava_callback - ERROR - Error processing callback for node 69cd3c5574bda3ec93a43233: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35644 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:41:42,363 - lava_callback - INFO - Received callback for node 69cd3ca374bda3ec93a43242 INFO: 172.21.0.1:55964 - "POST /node/69cd3ca374bda3ec93a43242 HTTP/1.0" 202 Accepted 2026-04-03 15:41:42,369 - lava_callback - INFO - Starting async job submit for node 69cd3ca374bda3ec93a43242 2026-04-03 15:41:42,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:41:42,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3ca374bda3ec93a43242 HTTP/1.1" 200 None 2026-04-03 15:41:42,385 - lava_callback - DEBUG - Processing node 69cd3ca374bda3ec93a43242: baseline-arm64-baylibre 2026-04-03 15:41:42,385 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3ca374bda3ec93a43242 2026-04-03 15:41:42,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:41:42,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3ca374bda3ec93a43242 HTTP/1.1" 200 None 2026-04-03 15:41:42,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:41:42,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:41:42,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:41:42,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3ca374bda3ec93a43242 HTTP/1.1" 200 None 2026-04-03 15:41:42,441 - lava_callback - ERROR - Error processing callback for node 69cd3ca374bda3ec93a43242: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55024 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:55030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46910 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:43:10,565 - lava_callback - INFO - Received callback for node 69cd3cff74bda3ec93a43253 INFO: 172.21.0.1:42848 - "POST /node/69cd3cff74bda3ec93a43253 HTTP/1.0" 202 Accepted 2026-04-03 15:43:10,570 - lava_callback - INFO - Starting async job submit for node 69cd3cff74bda3ec93a43253 2026-04-03 15:43:10,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:43:10,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3cff74bda3ec93a43253 HTTP/1.1" 200 None 2026-04-03 15:43:10,587 - lava_callback - DEBUG - Processing node 69cd3cff74bda3ec93a43253: baseline-arm64-clabbe 2026-04-03 15:43:10,587 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3cff74bda3ec93a43253 2026-04-03 15:43:10,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:43:10,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3cff74bda3ec93a43253 HTTP/1.1" 200 None 2026-04-03 15:43:10,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:43:10,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:43:10,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:43:10,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3cff74bda3ec93a43253 HTTP/1.1" 200 None 2026-04-03 15:43:10,657 - lava_callback - ERROR - Error processing callback for node 69cd3cff74bda3ec93a43253: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57056 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:44:30,678 - lava_callback - INFO - Received callback for node 69cd3d5374bda3ec93a43263 INFO: 172.21.0.1:56214 - "POST /node/69cd3d5374bda3ec93a43263 HTTP/1.0" 202 Accepted 2026-04-03 15:44:30,683 - lava_callback - INFO - Starting async job submit for node 69cd3d5374bda3ec93a43263 2026-04-03 15:44:30,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:44:30,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3d5374bda3ec93a43263 HTTP/1.1" 200 None 2026-04-03 15:44:30,698 - lava_callback - DEBUG - Processing node 69cd3d5374bda3ec93a43263: baseline-arm64-clabbe 2026-04-03 15:44:30,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3d5374bda3ec93a43263 2026-04-03 15:44:30,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:44:30,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3d5374bda3ec93a43263 HTTP/1.1" 200 None 2026-04-03 15:44:30,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:44:30,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:44:30,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:44:30,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3d5374bda3ec93a43263 HTTP/1.1" 200 None 2026-04-03 15:44:30,770 - lava_callback - ERROR - Error processing callback for node 69cd3d5374bda3ec93a43263: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58254 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:45:50,686 - lava_callback - INFO - Received callback for node 69cd3dab74bda3ec93a43273 INFO: 172.21.0.1:36400 - "POST /node/69cd3dab74bda3ec93a43273 HTTP/1.0" 202 Accepted 2026-04-03 15:45:50,691 - lava_callback - INFO - Starting async job submit for node 69cd3dab74bda3ec93a43273 2026-04-03 15:45:50,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:45:50,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3dab74bda3ec93a43273 HTTP/1.1" 200 None 2026-04-03 15:45:50,708 - lava_callback - DEBUG - Processing node 69cd3dab74bda3ec93a43273: baseline-arm64-clabbe 2026-04-03 15:45:50,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3dab74bda3ec93a43273 2026-04-03 15:45:50,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:45:50,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3dab74bda3ec93a43273 HTTP/1.1" 200 None 2026-04-03 15:45:50,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:45:50,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:45:50,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:45:50,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3dab74bda3ec93a43273 HTTP/1.1" 200 None 2026-04-03 15:45:50,771 - lava_callback - ERROR - Error processing callback for node 69cd3dab74bda3ec93a43273: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37366 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:47:30,738 - lava_callback - INFO - Received callback for node 69cd3dff74bda3ec93a43281 INFO: 172.21.0.1:60036 - "POST /node/69cd3dff74bda3ec93a43281 HTTP/1.0" 202 Accepted 2026-04-03 15:47:30,747 - lava_callback - INFO - Starting async job submit for node 69cd3dff74bda3ec93a43281 2026-04-03 15:47:30,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:47:30,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3dff74bda3ec93a43281 HTTP/1.1" 200 None 2026-04-03 15:47:30,763 - lava_callback - DEBUG - Processing node 69cd3dff74bda3ec93a43281: baseline-arm64-clabbe 2026-04-03 15:47:30,763 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3dff74bda3ec93a43281 2026-04-03 15:47:30,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:47:30,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3dff74bda3ec93a43281 HTTP/1.1" 200 None 2026-04-03 15:47:30,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:47:30,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:47:30,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:47:30,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3dff74bda3ec93a43281 HTTP/1.1" 200 None 2026-04-03 15:47:30,834 - lava_callback - ERROR - Error processing callback for node 69cd3dff74bda3ec93a43281: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55832 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:48:50,765 - lava_callback - INFO - Received callback for node 69cd3e5474bda3ec93a4328d INFO: 172.21.0.1:39470 - "POST /node/69cd3e5474bda3ec93a4328d HTTP/1.0" 202 Accepted 2026-04-03 15:48:50,770 - lava_callback - INFO - Starting async job submit for node 69cd3e5474bda3ec93a4328d 2026-04-03 15:48:50,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:48:50,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3e5474bda3ec93a4328d HTTP/1.1" 200 None 2026-04-03 15:48:50,787 - lava_callback - DEBUG - Processing node 69cd3e5474bda3ec93a4328d: baseline-arm64-clabbe 2026-04-03 15:48:50,787 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3e5474bda3ec93a4328d 2026-04-03 15:48:50,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:48:50,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3e5474bda3ec93a4328d HTTP/1.1" 200 None 2026-04-03 15:48:50,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:48:50,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:48:50,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:48:50,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3e5474bda3ec93a4328d HTTP/1.1" 200 None 2026-04-03 15:48:50,854 - lava_callback - ERROR - Error processing callback for node 69cd3e5474bda3ec93a4328d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45396 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:50:10,819 - lava_callback - INFO - Received callback for node 69cd3eaa74bda3ec93a43294 INFO: 172.21.0.1:42260 - "POST /node/69cd3eaa74bda3ec93a43294 HTTP/1.0" 202 Accepted 2026-04-03 15:50:10,825 - lava_callback - INFO - Starting async job submit for node 69cd3eaa74bda3ec93a43294 2026-04-03 15:50:10,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:50:10,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3eaa74bda3ec93a43294 HTTP/1.1" 200 None 2026-04-03 15:50:10,841 - lava_callback - DEBUG - Processing node 69cd3eaa74bda3ec93a43294: baseline-arm64-clabbe 2026-04-03 15:50:10,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3eaa74bda3ec93a43294 2026-04-03 15:50:10,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:50:10,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3eaa74bda3ec93a43294 HTTP/1.1" 200 None 2026-04-03 15:50:10,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:50:10,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:50:10,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:50:10,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3eaa74bda3ec93a43294 HTTP/1.1" 200 None 2026-04-03 15:50:10,904 - lava_callback - ERROR - Error processing callback for node 69cd3eaa74bda3ec93a43294: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42260 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:51:30,922 - lava_callback - INFO - Received callback for node 69cd3eff74bda3ec93a432a3 INFO: 172.21.0.1:35440 - "POST /node/69cd3eff74bda3ec93a432a3 HTTP/1.0" 202 Accepted 2026-04-03 15:51:30,928 - lava_callback - INFO - Starting async job submit for node 69cd3eff74bda3ec93a432a3 2026-04-03 15:51:30,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:51:30,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3eff74bda3ec93a432a3 HTTP/1.1" 200 None 2026-04-03 15:51:30,955 - lava_callback - DEBUG - Processing node 69cd3eff74bda3ec93a432a3: baseline-arm64-clabbe 2026-04-03 15:51:30,955 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3eff74bda3ec93a432a3 2026-04-03 15:51:30,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:51:30,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3eff74bda3ec93a432a3 HTTP/1.1" 200 None 2026-04-03 15:51:30,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:51:30,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:51:30,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:51:31,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3eff74bda3ec93a432a3 HTTP/1.1" 200 None 2026-04-03 15:51:31,019 - lava_callback - ERROR - Error processing callback for node 69cd3eff74bda3ec93a432a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32898 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:53:10,970 - lava_callback - INFO - Received callback for node 69cd3f5274bda3ec93a432ae INFO: 172.21.0.1:34474 - "POST /node/69cd3f5274bda3ec93a432ae HTTP/1.0" 202 Accepted 2026-04-03 15:53:10,975 - lava_callback - INFO - Starting async job submit for node 69cd3f5274bda3ec93a432ae 2026-04-03 15:53:10,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:53:10,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3f5274bda3ec93a432ae HTTP/1.1" 200 None 2026-04-03 15:53:10,991 - lava_callback - DEBUG - Processing node 69cd3f5274bda3ec93a432ae: baseline-arm64-clabbe 2026-04-03 15:53:10,991 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3f5274bda3ec93a432ae 2026-04-03 15:53:10,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:53:11,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3f5274bda3ec93a432ae HTTP/1.1" 200 None 2026-04-03 15:53:11,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:53:11,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:53:11,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:53:11,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3f5274bda3ec93a432ae HTTP/1.1" 200 None 2026-04-03 15:53:11,051 - lava_callback - ERROR - Error processing callback for node 69cd3f5274bda3ec93a432ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53836 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:54:31,072 - lava_callback - INFO - Received callback for node 69cd3fa974bda3ec93a432ba INFO: 172.21.0.1:50196 - "POST /node/69cd3fa974bda3ec93a432ba HTTP/1.0" 202 Accepted 2026-04-03 15:54:31,077 - lava_callback - INFO - Starting async job submit for node 69cd3fa974bda3ec93a432ba 2026-04-03 15:54:31,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:54:31,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3fa974bda3ec93a432ba HTTP/1.1" 200 None 2026-04-03 15:54:31,094 - lava_callback - DEBUG - Processing node 69cd3fa974bda3ec93a432ba: baseline-arm64-clabbe 2026-04-03 15:54:31,095 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3fa974bda3ec93a432ba 2026-04-03 15:54:31,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:54:31,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3fa974bda3ec93a432ba HTTP/1.1" 200 None 2026-04-03 15:54:31,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:54:31,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:54:31,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:54:31,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3fa974bda3ec93a432ba HTTP/1.1" 200 None 2026-04-03 15:54:31,155 - lava_callback - ERROR - Error processing callback for node 69cd3fa974bda3ec93a432ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59228 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:55:51,126 - lava_callback - INFO - Received callback for node 69cd3ffe74bda3ec93a432c7 INFO: 172.21.0.1:39870 - "POST /node/69cd3ffe74bda3ec93a432c7 HTTP/1.0" 202 Accepted 2026-04-03 15:55:51,131 - lava_callback - INFO - Starting async job submit for node 69cd3ffe74bda3ec93a432c7 2026-04-03 15:55:51,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:55:51,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3ffe74bda3ec93a432c7 HTTP/1.1" 200 None 2026-04-03 15:55:51,148 - lava_callback - DEBUG - Processing node 69cd3ffe74bda3ec93a432c7: baseline-arm64-clabbe 2026-04-03 15:55:51,148 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd3ffe74bda3ec93a432c7 2026-04-03 15:55:51,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:55:51,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd3ffe74bda3ec93a432c7 HTTP/1.1" 200 None 2026-04-03 15:55:51,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:55:51,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ca31ff0e33d030878a87ff HTTP/1.1" 200 None 2026-04-03 15:55:51,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:55:51,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd3ffe74bda3ec93a432c7 HTTP/1.1" 200 None 2026-04-03 15:55:51,209 - lava_callback - ERROR - Error processing callback for node 69cd3ffe74bda3ec93a432c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42302 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:57:03,257 - lava_callback - INFO - Received callback for node 69cd404974bda3ec93a432cf INFO: 172.21.0.1:47690 - "POST /node/69cd404974bda3ec93a432cf HTTP/1.0" 202 Accepted 2026-04-03 15:57:03,262 - lava_callback - INFO - Starting async job submit for node 69cd404974bda3ec93a432cf 2026-04-03 15:57:03,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:57:03,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd404974bda3ec93a432cf HTTP/1.1" 200 None 2026-04-03 15:57:03,278 - lava_callback - DEBUG - Processing node 69cd404974bda3ec93a432cf: baseline-arm64-baylibre 2026-04-03 15:57:03,278 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd404974bda3ec93a432cf 2026-04-03 15:57:03,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:57:03,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd404974bda3ec93a432cf HTTP/1.1" 200 None 2026-04-03 15:57:03,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:57:03,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 15:57:03,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:57:03,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd404974bda3ec93a432cf HTTP/1.1" 200 None 2026-04-03 15:57:03,329 - lava_callback - ERROR - Error processing callback for node 69cd404974bda3ec93a432cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49390 - "GET / HTTP/1.0" 200 OK 2026-04-03 15:58:43,306 - lava_callback - INFO - Received callback for node 69cd409f74bda3ec93a432dc INFO: 172.21.0.1:43040 - "POST /node/69cd409f74bda3ec93a432dc HTTP/1.0" 202 Accepted 2026-04-03 15:58:43,314 - lava_callback - INFO - Starting async job submit for node 69cd409f74bda3ec93a432dc 2026-04-03 15:58:43,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:58:43,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd409f74bda3ec93a432dc HTTP/1.1" 200 None 2026-04-03 15:58:43,342 - lava_callback - DEBUG - Processing node 69cd409f74bda3ec93a432dc: baseline-arm64-baylibre 2026-04-03 15:58:43,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd409f74bda3ec93a432dc 2026-04-03 15:58:43,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:58:43,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd409f74bda3ec93a432dc HTTP/1.1" 200 None 2026-04-03 15:58:43,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:58:43,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 15:58:43,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 15:58:43,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd409f74bda3ec93a432dc HTTP/1.1" 200 None 2026-04-03 15:58:43,422 - lava_callback - ERROR - Error processing callback for node 69cd409f74bda3ec93a432dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40916 - "GET / HTTP/1.0" 200 OK 2026-04-03 16:00:03,362 - lava_callback - INFO - Received callback for node 69cd40ef74bda3ec93a432e8 INFO: 172.21.0.1:58224 - "POST /node/69cd40ef74bda3ec93a432e8 HTTP/1.0" 202 Accepted 2026-04-03 16:00:03,370 - lava_callback - INFO - Starting async job submit for node 69cd40ef74bda3ec93a432e8 2026-04-03 16:00:03,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:00:03,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd40ef74bda3ec93a432e8 HTTP/1.1" 200 None 2026-04-03 16:00:03,391 - lava_callback - DEBUG - Processing node 69cd40ef74bda3ec93a432e8: baseline-arm64-baylibre 2026-04-03 16:00:03,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd40ef74bda3ec93a432e8 2026-04-03 16:00:03,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:00:03,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd40ef74bda3ec93a432e8 HTTP/1.1" 200 None 2026-04-03 16:00:03,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:00:03,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 16:00:03,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:00:03,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd40ef74bda3ec93a432e8 HTTP/1.1" 200 None 2026-04-03 16:00:03,447 - lava_callback - ERROR - Error processing callback for node 69cd40ef74bda3ec93a432e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49576 - "GET / HTTP/1.0" 200 OK 2026-04-03 16:01:31,325 - lava_callback - INFO - Received callback for node 69cd414f74bda3ec93a432f5 INFO: 172.21.0.1:59430 - "POST /node/69cd414f74bda3ec93a432f5 HTTP/1.0" 202 Accepted 2026-04-03 16:01:31,331 - lava_callback - INFO - Starting async job submit for node 69cd414f74bda3ec93a432f5 2026-04-03 16:01:31,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:01:31,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd414f74bda3ec93a432f5 HTTP/1.1" 200 None 2026-04-03 16:01:31,347 - lava_callback - DEBUG - Processing node 69cd414f74bda3ec93a432f5: baseline-arm64-clabbe 2026-04-03 16:01:31,347 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd414f74bda3ec93a432f5 2026-04-03 16:01:31,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:01:31,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd414f74bda3ec93a432f5 HTTP/1.1" 200 None 2026-04-03 16:01:31,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:01:31,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 16:01:31,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:01:31,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd414f74bda3ec93a432f5 HTTP/1.1" 200 None 2026-04-03 16:01:31,405 - lava_callback - ERROR - Error processing callback for node 69cd414f74bda3ec93a432f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46248 - "GET / HTTP/1.0" 200 OK 2026-04-03 16:02:43,544 - lava_callback - INFO - Received callback for node 69cd419b74bda3ec93a432fb INFO: 172.21.0.1:60984 - "POST /node/69cd419b74bda3ec93a432fb HTTP/1.0" 202 Accepted 2026-04-03 16:02:43,550 - lava_callback - INFO - Starting async job submit for node 69cd419b74bda3ec93a432fb 2026-04-03 16:02:43,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:02:43,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd419b74bda3ec93a432fb HTTP/1.1" 200 None 2026-04-03 16:02:43,566 - lava_callback - DEBUG - Processing node 69cd419b74bda3ec93a432fb: baseline-arm64-baylibre 2026-04-03 16:02:43,566 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd419b74bda3ec93a432fb 2026-04-03 16:02:43,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:02:43,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd419b74bda3ec93a432fb HTTP/1.1" 200 None 2026-04-03 16:02:43,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:02:43,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 16:02:43,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:02:43,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd419b74bda3ec93a432fb HTTP/1.1" 200 None 2026-04-03 16:02:43,623 - lava_callback - ERROR - Error processing callback for node 69cd419b74bda3ec93a432fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47686 - "GET / HTTP/1.0" 200 OK 2026-04-03 16:04:03,585 - lava_callback - INFO - Received callback for node 69cd41ee74bda3ec93a43307 INFO: 172.21.0.1:41608 - "POST /node/69cd41ee74bda3ec93a43307 HTTP/1.0" 202 Accepted 2026-04-03 16:04:03,591 - lava_callback - INFO - Starting async job submit for node 69cd41ee74bda3ec93a43307 2026-04-03 16:04:03,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:04:03,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd41ee74bda3ec93a43307 HTTP/1.1" 200 None 2026-04-03 16:04:03,607 - lava_callback - DEBUG - Processing node 69cd41ee74bda3ec93a43307: baseline-arm64-baylibre 2026-04-03 16:04:03,607 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd41ee74bda3ec93a43307 2026-04-03 16:04:03,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:04:03,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd41ee74bda3ec93a43307 HTTP/1.1" 200 None 2026-04-03 16:04:03,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:04:03,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 16:04:03,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:04:03,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd41ee74bda3ec93a43307 HTTP/1.1" 200 None 2026-04-03 16:04:03,687 - lava_callback - ERROR - Error processing callback for node 69cd41ee74bda3ec93a43307: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33942 - "GET / HTTP/1.0" 200 OK 2026-04-03 16:05:51,516 - lava_callback - INFO - Received callback for node 69cd424d74bda3ec93a43311 INFO: 172.21.0.1:59540 - "POST /node/69cd424d74bda3ec93a43311 HTTP/1.0" 202 Accepted 2026-04-03 16:05:51,521 - lava_callback - INFO - Starting async job submit for node 69cd424d74bda3ec93a43311 2026-04-03 16:05:51,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:05:51,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd424d74bda3ec93a43311 HTTP/1.1" 200 None 2026-04-03 16:05:51,537 - lava_callback - DEBUG - Processing node 69cd424d74bda3ec93a43311: baseline-arm64-clabbe 2026-04-03 16:05:51,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd424d74bda3ec93a43311 2026-04-03 16:05:51,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:05:51,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd424d74bda3ec93a43311 HTTP/1.1" 200 None 2026-04-03 16:05:51,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:05:51,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 16:05:51,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:05:51,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd424d74bda3ec93a43311 HTTP/1.1" 200 None 2026-04-03 16:05:51,592 - lava_callback - ERROR - Error processing callback for node 69cd424d74bda3ec93a43311: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59986 - "GET / HTTP/1.0" 200 OK 2026-04-03 16:07:03,652 - lava_callback - INFO - Received callback for node 69cd429a74bda3ec93a4331c INFO: 172.21.0.1:42518 - "POST /node/69cd429a74bda3ec93a4331c HTTP/1.0" 202 Accepted 2026-04-03 16:07:03,658 - lava_callback - INFO - Starting async job submit for node 69cd429a74bda3ec93a4331c 2026-04-03 16:07:03,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:07:03,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd429a74bda3ec93a4331c HTTP/1.1" 200 None 2026-04-03 16:07:03,683 - lava_callback - DEBUG - Processing node 69cd429a74bda3ec93a4331c: baseline-arm64-baylibre 2026-04-03 16:07:03,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd429a74bda3ec93a4331c 2026-04-03 16:07:03,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:07:03,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd429a74bda3ec93a4331c HTTP/1.1" 200 None 2026-04-03 16:07:03,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:07:03,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 16:07:03,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:07:03,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd429a74bda3ec93a4331c HTTP/1.1" 200 None 2026-04-03 16:07:03,741 - lava_callback - ERROR - Error processing callback for node 69cd429a74bda3ec93a4331c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43744 - "GET / HTTP/1.0" 200 OK 2026-04-03 16:08:23,759 - lava_callback - INFO - Received callback for node 69cd42eb74bda3ec93a43327 INFO: 172.21.0.1:41896 - "POST /node/69cd42eb74bda3ec93a43327 HTTP/1.0" 202 Accepted 2026-04-03 16:08:23,764 - lava_callback - INFO - Starting async job submit for node 69cd42eb74bda3ec93a43327 2026-04-03 16:08:23,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:08:23,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd42eb74bda3ec93a43327 HTTP/1.1" 200 None 2026-04-03 16:08:23,781 - lava_callback - DEBUG - Processing node 69cd42eb74bda3ec93a43327: baseline-arm64-baylibre 2026-04-03 16:08:23,781 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd42eb74bda3ec93a43327 2026-04-03 16:08:23,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:08:23,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd42eb74bda3ec93a43327 HTTP/1.1" 200 None 2026-04-03 16:08:23,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:08:23,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 16:08:23,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:08:23,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd42eb74bda3ec93a43327 HTTP/1.1" 200 None 2026-04-03 16:08:23,837 - lava_callback - ERROR - Error processing callback for node 69cd42eb74bda3ec93a43327: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45842 - "GET / HTTP/1.0" 200 OK 2026-04-03 16:09:43,833 - lava_callback - INFO - Received callback for node 69cd433c74bda3ec93a4332d INFO: 172.21.0.1:54092 - "POST /node/69cd433c74bda3ec93a4332d HTTP/1.0" 202 Accepted 2026-04-03 16:09:43,838 - lava_callback - INFO - Starting async job submit for node 69cd433c74bda3ec93a4332d 2026-04-03 16:09:43,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:09:43,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd433c74bda3ec93a4332d HTTP/1.1" 200 None 2026-04-03 16:09:43,856 - lava_callback - DEBUG - Processing node 69cd433c74bda3ec93a4332d: baseline-arm64-baylibre 2026-04-03 16:09:43,856 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd433c74bda3ec93a4332d 2026-04-03 16:09:43,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:09:43,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd433c74bda3ec93a4332d HTTP/1.1" 200 None 2026-04-03 16:09:43,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:09:43,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 16:09:43,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:09:43,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd433c74bda3ec93a4332d HTTP/1.1" 200 None 2026-04-03 16:09:43,913 - lava_callback - ERROR - Error processing callback for node 69cd433c74bda3ec93a4332d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47642 - "GET / HTTP/1.0" 200 OK 2026-04-03 16:11:03,876 - lava_callback - INFO - Received callback for node 69cd438d74bda3ec93a43337 INFO: 172.21.0.1:36540 - "POST /node/69cd438d74bda3ec93a43337 HTTP/1.0" 202 Accepted 2026-04-03 16:11:03,882 - lava_callback - INFO - Starting async job submit for node 69cd438d74bda3ec93a43337 2026-04-03 16:11:03,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:11:03,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd438d74bda3ec93a43337 HTTP/1.1" 200 None 2026-04-03 16:11:03,923 - lava_callback - DEBUG - Processing node 69cd438d74bda3ec93a43337: baseline-arm64-baylibre 2026-04-03 16:11:03,924 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd438d74bda3ec93a43337 2026-04-03 16:11:03,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:11:03,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd438d74bda3ec93a43337 HTTP/1.1" 200 None 2026-04-03 16:11:03,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:11:03,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 16:11:03,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:11:03,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd438d74bda3ec93a43337 HTTP/1.1" 200 None 2026-04-03 16:11:03,979 - lava_callback - ERROR - Error processing callback for node 69cd438d74bda3ec93a43337: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42424 - "GET / HTTP/1.0" 200 OK 2026-04-03 16:12:23,968 - lava_callback - INFO - Received callback for node 69cd43e174bda3ec93a43344 INFO: 172.21.0.1:44326 - "POST /node/69cd43e174bda3ec93a43344 HTTP/1.0" 202 Accepted 2026-04-03 16:12:23,974 - lava_callback - INFO - Starting async job submit for node 69cd43e174bda3ec93a43344 2026-04-03 16:12:23,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:12:23,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd43e174bda3ec93a43344 HTTP/1.1" 200 None 2026-04-03 16:12:23,989 - lava_callback - DEBUG - Processing node 69cd43e174bda3ec93a43344: baseline-arm64-baylibre 2026-04-03 16:12:23,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cd43e174bda3ec93a43344 2026-04-03 16:12:23,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:12:24,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cd43e174bda3ec93a43344 HTTP/1.1" 200 None 2026-04-03 16:12:24,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:12:24,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c78eb9be430e18c5d159ee HTTP/1.1" 200 None 2026-04-03 16:12:24,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 16:12:24,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cd43e174bda3ec93a43344 HTTP/1.1" 200 None 2026-04-03 16:12:24,055 - lava_callback - ERROR - Error processing callback for node 69cd43e174bda3ec93a43344: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]