INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:35852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46016 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:15:54,008 - lava_callback - INFO - Received callback for node 69bfa5257544ab5cd4a7d9b6 2026-03-24 08:15:54,015 - lava_callback - INFO - Starting async job submit for node 69bfa5257544ab5cd4a7d9b6 INFO: 172.20.0.1:47750 - "POST /node/69bfa5257544ab5cd4a7d9b6 HTTP/1.0" 202 Accepted 2026-03-24 08:15:54,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:15:54,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5257544ab5cd4a7d9b6 HTTP/1.1" 200 None 2026-03-24 08:15:54,048 - lava_callback - DEBUG - Processing node 69bfa5257544ab5cd4a7d9b6: baseline-arm64-clabbe 2026-03-24 08:15:54,049 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa5257544ab5cd4a7d9b6 2026-03-24 08:15:54,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:15:54,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5257544ab5cd4a7d9b6 HTTP/1.1" 200 None 2026-03-24 08:15:54,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:15:54,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 08:15:54,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:15:54,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa5257544ab5cd4a7d9b6 HTTP/1.1" 200 None 2026-03-24 08:15:54,161 - lava_callback - ERROR - Error processing callback for node 69bfa5257544ab5cd4a7d9b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36210 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:16:46,182 - lava_callback - INFO - Received callback for node 69c1e3b40efaf21d93106084 INFO: 172.20.0.1:56348 - "POST /node/69c1e3b40efaf21d93106084 HTTP/1.0" 202 Accepted 2026-03-24 08:16:46,189 - lava_callback - INFO - Starting async job submit for node 69c1e3b40efaf21d93106084 2026-03-24 08:16:46,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:16:46,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b40efaf21d93106084 HTTP/1.1" 200 None 2026-03-24 08:16:46,252 - lava_callback - DEBUG - Processing node 69c1e3b40efaf21d93106084: baseline-arm64-broonie 2026-03-24 08:16:46,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:56362 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:16:46,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:16:46,631 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3b40efaf21d93106084/log.txt.gz 2026-03-24 08:16:46,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:16:47,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:16:47,002 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3b40efaf21d93106084/lava-logs.yaml 2026-03-24 08:16:47,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:16:47,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:16:47,170 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3b40efaf21d93106084/lava_callback.json.gz 2026-03-24 08:16:47,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:16:47,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3b40efaf21d93106084 HTTP/1.1" 200 None 2026-03-24 08:16:47,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:16:47,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 08:16:47,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:16:47,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3b40efaf21d93106084 HTTP/1.1" 200 None 2026-03-24 08:16:47,329 - lava_callback - INFO - Completed processing callback for node 69c1e3b40efaf21d93106084 2026-03-24 08:16:54,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:16:54,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56322 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:17:48,443 - lava_callback - INFO - Received callback for node 69c23bd50efaf21d93107f40 INFO: 172.20.0.1:56336 - "POST /node/69c23bd50efaf21d93107f40 HTTP/1.0" 202 Accepted 2026-03-24 08:17:48,450 - lava_callback - INFO - Starting async job submit for node 69c23bd50efaf21d93107f40 2026-03-24 08:17:48,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:17:48,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23bd50efaf21d93107f40 HTTP/1.1" 200 None 2026-03-24 08:17:48,501 - lava_callback - DEBUG - Processing node 69c23bd50efaf21d93107f40: baseline-arm64-broonie 2026-03-24 08:17:48,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:17:48,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:17:48,933 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c23bd50efaf21d93107f40/log.txt.gz 2026-03-24 08:17:49,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:17:49,300 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:17:49,303 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c23bd50efaf21d93107f40/lava-logs.yaml 2026-03-24 08:17:49,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:17:49,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:17:49,454 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c23bd50efaf21d93107f40/lava_callback.json.gz 2026-03-24 08:17:49,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:17:49,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23bd50efaf21d93107f40 HTTP/1.1" 200 None 2026-03-24 08:17:49,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:17:49,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 08:17:49,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:17:49,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23bd50efaf21d93107f40 HTTP/1.1" 200 None 2026-03-24 08:17:49,635 - lava_callback - INFO - Completed processing callback for node 69c23bd50efaf21d93107f40 2026-03-24 08:17:54,027 - lava_callback - INFO - Received callback for node 69bfa59d7544ab5cd4a7da02 INFO: 172.20.0.1:60472 - "POST /node/69bfa59d7544ab5cd4a7da02 HTTP/1.0" 202 Accepted 2026-03-24 08:17:54,033 - lava_callback - INFO - Starting async job submit for node 69bfa59d7544ab5cd4a7da02 2026-03-24 08:17:54,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:17:54,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa59d7544ab5cd4a7da02 HTTP/1.1" 200 None 2026-03-24 08:17:54,060 - lava_callback - DEBUG - Processing node 69bfa59d7544ab5cd4a7da02: baseline-arm64-clabbe 2026-03-24 08:17:54,060 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa59d7544ab5cd4a7da02 2026-03-24 08:17:54,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:17:54,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa59d7544ab5cd4a7da02 HTTP/1.1" 200 None 2026-03-24 08:17:54,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:17:54,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 08:17:54,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:17:54,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa59d7544ab5cd4a7da02 HTTP/1.1" 200 None 2026-03-24 08:17:54,134 - lava_callback - ERROR - Error processing callback for node 69bfa59d7544ab5cd4a7da02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:17:54,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:17:54,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60898 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:20:54,222 - lava_callback - INFO - Received callback for node 69bfa6527544ab5cd4a7da20 INFO: 172.20.0.1:36902 - "POST /node/69bfa6527544ab5cd4a7da20 HTTP/1.0" 202 Accepted 2026-03-24 08:20:54,227 - lava_callback - INFO - Starting async job submit for node 69bfa6527544ab5cd4a7da20 2026-03-24 08:20:54,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:20:54,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa6527544ab5cd4a7da20 HTTP/1.1" 200 None 2026-03-24 08:20:54,244 - lava_callback - DEBUG - Processing node 69bfa6527544ab5cd4a7da20: baseline-arm64-clabbe 2026-03-24 08:20:54,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa6527544ab5cd4a7da20 2026-03-24 08:20:54,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:20:54,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa6527544ab5cd4a7da20 HTTP/1.1" 200 None 2026-03-24 08:20:54,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:20:54,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 08:20:54,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:20:54,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa6527544ab5cd4a7da20 HTTP/1.1" 200 None 2026-03-24 08:20:54,328 - lava_callback - ERROR - Error processing callback for node 69bfa6527544ab5cd4a7da20: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56164 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:22:54,247 - lava_callback - INFO - Received callback for node 69bfa6ca7544ab5cd4a7da30 INFO: 172.20.0.1:56454 - "POST /node/69bfa6ca7544ab5cd4a7da30 HTTP/1.0" 202 Accepted 2026-03-24 08:22:54,252 - lava_callback - INFO - Starting async job submit for node 69bfa6ca7544ab5cd4a7da30 2026-03-24 08:22:54,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:22:54,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa6ca7544ab5cd4a7da30 HTTP/1.1" 200 None 2026-03-24 08:22:54,268 - lava_callback - DEBUG - Processing node 69bfa6ca7544ab5cd4a7da30: baseline-arm64-clabbe 2026-03-24 08:22:54,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa6ca7544ab5cd4a7da30 2026-03-24 08:22:54,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:22:54,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa6ca7544ab5cd4a7da30 HTTP/1.1" 200 None 2026-03-24 08:22:54,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:22:54,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 08:22:54,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:22:54,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa6ca7544ab5cd4a7da30 HTTP/1.1" 200 None 2026-03-24 08:22:54,335 - lava_callback - ERROR - Error processing callback for node 69bfa6ca7544ab5cd4a7da30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37146 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:23:54,330 - lava_callback - INFO - Received callback for node 69bfa7157544ab5cd4a7da48 INFO: 172.20.0.1:50342 - "POST /node/69bfa7157544ab5cd4a7da48 HTTP/1.0" 202 Accepted 2026-03-24 08:23:54,335 - lava_callback - INFO - Starting async job submit for node 69bfa7157544ab5cd4a7da48 2026-03-24 08:23:54,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:23:54,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa7157544ab5cd4a7da48 HTTP/1.1" 200 None 2026-03-24 08:23:54,373 - lava_callback - DEBUG - Processing node 69bfa7157544ab5cd4a7da48: baseline-arm64-clabbe 2026-03-24 08:23:54,374 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfa7157544ab5cd4a7da48 2026-03-24 08:23:54,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:23:54,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa7157544ab5cd4a7da48 HTTP/1.1" 200 None 2026-03-24 08:23:54,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:23:54,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc21b717e5cb936602bc2b HTTP/1.1" 200 None 2026-03-24 08:23:54,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:23:54,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfa7157544ab5cd4a7da48 HTTP/1.1" 200 None 2026-03-24 08:23:54,441 - lava_callback - ERROR - Error processing callback for node 69bfa7157544ab5cd4a7da48: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50372 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41156 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:33:27,346 - lava_callback - INFO - Received callback for node 69c24bd74a2c5046eb87f906 INFO: 172.20.0.1:41164 - "POST /node/69c24bd74a2c5046eb87f906 HTTP/1.0" 202 Accepted 2026-03-24 08:33:27,355 - lava_callback - INFO - Starting async job submit for node 69c24bd74a2c5046eb87f906 2026-03-24 08:33:27,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:33:27,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24bd74a2c5046eb87f906 HTTP/1.1" 200 None 2026-03-24 08:33:27,447 - lava_callback - DEBUG - Processing node 69c24bd74a2c5046eb87f906: baseline-arm 2026-03-24 08:33:27,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:33:28,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:33:28,203 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c24bd74a2c5046eb87f906/log.txt.gz 2026-03-24 08:33:28,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:33:28,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:33:28,953 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c24bd74a2c5046eb87f906/lava-logs.yaml 2026-03-24 08:33:28,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:33:29,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:33:29,377 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c24bd74a2c5046eb87f906/lava_callback.json.gz 2026-03-24 08:33:29,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:33:29,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24bd74a2c5046eb87f906 HTTP/1.1" 200 None 2026-03-24 08:33:29,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:33:29,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248de4a2c5046eb87f896 HTTP/1.1" 200 None 2026-03-24 08:33:29,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:33:29,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24bd74a2c5046eb87f906 HTTP/1.1" 200 None 2026-03-24 08:33:29,520 - lava_callback - INFO - Completed processing callback for node 69c24bd74a2c5046eb87f906 INFO: 172.20.0.1:45220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36834 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:33:54,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:33:54,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56102 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:37:54,250 - lava_callback - INFO - Received callback for node 69c24d474a2c5046eb87ff06 INFO: 172.20.0.1:39144 - "POST /node/69c24d474a2c5046eb87ff06 HTTP/1.0" 202 Accepted 2026-03-24 08:37:54,256 - lava_callback - INFO - Starting async job submit for node 69c24d474a2c5046eb87ff06 2026-03-24 08:37:54,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:37:54,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d474a2c5046eb87ff06 HTTP/1.1" 200 None 2026-03-24 08:37:54,289 - lava_callback - DEBUG - Processing node 69c24d474a2c5046eb87ff06: baseline-x86 2026-03-24 08:37:54,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:37:54,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:37:54,615 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d474a2c5046eb87ff06/log.txt.gz 2026-03-24 08:37:54,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:37:54,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:37:54,960 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d474a2c5046eb87ff06/lava-logs.yaml 2026-03-24 08:37:54,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:37:55,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:37:55,090 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d474a2c5046eb87ff06/lava_callback.json.gz 2026-03-24 08:37:55,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:37:55,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d474a2c5046eb87ff06 HTTP/1.1" 200 None 2026-03-24 08:37:55,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:37:55,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248fb4a2c5046eb87f8bb HTTP/1.1" 200 None 2026-03-24 08:37:55,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:37:55,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24d474a2c5046eb87ff06 HTTP/1.1" 200 None 2026-03-24 08:37:55,179 - lava_callback - INFO - Completed processing callback for node 69c24d474a2c5046eb87ff06 INFO: 172.20.0.1:39150 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:38:02,080 - lava_callback - INFO - Received callback for node 69c24d4a4a2c5046eb87ff0f INFO: 172.20.0.1:53046 - "POST /node/69c24d4a4a2c5046eb87ff0f HTTP/1.0" 202 Accepted 2026-03-24 08:38:02,086 - lava_callback - INFO - Starting async job submit for node 69c24d4a4a2c5046eb87ff0f 2026-03-24 08:38:02,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:38:02,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d4a4a2c5046eb87ff0f HTTP/1.1" 200 None 2026-03-24 08:38:02,119 - lava_callback - DEBUG - Processing node 69c24d4a4a2c5046eb87ff0f: baseline-x86 2026-03-24 08:38:02,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:38:02,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:38:02,413 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d4a4a2c5046eb87ff0f/log.txt.gz 2026-03-24 08:38:02,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:38:02,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:38:02,724 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d4a4a2c5046eb87ff0f/lava-logs.yaml 2026-03-24 08:38:02,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:38:02,865 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:38:02,867 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d4a4a2c5046eb87ff0f/lava_callback.json.gz 2026-03-24 08:38:02,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:38:02,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d4a4a2c5046eb87ff0f HTTP/1.1" 200 None 2026-03-24 08:38:02,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:38:02,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248fb4a2c5046eb87f8bb HTTP/1.1" 200 None 2026-03-24 08:38:02,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:38:02,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24d4a4a2c5046eb87ff0f HTTP/1.1" 200 None 2026-03-24 08:38:02,956 - lava_callback - INFO - Completed processing callback for node 69c24d4a4a2c5046eb87ff0f INFO: 172.20.0.1:53050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60514 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:38:24,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:38:24,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58992 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:38:38,603 - lava_callback - INFO - Received callback for node 69bfaa817544ab5cd4a7e031 INFO: 172.20.0.1:59000 - "POST /node/69bfaa817544ab5cd4a7e031 HTTP/1.0" 202 Accepted 2026-03-24 08:38:38,609 - lava_callback - INFO - Starting async job submit for node 69bfaa817544ab5cd4a7e031 2026-03-24 08:38:38,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:38:38,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfaa817544ab5cd4a7e031 HTTP/1.1" 200 None 2026-03-24 08:38:38,645 - lava_callback - DEBUG - Processing node 69bfaa817544ab5cd4a7e031: baseline-arm-baylibre 2026-03-24 08:38:38,645 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfaa817544ab5cd4a7e031 2026-03-24 08:38:38,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:38:38,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfaa817544ab5cd4a7e031 HTTP/1.1" 200 None 2026-03-24 08:38:38,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:38:38,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 08:38:38,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:38:38,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfaa817544ab5cd4a7e031 HTTP/1.1" 200 None 2026-03-24 08:38:38,710 - lava_callback - ERROR - Error processing callback for node 69bfaa817544ab5cd4a7e031: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37978 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:39:14,167 - lava_callback - INFO - Received callback for node 69c24d494a2c5046eb87ff0c INFO: 172.20.0.1:34706 - "POST /node/69c24d494a2c5046eb87ff0c HTTP/1.0" 202 Accepted 2026-03-24 08:39:14,174 - lava_callback - INFO - Starting async job submit for node 69c24d494a2c5046eb87ff0c 2026-03-24 08:39:14,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:39:14,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d494a2c5046eb87ff0c HTTP/1.1" 200 None 2026-03-24 08:39:14,215 - lava_callback - DEBUG - Processing node 69c24d494a2c5046eb87ff0c: baseline-x86 2026-03-24 08:39:14,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:39:14,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:39:14,605 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d494a2c5046eb87ff0c/log.txt.gz 2026-03-24 08:39:14,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:39:14,976 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:39:14,978 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d494a2c5046eb87ff0c/lava-logs.yaml 2026-03-24 08:39:14,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:39:15,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:39:15,145 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d494a2c5046eb87ff0c/lava_callback.json.gz 2026-03-24 08:39:15,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:34708 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 08:39:15,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d494a2c5046eb87ff0c HTTP/1.1" 200 None 2026-03-24 08:39:15,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:39:15,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248fb4a2c5046eb87f8bb HTTP/1.1" 200 None 2026-03-24 08:39:15,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:39:15,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24d494a2c5046eb87ff0c HTTP/1.1" 200 None 2026-03-24 08:39:15,227 - lava_callback - INFO - Completed processing callback for node 69c24d494a2c5046eb87ff0c INFO: 172.20.0.1:34724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34734 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:39:24,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:39:24,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55898 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:39:38,269 - lava_callback - INFO - Received callback for node 69c24d484a2c5046eb87ff09 INFO: 172.20.0.1:55906 - "POST /node/69c24d484a2c5046eb87ff09 HTTP/1.0" 202 Accepted 2026-03-24 08:39:38,276 - lava_callback - INFO - Starting async job submit for node 69c24d484a2c5046eb87ff09 2026-03-24 08:39:38,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:39:38,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d484a2c5046eb87ff09 HTTP/1.1" 200 None 2026-03-24 08:39:38,310 - lava_callback - DEBUG - Processing node 69c24d484a2c5046eb87ff09: baseline-x86 2026-03-24 08:39:38,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:39:38,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:39:38,718 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d484a2c5046eb87ff09/log.txt.gz 2026-03-24 08:39:38,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:39:39,102 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:39:39,104 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d484a2c5046eb87ff09/lava-logs.yaml 2026-03-24 08:39:39,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:39:39,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:39:39,242 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c24d484a2c5046eb87ff09/lava_callback.json.gz 2026-03-24 08:39:39,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:39:39,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24d484a2c5046eb87ff09 HTTP/1.1" 200 None 2026-03-24 08:39:39,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:39:39,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248fb4a2c5046eb87f8bb HTTP/1.1" 200 None 2026-03-24 08:39:39,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:39:39,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24d484a2c5046eb87ff09 HTTP/1.1" 200 None 2026-03-24 08:39:39,321 - lava_callback - INFO - Completed processing callback for node 69c24d484a2c5046eb87ff09 INFO: 172.20.0.1:39496 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:39:54,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:39:54,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51912 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:40:18,721 - lava_callback - INFO - Received callback for node 69bfaae97544ab5cd4a7e063 INFO: 172.20.0.1:51918 - "POST /node/69bfaae97544ab5cd4a7e063 HTTP/1.0" 202 Accepted 2026-03-24 08:40:18,726 - lava_callback - INFO - Starting async job submit for node 69bfaae97544ab5cd4a7e063 2026-03-24 08:40:18,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:40:18,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfaae97544ab5cd4a7e063 HTTP/1.1" 200 None 2026-03-24 08:40:18,750 - lava_callback - DEBUG - Processing node 69bfaae97544ab5cd4a7e063: baseline-arm-baylibre 2026-03-24 08:40:18,750 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfaae97544ab5cd4a7e063 2026-03-24 08:40:18,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:40:18,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfaae97544ab5cd4a7e063 HTTP/1.1" 200 None 2026-03-24 08:40:18,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:40:18,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 08:40:18,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:40:18,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfaae97544ab5cd4a7e063 HTTP/1.1" 200 None 2026-03-24 08:40:18,815 - lava_callback - ERROR - Error processing callback for node 69bfaae97544ab5cd4a7e063: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32782 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:40:58,762 - lava_callback - INFO - Received callback for node 69bfab0e7544ab5cd4a7e082 INFO: 172.20.0.1:32790 - "POST /node/69bfab0e7544ab5cd4a7e082 HTTP/1.0" 202 Accepted 2026-03-24 08:40:58,770 - lava_callback - INFO - Starting async job submit for node 69bfab0e7544ab5cd4a7e082 2026-03-24 08:40:58,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:40:58,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfab0e7544ab5cd4a7e082 HTTP/1.1" 200 None 2026-03-24 08:40:58,797 - lava_callback - DEBUG - Processing node 69bfab0e7544ab5cd4a7e082: baseline-arm-baylibre 2026-03-24 08:40:58,797 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfab0e7544ab5cd4a7e082 2026-03-24 08:40:58,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:40:58,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfab0e7544ab5cd4a7e082 HTTP/1.1" 200 None 2026-03-24 08:40:58,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:40:58,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 08:40:58,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:40:58,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfab0e7544ab5cd4a7e082 HTTP/1.1" 200 None 2026-03-24 08:40:58,901 - lava_callback - ERROR - Error processing callback for node 69bfab0e7544ab5cd4a7e082: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60348 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:41:38,772 - lava_callback - INFO - Received callback for node 69bfab327544ab5cd4a7e097 INFO: 172.20.0.1:60362 - "POST /node/69bfab327544ab5cd4a7e097 HTTP/1.0" 202 Accepted 2026-03-24 08:41:38,780 - lava_callback - INFO - Starting async job submit for node 69bfab327544ab5cd4a7e097 2026-03-24 08:41:38,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:41:38,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfab327544ab5cd4a7e097 HTTP/1.1" 200 None 2026-03-24 08:41:38,814 - lava_callback - DEBUG - Processing node 69bfab327544ab5cd4a7e097: baseline-arm-baylibre 2026-03-24 08:41:38,814 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfab327544ab5cd4a7e097 2026-03-24 08:41:38,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:41:38,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfab327544ab5cd4a7e097 HTTP/1.1" 200 None 2026-03-24 08:41:38,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:41:38,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 08:41:38,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:41:38,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfab327544ab5cd4a7e097 HTTP/1.1" 200 None 2026-03-24 08:41:38,910 - lava_callback - ERROR - Error processing callback for node 69bfab327544ab5cd4a7e097: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59946 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:42:18,766 - lava_callback - INFO - Received callback for node 69bfab577544ab5cd4a7e0b0 INFO: 172.20.0.1:59954 - "POST /node/69bfab577544ab5cd4a7e0b0 HTTP/1.0" 202 Accepted 2026-03-24 08:42:18,775 - lava_callback - INFO - Starting async job submit for node 69bfab577544ab5cd4a7e0b0 2026-03-24 08:42:18,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:42:18,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfab577544ab5cd4a7e0b0 HTTP/1.1" 200 None 2026-03-24 08:42:18,804 - lava_callback - DEBUG - Processing node 69bfab577544ab5cd4a7e0b0: baseline-arm-baylibre 2026-03-24 08:42:18,804 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfab577544ab5cd4a7e0b0 2026-03-24 08:42:18,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:42:18,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfab577544ab5cd4a7e0b0 HTTP/1.1" 200 None 2026-03-24 08:42:18,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:42:18,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 08:42:18,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:42:18,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfab577544ab5cd4a7e0b0 HTTP/1.1" 200 None 2026-03-24 08:42:18,913 - lava_callback - ERROR - Error processing callback for node 69bfab577544ab5cd4a7e0b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42652 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:42:38,774 - lava_callback - INFO - Received callback for node 69bfab7b7544ab5cd4a7e0bf INFO: 172.20.0.1:42654 - "POST /node/69bfab7b7544ab5cd4a7e0bf HTTP/1.0" 202 Accepted 2026-03-24 08:42:38,783 - lava_callback - INFO - Starting async job submit for node 69bfab7b7544ab5cd4a7e0bf 2026-03-24 08:42:38,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:42:38,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfab7b7544ab5cd4a7e0bf HTTP/1.1" 200 None 2026-03-24 08:42:38,824 - lava_callback - DEBUG - Processing node 69bfab7b7544ab5cd4a7e0bf: baseline-arm-baylibre 2026-03-24 08:42:38,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfab7b7544ab5cd4a7e0bf 2026-03-24 08:42:38,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:42:38,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfab7b7544ab5cd4a7e0bf HTTP/1.1" 200 None 2026-03-24 08:42:38,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:42:38,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 08:42:38,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:42:38,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfab7b7544ab5cd4a7e0bf HTTP/1.1" 200 None 2026-03-24 08:42:38,924 - lava_callback - ERROR - Error processing callback for node 69bfab7b7544ab5cd4a7e0bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35084 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:43:24,770 - lava_callback - INFO - Received callback for node 69c24ea34a2c5046eb87ff8e INFO: 172.20.0.1:39912 - "POST /node/69c24ea34a2c5046eb87ff8e HTTP/1.0" 202 Accepted 2026-03-24 08:43:24,778 - lava_callback - INFO - Starting async job submit for node 69c24ea34a2c5046eb87ff8e 2026-03-24 08:43:24,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:24,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea34a2c5046eb87ff8e HTTP/1.1" 200 None 2026-03-24 08:43:24,806 - lava_callback - DEBUG - Processing node 69c24ea34a2c5046eb87ff8e: baseline-arm 2026-03-24 08:43:24,812 - lava_callback - WARNING - Failed to upload log 2026-03-24 08:43:24,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:43:25,534 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:43:25,536 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea34a2c5046eb87ff8e/lava-logs.yaml 2026-03-24 08:43:25,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:43:26,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:43:26,268 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea34a2c5046eb87ff8e/lava_callback.json.gz 2026-03-24 08:43:26,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:26,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea34a2c5046eb87ff8e HTTP/1.1" 200 None 2026-03-24 08:43:26,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:26,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:43:26,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:26,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ea34a2c5046eb87ff8e HTTP/1.1" 200 None 2026-03-24 08:43:26,373 - lava_callback - INFO - Completed processing callback for node 69c24ea34a2c5046eb87ff8e INFO: 172.20.0.1:39918 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:43:35,196 - lava_callback - INFO - Received callback for node 69bfaba17544ab5cd4a7e0d3 INFO: 172.20.0.1:40028 - "POST /node/69bfaba17544ab5cd4a7e0d3 HTTP/1.0" 202 Accepted 2026-03-24 08:43:35,207 - lava_callback - INFO - Starting async job submit for node 69bfaba17544ab5cd4a7e0d3 2026-03-24 08:43:35,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:35,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfaba17544ab5cd4a7e0d3 HTTP/1.1" 200 None 2026-03-24 08:43:35,233 - lava_callback - DEBUG - Processing node 69bfaba17544ab5cd4a7e0d3: baseline-arm-clabbe 2026-03-24 08:43:35,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfaba17544ab5cd4a7e0d3 2026-03-24 08:43:35,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:35,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfaba17544ab5cd4a7e0d3 HTTP/1.1" 200 None 2026-03-24 08:43:35,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:35,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 08:43:35,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:35,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfaba17544ab5cd4a7e0d3 HTTP/1.1" 200 None 2026-03-24 08:43:35,309 - lava_callback - ERROR - Error processing callback for node 69bfaba17544ab5cd4a7e0d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53448 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:43:54,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:54,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 08:43:55,184 - lava_callback - INFO - Received callback for node 69bfabc67544ab5cd4a7e0dc INFO: 172.20.0.1:56112 - "POST /node/69bfabc67544ab5cd4a7e0dc HTTP/1.0" 202 Accepted 2026-03-24 08:43:55,189 - lava_callback - INFO - Starting async job submit for node 69bfabc67544ab5cd4a7e0dc 2026-03-24 08:43:55,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:55,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfabc67544ab5cd4a7e0dc HTTP/1.1" 200 None 2026-03-24 08:43:55,213 - lava_callback - DEBUG - Processing node 69bfabc67544ab5cd4a7e0dc: baseline-arm-clabbe 2026-03-24 08:43:55,213 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfabc67544ab5cd4a7e0dc 2026-03-24 08:43:55,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:55,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfabc67544ab5cd4a7e0dc HTTP/1.1" 200 None 2026-03-24 08:43:55,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:55,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 08:43:55,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:55,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfabc67544ab5cd4a7e0dc HTTP/1.1" 200 None 2026-03-24 08:43:55,269 - lava_callback - ERROR - Error processing callback for node 69bfabc67544ab5cd4a7e0dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56122 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:43:57,907 - lava_callback - INFO - Received callback for node 69c24ea44a2c5046eb87ff91 INFO: 172.20.0.1:56130 - "POST /node/69c24ea44a2c5046eb87ff91 HTTP/1.0" 202 Accepted 2026-03-24 08:43:57,991 - lava_callback - INFO - Starting async job submit for node 69c24ea44a2c5046eb87ff91 2026-03-24 08:43:58,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:58,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea44a2c5046eb87ff91 HTTP/1.1" 200 None 2026-03-24 08:43:58,094 - lava_callback - DEBUG - Processing node 69c24ea44a2c5046eb87ff91: baseline-arm 2026-03-24 08:43:58,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:43:58,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:43:58,469 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea44a2c5046eb87ff91/log.txt.gz 2026-03-24 08:43:58,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:43:59,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:43:59,049 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea44a2c5046eb87ff91/lava-logs.yaml 2026-03-24 08:43:59,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:43:59,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:43:59,183 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea44a2c5046eb87ff91/lava_callback.json.gz 2026-03-24 08:43:59,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:59,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea44a2c5046eb87ff91 HTTP/1.1" 200 None 2026-03-24 08:43:59,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:59,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:43:59,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:43:59,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ea44a2c5046eb87ff91 HTTP/1.1" 200 None 2026-03-24 08:43:59,262 - lava_callback - INFO - Completed processing callback for node 69c24ea44a2c5046eb87ff91 2026-03-24 08:44:06,090 - lava_callback - INFO - Received callback for node 69c24ea84a2c5046eb87ffa2 INFO: 172.20.0.1:40192 - "POST /node/69c24ea84a2c5046eb87ffa2 HTTP/1.0" 202 Accepted 2026-03-24 08:44:06,144 - lava_callback - INFO - Starting async job submit for node 69c24ea84a2c5046eb87ffa2 2026-03-24 08:44:06,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:06,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea84a2c5046eb87ffa2 HTTP/1.1" 200 None 2026-03-24 08:44:06,193 - lava_callback - DEBUG - Processing node 69c24ea84a2c5046eb87ffa2: baseline-arm 2026-03-24 08:44:06,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:40206 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:44:06,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:06,747 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea84a2c5046eb87ffa2/log.txt.gz 2026-03-24 08:44:06,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:06,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:06,993 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea84a2c5046eb87ffa2/lava-logs.yaml 2026-03-24 08:44:06,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:07,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:07,413 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea84a2c5046eb87ffa2/lava_callback.json.gz 2026-03-24 08:44:07,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:07,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea84a2c5046eb87ffa2 HTTP/1.1" 200 None 2026-03-24 08:44:07,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:07,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:44:07,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:07,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ea84a2c5046eb87ffa2 HTTP/1.1" 200 None 2026-03-24 08:44:07,492 - lava_callback - INFO - Completed processing callback for node 69c24ea84a2c5046eb87ffa2 INFO: 172.20.0.1:51246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51266 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:44:24,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:24,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55498 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:44:35,241 - lava_callback - INFO - Received callback for node 69bfabeb7544ab5cd4a7e0ea INFO: 172.20.0.1:39738 - "POST /node/69bfabeb7544ab5cd4a7e0ea HTTP/1.0" 202 Accepted 2026-03-24 08:44:35,250 - lava_callback - INFO - Starting async job submit for node 69bfabeb7544ab5cd4a7e0ea 2026-03-24 08:44:35,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:35,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfabeb7544ab5cd4a7e0ea HTTP/1.1" 200 None 2026-03-24 08:44:35,276 - lava_callback - DEBUG - Processing node 69bfabeb7544ab5cd4a7e0ea: baseline-arm-clabbe 2026-03-24 08:44:35,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfabeb7544ab5cd4a7e0ea 2026-03-24 08:44:35,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:35,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfabeb7544ab5cd4a7e0ea HTTP/1.1" 200 None 2026-03-24 08:44:35,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:35,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 08:44:35,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:35,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfabeb7544ab5cd4a7e0ea HTTP/1.1" 200 None 2026-03-24 08:44:35,342 - lava_callback - ERROR - Error processing callback for node 69bfabeb7544ab5cd4a7e0ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43496 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:44:47,259 - lava_callback - INFO - Received callback for node 69c24eba4a2c5046eb87ffcb INFO: 172.20.0.1:43502 - "POST /node/69c24eba4a2c5046eb87ffcb HTTP/1.0" 202 Accepted 2026-03-24 08:44:47,267 - lava_callback - INFO - Starting async job submit for node 69c24eba4a2c5046eb87ffcb 2026-03-24 08:44:47,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:47,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eba4a2c5046eb87ffcb HTTP/1.1" 200 None 2026-03-24 08:44:47,314 - lava_callback - DEBUG - Processing node 69c24eba4a2c5046eb87ffcb: baseline-arm-broonie 2026-03-24 08:44:47,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:47,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:47,765 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24eba4a2c5046eb87ffcb/log.txt.gz 2026-03-24 08:44:47,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:48,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:48,094 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24eba4a2c5046eb87ffcb/lava-logs.yaml 2026-03-24 08:44:48,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:48,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:48,234 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24eba4a2c5046eb87ffcb/lava_callback.json.gz 2026-03-24 08:44:48,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:48,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eba4a2c5046eb87ffcb HTTP/1.1" 200 None 2026-03-24 08:44:48,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:48,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:44:48,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:48,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eba4a2c5046eb87ffcb HTTP/1.1" 200 None 2026-03-24 08:44:48,325 - lava_callback - INFO - Completed processing callback for node 69c24eba4a2c5046eb87ffcb 2026-03-24 08:44:49,058 - lava_callback - INFO - Received callback for node 69c24ea24a2c5046eb87ff8b INFO: 172.20.0.1:43504 - "POST /node/69c24ea24a2c5046eb87ff8b HTTP/1.0" 202 Accepted 2026-03-24 08:44:49,064 - lava_callback - INFO - Starting async job submit for node 69c24ea24a2c5046eb87ff8b 2026-03-24 08:44:49,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:49,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea24a2c5046eb87ff8b HTTP/1.1" 200 None 2026-03-24 08:44:49,108 - lava_callback - DEBUG - Processing node 69c24ea24a2c5046eb87ff8b: baseline-arm 2026-03-24 08:44:49,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:49,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:49,424 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea24a2c5046eb87ff8b/log.txt.gz 2026-03-24 08:44:49,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:49,756 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:49,757 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea24a2c5046eb87ff8b/lava-logs.yaml 2026-03-24 08:44:49,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:49,928 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:49,930 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea24a2c5046eb87ff8b/lava_callback.json.gz 2026-03-24 08:44:49,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:49,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea24a2c5046eb87ff8b HTTP/1.1" 200 None 2026-03-24 08:44:49,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:49,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:44:49,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:50,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ea24a2c5046eb87ff8b HTTP/1.1" 200 None 2026-03-24 08:44:50,016 - lava_callback - INFO - Completed processing callback for node 69c24ea24a2c5046eb87ff8b 2026-03-24 08:44:54,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:54,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39562 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:44:56,838 - lava_callback - INFO - Received callback for node 69c24ed34a2c5046eb880030 INFO: 172.20.0.1:39576 - "POST /node/69c24ed34a2c5046eb880030 HTTP/1.0" 202 Accepted 2026-03-24 08:44:56,889 - lava_callback - INFO - Starting async job submit for node 69c24ed34a2c5046eb880030 2026-03-24 08:44:56,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:56,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ed34a2c5046eb880030 HTTP/1.1" 200 None 2026-03-24 08:44:56,955 - lava_callback - DEBUG - Processing node 69c24ed34a2c5046eb880030: baseline-arm-pengutronix 2026-03-24 08:44:57,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:57,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:57,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ed34a2c5046eb880030/log.txt.gz 2026-03-24 08:44:57,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:58,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:58,075 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ed34a2c5046eb880030/lava-logs.yaml 2026-03-24 08:44:58,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:44:58,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:44:58,388 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ed34a2c5046eb880030/lava_callback.json.gz 2026-03-24 08:44:58,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:58,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ed34a2c5046eb880030 HTTP/1.1" 200 None 2026-03-24 08:44:58,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:58,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:44:58,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:44:58,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ed34a2c5046eb880030 HTTP/1.1" 200 None 2026-03-24 08:44:58,538 - lava_callback - INFO - Completed processing callback for node 69c24ed34a2c5046eb880030 2026-03-24 08:45:01,155 - lava_callback - INFO - Received callback for node 69c24ece4a2c5046eb880023 INFO: 172.20.0.1:48344 - "POST /node/69c24ece4a2c5046eb880023 HTTP/1.0" 202 Accepted 2026-03-24 08:45:01,164 - lava_callback - INFO - Starting async job submit for node 69c24ece4a2c5046eb880023 2026-03-24 08:45:01,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:01,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ece4a2c5046eb880023 HTTP/1.1" 200 None 2026-03-24 08:45:01,249 - lava_callback - DEBUG - Processing node 69c24ece4a2c5046eb880023: baseline-arm-pengutronix 2026-03-24 08:45:01,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:01,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:01,881 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ece4a2c5046eb880023/log.txt.gz 2026-03-24 08:45:02,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:02,506 - lava_callback - INFO - Received callback for node 69c24ed04a2c5046eb88002a 2026-03-24 08:45:02,517 - lava_callback - INFO - Starting async job submit for node 69c24ed04a2c5046eb88002a INFO: 172.20.0.1:48352 - "POST /node/69c24ed04a2c5046eb88002a HTTP/1.0" 202 Accepted 2026-03-24 08:45:02,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:02,534 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ece4a2c5046eb880023/lava-logs.yaml 2026-03-24 08:45:02,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:02,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:02,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ed04a2c5046eb88002a HTTP/1.1" 200 None 2026-03-24 08:45:02,601 - lava_callback - DEBUG - Processing node 69c24ed04a2c5046eb88002a: baseline-arm-pengutronix 2026-03-24 08:45:02,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:03,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:03,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ece4a2c5046eb880023/lava_callback.json.gz 2026-03-24 08:45:03,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:03,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ece4a2c5046eb880023 HTTP/1.1" 200 None 2026-03-24 08:45:03,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:03,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:45:03,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:03,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ece4a2c5046eb880023 HTTP/1.1" 200 None 2026-03-24 08:45:03,157 - lava_callback - INFO - Completed processing callback for node 69c24ece4a2c5046eb880023 2026-03-24 08:45:03,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:03,370 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ed04a2c5046eb88002a/log.txt.gz 2026-03-24 08:45:03,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:04,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:04,144 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ed04a2c5046eb88002a/lava-logs.yaml 2026-03-24 08:45:04,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:04,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:04,452 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ed04a2c5046eb88002a/lava_callback.json.gz 2026-03-24 08:45:04,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:04,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ed04a2c5046eb88002a HTTP/1.1" 200 None 2026-03-24 08:45:04,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:04,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:45:04,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:04,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ed04a2c5046eb88002a HTTP/1.1" 200 None 2026-03-24 08:45:04,584 - lava_callback - INFO - Completed processing callback for node 69c24ed04a2c5046eb88002a INFO: 172.20.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38244 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 08:45:15,339 - lava_callback - INFO - Received callback for node 69bfac0f7544ab5cd4a7e0fa INFO: 172.20.0.1:38252 - "POST /node/69bfac0f7544ab5cd4a7e0fa HTTP/1.0" 202 Accepted 2026-03-24 08:45:15,347 - lava_callback - INFO - Starting async job submit for node 69bfac0f7544ab5cd4a7e0fa 2026-03-24 08:45:15,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:15,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfac0f7544ab5cd4a7e0fa HTTP/1.1" 200 None 2026-03-24 08:45:15,380 - lava_callback - DEBUG - Processing node 69bfac0f7544ab5cd4a7e0fa: baseline-arm-clabbe 2026-03-24 08:45:15,380 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfac0f7544ab5cd4a7e0fa 2026-03-24 08:45:15,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:15,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfac0f7544ab5cd4a7e0fa HTTP/1.1" 200 None 2026-03-24 08:45:15,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:15,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 08:45:15,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:15,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfac0f7544ab5cd4a7e0fa HTTP/1.1" 200 None 2026-03-24 08:45:15,478 - lava_callback - ERROR - Error processing callback for node 69bfac0f7544ab5cd4a7e0fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38270 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:45:23,497 - lava_callback - INFO - Received callback for node 69c24ec14a2c5046eb87ffe0 INFO: 172.20.0.1:58634 - "POST /node/69c24ec14a2c5046eb87ffe0 HTTP/1.0" 202 Accepted 2026-03-24 08:45:23,507 - lava_callback - INFO - Starting async job submit for node 69c24ec14a2c5046eb87ffe0 2026-03-24 08:45:23,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:23,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec14a2c5046eb87ffe0 HTTP/1.1" 200 None 2026-03-24 08:45:23,584 - lava_callback - DEBUG - Processing node 69c24ec14a2c5046eb87ffe0: baseline-arm-clabbe 2026-03-24 08:45:23,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:24,305 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:24,307 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec14a2c5046eb87ffe0/log.txt.gz 2026-03-24 08:45:24,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:24,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 08:45:24,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:25,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:25,064 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec14a2c5046eb87ffe0/lava-logs.yaml 2026-03-24 08:45:25,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:25,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:25,515 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec14a2c5046eb87ffe0/lava_callback.json.gz 2026-03-24 08:45:25,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:25,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec14a2c5046eb87ffe0 HTTP/1.1" 200 None 2026-03-24 08:45:25,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:25,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:45:25,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:25,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ec14a2c5046eb87ffe0 HTTP/1.1" 200 None 2026-03-24 08:45:25,690 - lava_callback - INFO - Completed processing callback for node 69c24ec14a2c5046eb87ffe0 INFO: 172.20.0.1:58646 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:45:27,258 - lava_callback - INFO - Received callback for node 69c24ebb4a2c5046eb87ffce INFO: 172.20.0.1:58660 - "POST /node/69c24ebb4a2c5046eb87ffce HTTP/1.0" 202 Accepted 2026-03-24 08:45:27,269 - lava_callback - INFO - Starting async job submit for node 69c24ebb4a2c5046eb87ffce 2026-03-24 08:45:27,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:27,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ebb4a2c5046eb87ffce HTTP/1.1" 200 None 2026-03-24 08:45:27,362 - lava_callback - DEBUG - Processing node 69c24ebb4a2c5046eb87ffce: baseline-arm-broonie 2026-03-24 08:45:27,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:28,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:28,063 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24ebb4a2c5046eb87ffce/log.txt.gz 2026-03-24 08:45:28,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:28,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:28,788 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24ebb4a2c5046eb87ffce/lava-logs.yaml 2026-03-24 08:45:28,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:29,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:29,206 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24ebb4a2c5046eb87ffce/lava_callback.json.gz 2026-03-24 08:45:29,234 - lava_callback - INFO - Received callback for node 69c24ea74a2c5046eb87ff97 INFO: 172.20.0.1:58676 - "POST /node/69c24ea74a2c5046eb87ff97 HTTP/1.0" 202 Accepted 2026-03-24 08:45:29,248 - lava_callback - INFO - Starting async job submit for node 69c24ea74a2c5046eb87ff97 2026-03-24 08:45:29,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:29,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:29,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ebb4a2c5046eb87ffce HTTP/1.1" 200 None 2026-03-24 08:45:29,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:29,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea74a2c5046eb87ff97 HTTP/1.1" 200 None 2026-03-24 08:45:29,327 - lava_callback - DEBUG - Processing node 69c24ea74a2c5046eb87ff97: baseline-arm 2026-03-24 08:45:29,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:45:29,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:29,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ebb4a2c5046eb87ffce HTTP/1.1" 200 None 2026-03-24 08:45:29,505 - lava_callback - INFO - Completed processing callback for node 69c24ebb4a2c5046eb87ffce 2026-03-24 08:45:29,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:30,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:30,308 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea74a2c5046eb87ff97/log.txt.gz 2026-03-24 08:45:30,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:31,157 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:31,159 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea74a2c5046eb87ff97/lava-logs.yaml 2026-03-24 08:45:31,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:31,638 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:31,640 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea74a2c5046eb87ff97/lava_callback.json.gz 2026-03-24 08:45:31,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:31,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea74a2c5046eb87ff97 HTTP/1.1" 200 None 2026-03-24 08:45:31,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:31,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:45:31,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:31,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ea74a2c5046eb87ff97 HTTP/1.1" 200 None 2026-03-24 08:45:31,830 - lava_callback - INFO - Completed processing callback for node 69c24ea74a2c5046eb87ff97 INFO: 172.20.0.1:43266 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:45:39,913 - lava_callback - INFO - Received callback for node 69c24ebd4a2c5046eb87ffd1 INFO: 172.20.0.1:36324 - "POST /node/69c24ebd4a2c5046eb87ffd1 HTTP/1.0" 202 Accepted 2026-03-24 08:45:39,923 - lava_callback - INFO - Starting async job submit for node 69c24ebd4a2c5046eb87ffd1 2026-03-24 08:45:39,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:40,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ebd4a2c5046eb87ffd1 HTTP/1.1" 200 None 2026-03-24 08:45:40,003 - lava_callback - DEBUG - Processing node 69c24ebd4a2c5046eb87ffd1: baseline-arm-broonie 2026-03-24 08:45:40,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:40,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:40,792 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24ebd4a2c5046eb87ffd1/log.txt.gz 2026-03-24 08:45:41,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:41,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:41,605 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24ebd4a2c5046eb87ffd1/lava-logs.yaml 2026-03-24 08:45:41,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:45:42,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:45:42,067 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24ebd4a2c5046eb87ffd1/lava_callback.json.gz 2026-03-24 08:45:42,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:42,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ebd4a2c5046eb87ffd1 HTTP/1.1" 200 None 2026-03-24 08:45:42,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:42,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:45:42,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:42,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ebd4a2c5046eb87ffd1 HTTP/1.1" 200 None 2026-03-24 08:45:42,229 - lava_callback - INFO - Completed processing callback for node 69c24ebd4a2c5046eb87ffd1 INFO: 172.20.0.1:36332 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:45:54,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:54,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-24 08:45:55,152 - lava_callback - INFO - Received callback for node 69bfac367544ab5cd4a7e10f INFO: 172.20.0.1:45816 - "POST /node/69bfac367544ab5cd4a7e10f HTTP/1.0" 202 Accepted 2026-03-24 08:45:55,157 - lava_callback - INFO - Starting async job submit for node 69bfac367544ab5cd4a7e10f 2026-03-24 08:45:55,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:55,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfac367544ab5cd4a7e10f HTTP/1.1" 200 None 2026-03-24 08:45:55,183 - lava_callback - DEBUG - Processing node 69bfac367544ab5cd4a7e10f: baseline-arm-clabbe 2026-03-24 08:45:55,183 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfac367544ab5cd4a7e10f 2026-03-24 08:45:55,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:55,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfac367544ab5cd4a7e10f HTTP/1.1" 200 None 2026-03-24 08:45:55,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:55,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 08:45:55,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:45:55,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfac367544ab5cd4a7e10f HTTP/1.1" 200 None 2026-03-24 08:45:55,236 - lava_callback - ERROR - Error processing callback for node 69bfac367544ab5cd4a7e10f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46212 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:46:32,115 - lava_callback - INFO - Received callback for node 69c24f4d4a2c5046eb8800fe INFO: 172.20.0.1:51426 - "POST /node/69c24f4d4a2c5046eb8800fe HTTP/1.0" 202 Accepted 2026-03-24 08:46:32,121 - lava_callback - INFO - Starting async job submit for node 69c24f4d4a2c5046eb8800fe 2026-03-24 08:46:32,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:32,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24f4d4a2c5046eb8800fe HTTP/1.1" 200 None 2026-03-24 08:46:32,140 - lava_callback - DEBUG - Processing node 69c24f4d4a2c5046eb8800fe: baseline-arm 2026-03-24 08:46:32,145 - lava_callback - WARNING - Failed to upload log 2026-03-24 08:46:32,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:46:32,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:46:32,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c24f4d4a2c5046eb8800fe/lava-logs.yaml 2026-03-24 08:46:32,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:46:32,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:46:32,479 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c24f4d4a2c5046eb8800fe/lava_callback.json.gz 2026-03-24 08:46:32,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:32,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24f4d4a2c5046eb8800fe HTTP/1.1" 200 None 2026-03-24 08:46:32,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:32,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:46:32,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:32,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24f4d4a2c5046eb8800fe HTTP/1.1" 200 None 2026-03-24 08:46:32,541 - lava_callback - INFO - Completed processing callback for node 69c24f4d4a2c5046eb8800fe 2026-03-24 08:46:35,229 - lava_callback - INFO - Received callback for node 69bfac5c7544ab5cd4a7e122 INFO: 172.20.0.1:51432 - "POST /node/69bfac5c7544ab5cd4a7e122 HTTP/1.0" 202 Accepted 2026-03-24 08:46:35,234 - lava_callback - INFO - Starting async job submit for node 69bfac5c7544ab5cd4a7e122 2026-03-24 08:46:35,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:35,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfac5c7544ab5cd4a7e122 HTTP/1.1" 200 None 2026-03-24 08:46:35,250 - lava_callback - DEBUG - Processing node 69bfac5c7544ab5cd4a7e122: baseline-arm-clabbe 2026-03-24 08:46:35,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfac5c7544ab5cd4a7e122 2026-03-24 08:46:35,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:35,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfac5c7544ab5cd4a7e122 HTTP/1.1" 200 None 2026-03-24 08:46:35,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:35,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 08:46:35,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:35,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfac5c7544ab5cd4a7e122 HTTP/1.1" 200 None 2026-03-24 08:46:35,306 - lava_callback - ERROR - Error processing callback for node 69bfac5c7544ab5cd4a7e122: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51442 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:46:38,164 - lava_callback - INFO - Received callback for node 69c24eeb4a2c5046eb880065 INFO: 172.20.0.1:51456 - "POST /node/69c24eeb4a2c5046eb880065 HTTP/1.0" 202 Accepted 2026-03-24 08:46:38,171 - lava_callback - INFO - Starting async job submit for node 69c24eeb4a2c5046eb880065 2026-03-24 08:46:38,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:38,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eeb4a2c5046eb880065 HTTP/1.1" 200 None 2026-03-24 08:46:38,209 - lava_callback - DEBUG - Processing node 69c24eeb4a2c5046eb880065: ltp-dio 2026-03-24 08:46:38,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:46:38,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:46:38,811 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c24eeb4a2c5046eb880065/log.txt.gz 2026-03-24 08:46:39,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:46:39,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:46:39,228 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c24eeb4a2c5046eb880065/lava-logs.yaml 2026-03-24 08:46:39,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:46:39,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:46:39,412 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c24eeb4a2c5046eb880065/lava_callback.json.gz 2026-03-24 08:46:39,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:39,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eeb4a2c5046eb880065 HTTP/1.1" 200 None 2026-03-24 08:46:39,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:39,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:46:39,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:39,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eeb4a2c5046eb880065 HTTP/1.1" 200 None 2026-03-24 08:46:39,495 - lava_callback - INFO - Completed processing callback for node 69c24eeb4a2c5046eb880065 INFO: 172.20.0.1:34754 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:46:54,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:46:54,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51824 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:47:14,692 - lava_callback - INFO - Received callback for node 69c24ec34a2c5046eb87ffe4 INFO: 172.20.0.1:47614 - "POST /node/69c24ec34a2c5046eb87ffe4 HTTP/1.0" 202 Accepted 2026-03-24 08:47:14,698 - lava_callback - INFO - Starting async job submit for node 69c24ec34a2c5046eb87ffe4 2026-03-24 08:47:14,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:14,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec34a2c5046eb87ffe4 HTTP/1.1" 200 None 2026-03-24 08:47:14,738 - lava_callback - DEBUG - Processing node 69c24ec34a2c5046eb87ffe4: baseline-arm-clabbe 2026-03-24 08:47:14,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:47:15,103 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:47:15,105 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec34a2c5046eb87ffe4/log.txt.gz INFO: 172.20.0.1:47628 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 08:47:15,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:47:15,301 - lava_callback - INFO - Received callback for node 69bfac837544ab5cd4a7e136 INFO: 172.20.0.1:47640 - "POST /node/69bfac837544ab5cd4a7e136 HTTP/1.0" 202 Accepted 2026-03-24 08:47:15,307 - lava_callback - INFO - Starting async job submit for node 69bfac837544ab5cd4a7e136 2026-03-24 08:47:15,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:15,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfac837544ab5cd4a7e136 HTTP/1.1" 200 None 2026-03-24 08:47:15,344 - lava_callback - DEBUG - Processing node 69bfac837544ab5cd4a7e136: baseline-arm-clabbe 2026-03-24 08:47:15,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfac837544ab5cd4a7e136 2026-03-24 08:47:15,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:15,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfac837544ab5cd4a7e136 HTTP/1.1" 200 None 2026-03-24 08:47:15,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:15,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 08:47:15,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:15,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:47:15,417 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec34a2c5046eb87ffe4/lava-logs.yaml 2026-03-24 08:47:15,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:47:15,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfac837544ab5cd4a7e136 HTTP/1.1" 200 None 2026-03-24 08:47:15,430 - lava_callback - ERROR - Error processing callback for node 69bfac837544ab5cd4a7e136: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:47:15,688 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:47:15,690 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec34a2c5046eb87ffe4/lava_callback.json.gz 2026-03-24 08:47:15,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:15,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec34a2c5046eb87ffe4 HTTP/1.1" 200 None 2026-03-24 08:47:15,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:15,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:47:15,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:15,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ec34a2c5046eb87ffe4 HTTP/1.1" 200 None 2026-03-24 08:47:15,780 - lava_callback - INFO - Completed processing callback for node 69c24ec34a2c5046eb87ffe4 INFO: 172.20.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47668 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:47:24,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:24,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46390 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:47:55,315 - lava_callback - INFO - Received callback for node 69bfacab7544ab5cd4a7e146 INFO: 172.20.0.1:48130 - "POST /node/69bfacab7544ab5cd4a7e146 HTTP/1.0" 202 Accepted 2026-03-24 08:47:55,320 - lava_callback - INFO - Starting async job submit for node 69bfacab7544ab5cd4a7e146 2026-03-24 08:47:55,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:55,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacab7544ab5cd4a7e146 HTTP/1.1" 200 None 2026-03-24 08:47:55,336 - lava_callback - DEBUG - Processing node 69bfacab7544ab5cd4a7e146: baseline-arm-clabbe 2026-03-24 08:47:55,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfacab7544ab5cd4a7e146 2026-03-24 08:47:55,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:55,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacab7544ab5cd4a7e146 HTTP/1.1" 200 None 2026-03-24 08:47:55,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:55,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 08:47:55,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:47:55,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfacab7544ab5cd4a7e146 HTTP/1.1" 200 None 2026-03-24 08:47:55,399 - lava_callback - ERROR - Error processing callback for node 69bfacab7544ab5cd4a7e146: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36580 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:48:35,380 - lava_callback - INFO - Received callback for node 69bfacd27544ab5cd4a7e18f INFO: 172.20.0.1:46022 - "POST /node/69bfacd27544ab5cd4a7e18f HTTP/1.0" 202 Accepted 2026-03-24 08:48:35,386 - lava_callback - INFO - Starting async job submit for node 69bfacd27544ab5cd4a7e18f 2026-03-24 08:48:35,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:35,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacd27544ab5cd4a7e18f HTTP/1.1" 200 None 2026-03-24 08:48:35,405 - lava_callback - DEBUG - Processing node 69bfacd27544ab5cd4a7e18f: baseline-arm-clabbe 2026-03-24 08:48:35,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfacd27544ab5cd4a7e18f 2026-03-24 08:48:35,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:35,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacd27544ab5cd4a7e18f HTTP/1.1" 200 None 2026-03-24 08:48:35,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:35,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 08:48:35,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:35,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfacd27544ab5cd4a7e18f HTTP/1.1" 200 None 2026-03-24 08:48:35,470 - lava_callback - ERROR - Error processing callback for node 69bfacd27544ab5cd4a7e18f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46026 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:48:39,362 - lava_callback - INFO - Received callback for node 69c24ee44a2c5046eb880053 INFO: 172.20.0.1:46028 - "POST /node/69c24ee44a2c5046eb880053 HTTP/1.0" 202 Accepted 2026-03-24 08:48:39,368 - lava_callback - INFO - Starting async job submit for node 69c24ee44a2c5046eb880053 2026-03-24 08:48:39,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:39,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ee44a2c5046eb880053 HTTP/1.1" 200 None 2026-03-24 08:48:39,412 - lava_callback - DEBUG - Processing node 69c24ee44a2c5046eb880053: ltp-cap_bounds 2026-03-24 08:48:39,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:48:39,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:48:39,927 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c24ee44a2c5046eb880053/log.txt.gz 2026-03-24 08:48:40,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:48:40,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:48:40,370 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c24ee44a2c5046eb880053/lava-logs.yaml 2026-03-24 08:48:40,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:48:40,555 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:48:40,556 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c24ee44a2c5046eb880053/lava_callback.json.gz 2026-03-24 08:48:40,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:40,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ee44a2c5046eb880053 HTTP/1.1" 200 None 2026-03-24 08:48:40,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:40,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:48:40,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:40,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ee44a2c5046eb880053 HTTP/1.1" 200 None 2026-03-24 08:48:40,659 - lava_callback - INFO - Completed processing callback for node 69c24ee44a2c5046eb880053 INFO: 172.20.0.1:43118 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:48:47,448 - lava_callback - INFO - Received callback for node 69c24eb84a2c5046eb87ffc8 INFO: 172.20.0.1:43126 - "POST /node/69c24eb84a2c5046eb87ffc8 HTTP/1.0" 202 Accepted 2026-03-24 08:48:47,454 - lava_callback - INFO - Starting async job submit for node 69c24eb84a2c5046eb87ffc8 2026-03-24 08:48:47,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:47,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb84a2c5046eb87ffc8 HTTP/1.1" 200 None 2026-03-24 08:48:47,502 - lava_callback - DEBUG - Processing node 69c24eb84a2c5046eb87ffc8: baseline-arm-broonie 2026-03-24 08:48:47,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:48:47,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:48:47,916 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24eb84a2c5046eb87ffc8/log.txt.gz 2026-03-24 08:48:48,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:48:48,270 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:48:48,272 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24eb84a2c5046eb87ffc8/lava-logs.yaml 2026-03-24 08:48:48,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:48:48,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:48:48,419 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c24eb84a2c5046eb87ffc8/lava_callback.json.gz 2026-03-24 08:48:48,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:48,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eb84a2c5046eb87ffc8 HTTP/1.1" 200 None 2026-03-24 08:48:48,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:48,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:48:48,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:48,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eb84a2c5046eb87ffc8 HTTP/1.1" 200 None 2026-03-24 08:48:48,509 - lava_callback - INFO - Completed processing callback for node 69c24eb84a2c5046eb87ffc8 2026-03-24 08:48:54,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:54,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59856 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:48:59,248 - lava_callback - INFO - Received callback for node 69bfacee7544ab5cd4a7e1fd INFO: 172.20.0.1:59862 - "POST /node/69bfacee7544ab5cd4a7e1fd HTTP/1.0" 202 Accepted 2026-03-24 08:48:59,254 - lava_callback - INFO - Starting async job submit for node 69bfacee7544ab5cd4a7e1fd 2026-03-24 08:48:59,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:59,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacee7544ab5cd4a7e1fd HTTP/1.1" 200 None 2026-03-24 08:48:59,270 - lava_callback - DEBUG - Processing node 69bfacee7544ab5cd4a7e1fd: baseline-arm64-baylibre 2026-03-24 08:48:59,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfacee7544ab5cd4a7e1fd 2026-03-24 08:48:59,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:59,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacee7544ab5cd4a7e1fd HTTP/1.1" 200 None 2026-03-24 08:48:59,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:59,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 08:48:59,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:48:59,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfacee7544ab5cd4a7e1fd HTTP/1.1" 200 None 2026-03-24 08:48:59,348 - lava_callback - ERROR - Error processing callback for node 69bfacee7544ab5cd4a7e1fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:49:01,137 - lava_callback - INFO - Received callback for node 69c24ee64a2c5046eb880056 INFO: 172.20.0.1:59542 - "POST /node/69c24ee64a2c5046eb880056 HTTP/1.0" 202 Accepted 2026-03-24 08:49:01,144 - lava_callback - INFO - Starting async job submit for node 69c24ee64a2c5046eb880056 2026-03-24 08:49:01,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:01,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ee64a2c5046eb880056 HTTP/1.1" 200 None 2026-03-24 08:49:01,184 - lava_callback - DEBUG - Processing node 69c24ee64a2c5046eb880056: ltp-containers 2026-03-24 08:49:01,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:49:01,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:49:01,700 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c24ee64a2c5046eb880056/log.txt.gz 2026-03-24 08:49:01,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:49:02,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:49:02,144 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c24ee64a2c5046eb880056/lava-logs.yaml 2026-03-24 08:49:02,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:49:02,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:49:02,338 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c24ee64a2c5046eb880056/lava_callback.json.gz 2026-03-24 08:49:02,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:02,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ee64a2c5046eb880056 HTTP/1.1" 200 None 2026-03-24 08:49:02,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:02,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:49:02,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:02,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ee64a2c5046eb880056 HTTP/1.1" 200 None 2026-03-24 08:49:02,424 - lava_callback - INFO - Completed processing callback for node 69c24ee64a2c5046eb880056 2026-03-24 08:49:02,925 - lava_callback - INFO - Received callback for node 69c24ef74a2c5046eb88007f INFO: 172.20.0.1:59548 - "POST /node/69c24ef74a2c5046eb88007f HTTP/1.0" 202 Accepted 2026-03-24 08:49:02,933 - lava_callback - INFO - Starting async job submit for node 69c24ef74a2c5046eb88007f 2026-03-24 08:49:02,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:02,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef74a2c5046eb88007f HTTP/1.1" 200 None 2026-03-24 08:49:02,986 - lava_callback - DEBUG - Processing node 69c24ef74a2c5046eb88007f: ltp-syscalls 2026-03-24 08:49:03,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:49:03,866 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:49:03,868 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c24ef74a2c5046eb88007f/log.txt.gz 2026-03-24 08:49:04,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:49:04,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:49:04,507 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c24ef74a2c5046eb88007f/lava-logs.yaml 2026-03-24 08:49:04,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:49:04,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:49:04,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c24ef74a2c5046eb88007f/lava_callback.json.gz 2026-03-24 08:49:04,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:04,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef74a2c5046eb88007f HTTP/1.1" 200 None 2026-03-24 08:49:04,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:04,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:49:04,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:04,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ef74a2c5046eb88007f HTTP/1.1" 200 None 2026-03-24 08:49:04,760 - lava_callback - INFO - Completed processing callback for node 69c24ef74a2c5046eb88007f 2026-03-24 08:49:05,678 - lava_callback - INFO - Received callback for node 69c24ec84a2c5046eb88000f INFO: 172.20.0.1:59552 - "POST /node/69c24ec84a2c5046eb88000f HTTP/1.0" 202 Accepted 2026-03-24 08:49:05,684 - lava_callback - INFO - Starting async job submit for node 69c24ec84a2c5046eb88000f 2026-03-24 08:49:05,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:05,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec84a2c5046eb88000f HTTP/1.1" 200 None 2026-03-24 08:49:05,725 - lava_callback - DEBUG - Processing node 69c24ec84a2c5046eb88000f: baseline-arm-clabbe 2026-03-24 08:49:05,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:49:06,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:49:06,073 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec84a2c5046eb88000f/log.txt.gz 2026-03-24 08:49:06,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:49:06,369 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:49:06,371 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec84a2c5046eb88000f/lava-logs.yaml 2026-03-24 08:49:06,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:59560 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:49:06,521 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:49:06,523 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec84a2c5046eb88000f/lava_callback.json.gz 2026-03-24 08:49:06,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:06,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec84a2c5046eb88000f HTTP/1.1" 200 None 2026-03-24 08:49:06,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:06,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:49:06,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:06,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ec84a2c5046eb88000f HTTP/1.1" 200 None 2026-03-24 08:49:06,606 - lava_callback - INFO - Completed processing callback for node 69c24ec84a2c5046eb88000f INFO: 172.20.0.1:48296 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 08:49:15,481 - lava_callback - INFO - Received callback for node 69bfacf77544ab5cd4a7e211 INFO: 172.20.0.1:48310 - "POST /node/69bfacf77544ab5cd4a7e211 HTTP/1.0" 202 Accepted 2026-03-24 08:49:15,487 - lava_callback - INFO - Starting async job submit for node 69bfacf77544ab5cd4a7e211 2026-03-24 08:49:15,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:15,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacf77544ab5cd4a7e211 HTTP/1.1" 200 None 2026-03-24 08:49:15,512 - lava_callback - DEBUG - Processing node 69bfacf77544ab5cd4a7e211: baseline-arm-clabbe 2026-03-24 08:49:15,512 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfacf77544ab5cd4a7e211 2026-03-24 08:49:15,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:15,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacf77544ab5cd4a7e211 HTTP/1.1" 200 None 2026-03-24 08:49:15,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:15,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 08:49:15,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:15,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfacf77544ab5cd4a7e211 HTTP/1.1" 200 None 2026-03-24 08:49:15,571 - lava_callback - ERROR - Error processing callback for node 69bfacf77544ab5cd4a7e211: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48326 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:49:16,179 - lava_callback - INFO - Received callback for node 69bfacfe7544ab5cd4a7e221 INFO: 172.20.0.1:48340 - "POST /node/69bfacfe7544ab5cd4a7e221 HTTP/1.0" 202 Accepted 2026-03-24 08:49:16,184 - lava_callback - INFO - Starting async job submit for node 69bfacfe7544ab5cd4a7e221 2026-03-24 08:49:16,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:16,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacfe7544ab5cd4a7e221 HTTP/1.1" 200 None 2026-03-24 08:49:16,200 - lava_callback - DEBUG - Processing node 69bfacfe7544ab5cd4a7e221: baseline-arm64-clabbe 2026-03-24 08:49:16,200 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfacfe7544ab5cd4a7e221 2026-03-24 08:49:16,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:16,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacfe7544ab5cd4a7e221 HTTP/1.1" 200 None 2026-03-24 08:49:16,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:16,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 08:49:16,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:16,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfacfe7544ab5cd4a7e221 HTTP/1.1" 200 None 2026-03-24 08:49:16,268 - lava_callback - ERROR - Error processing callback for node 69bfacfe7544ab5cd4a7e221: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48346 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:49:16,854 - lava_callback - INFO - Received callback for node 69bfacff7544ab5cd4a7e224 INFO: 172.20.0.1:48356 - "POST /node/69bfacff7544ab5cd4a7e224 HTTP/1.0" 202 Accepted 2026-03-24 08:49:16,859 - lava_callback - INFO - Starting async job submit for node 69bfacff7544ab5cd4a7e224 2026-03-24 08:49:16,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:16,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacff7544ab5cd4a7e224 HTTP/1.1" 200 None 2026-03-24 08:49:16,876 - lava_callback - DEBUG - Processing node 69bfacff7544ab5cd4a7e224: baseline-arm64-clabbe 2026-03-24 08:49:16,876 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfacff7544ab5cd4a7e224 2026-03-24 08:49:16,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:16,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfacff7544ab5cd4a7e224 HTTP/1.1" 200 None 2026-03-24 08:49:16,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:16,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 08:49:16,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:16,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfacff7544ab5cd4a7e224 HTTP/1.1" 200 None 2026-03-24 08:49:16,933 - lava_callback - ERROR - Error processing callback for node 69bfacff7544ab5cd4a7e224: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:49:17,510 - lava_callback - INFO - Received callback for node 69bfad007544ab5cd4a7e227 INFO: 172.20.0.1:48368 - "POST /node/69bfad007544ab5cd4a7e227 HTTP/1.0" 202 Accepted 2026-03-24 08:49:17,515 - lava_callback - INFO - Starting async job submit for node 69bfad007544ab5cd4a7e227 2026-03-24 08:49:17,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:17,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad007544ab5cd4a7e227 HTTP/1.1" 200 None 2026-03-24 08:49:17,533 - lava_callback - DEBUG - Processing node 69bfad007544ab5cd4a7e227: baseline-arm64-clabbe 2026-03-24 08:49:17,533 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfad007544ab5cd4a7e227 2026-03-24 08:49:17,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:17,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad007544ab5cd4a7e227 HTTP/1.1" 200 None 2026-03-24 08:49:17,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:17,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 08:49:17,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:17,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfad007544ab5cd4a7e227 HTTP/1.1" 200 None 2026-03-24 08:49:17,584 - lava_callback - ERROR - Error processing callback for node 69bfad007544ab5cd4a7e227: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:49:18,169 - lava_callback - INFO - Received callback for node 69bfad007544ab5cd4a7e22a INFO: 172.20.0.1:48376 - "POST /node/69bfad007544ab5cd4a7e22a HTTP/1.0" 202 Accepted 2026-03-24 08:49:18,176 - lava_callback - INFO - Starting async job submit for node 69bfad007544ab5cd4a7e22a 2026-03-24 08:49:18,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:18,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad007544ab5cd4a7e22a HTTP/1.1" 200 None 2026-03-24 08:49:18,192 - lava_callback - DEBUG - Processing node 69bfad007544ab5cd4a7e22a: baseline-arm64-clabbe 2026-03-24 08:49:18,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfad007544ab5cd4a7e22a 2026-03-24 08:49:18,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:18,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad007544ab5cd4a7e22a HTTP/1.1" 200 None 2026-03-24 08:49:18,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:18,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 08:49:18,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:18,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfad007544ab5cd4a7e22a HTTP/1.1" 200 None 2026-03-24 08:49:18,247 - lava_callback - ERROR - Error processing callback for node 69bfad007544ab5cd4a7e22a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:49:18,848 - lava_callback - INFO - Received callback for node 69bfad027544ab5cd4a7e232 INFO: 172.20.0.1:48378 - "POST /node/69bfad027544ab5cd4a7e232 HTTP/1.0" 202 Accepted 2026-03-24 08:49:18,853 - lava_callback - INFO - Starting async job submit for node 69bfad027544ab5cd4a7e232 2026-03-24 08:49:18,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:18,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad027544ab5cd4a7e232 HTTP/1.1" 200 None 2026-03-24 08:49:18,870 - lava_callback - DEBUG - Processing node 69bfad027544ab5cd4a7e232: baseline-arm64-clabbe 2026-03-24 08:49:18,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfad027544ab5cd4a7e232 2026-03-24 08:49:18,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:18,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad027544ab5cd4a7e232 HTTP/1.1" 200 None 2026-03-24 08:49:18,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:18,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 08:49:18,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:18,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfad027544ab5cd4a7e232 HTTP/1.1" 200 None 2026-03-24 08:49:18,922 - lava_callback - ERROR - Error processing callback for node 69bfad027544ab5cd4a7e232: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:49:19,518 - lava_callback - INFO - Received callback for node 69bfad037544ab5cd4a7e235 INFO: 172.20.0.1:48392 - "POST /node/69bfad037544ab5cd4a7e235 HTTP/1.0" 202 Accepted 2026-03-24 08:49:19,524 - lava_callback - INFO - Starting async job submit for node 69bfad037544ab5cd4a7e235 2026-03-24 08:49:19,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:19,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad037544ab5cd4a7e235 HTTP/1.1" 200 None 2026-03-24 08:49:19,540 - lava_callback - DEBUG - Processing node 69bfad037544ab5cd4a7e235: baseline-arm64-clabbe 2026-03-24 08:49:19,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfad037544ab5cd4a7e235 2026-03-24 08:49:19,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:19,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad037544ab5cd4a7e235 HTTP/1.1" 200 None 2026-03-24 08:49:19,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:19,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 08:49:19,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:19,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfad037544ab5cd4a7e235 HTTP/1.1" 200 None 2026-03-24 08:49:19,591 - lava_callback - ERROR - Error processing callback for node 69bfad037544ab5cd4a7e235: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:49:24,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:24,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38226 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:49:59,132 - lava_callback - INFO - Received callback for node 69bfad277544ab5cd4a7e2b2 INFO: 172.20.0.1:38230 - "POST /node/69bfad277544ab5cd4a7e2b2 HTTP/1.0" 202 Accepted 2026-03-24 08:49:59,141 - lava_callback - INFO - Starting async job submit for node 69bfad277544ab5cd4a7e2b2 2026-03-24 08:49:59,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:59,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad277544ab5cd4a7e2b2 HTTP/1.1" 200 None 2026-03-24 08:49:59,168 - lava_callback - DEBUG - Processing node 69bfad277544ab5cd4a7e2b2: baseline-arm64-baylibre 2026-03-24 08:49:59,168 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfad277544ab5cd4a7e2b2 2026-03-24 08:49:59,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:59,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad277544ab5cd4a7e2b2 HTTP/1.1" 200 None 2026-03-24 08:49:59,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:59,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 08:49:59,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:49:59,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfad277544ab5cd4a7e2b2 HTTP/1.1" 200 None 2026-03-24 08:49:59,232 - lava_callback - ERROR - Error processing callback for node 69bfad277544ab5cd4a7e2b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52226 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:50:48,844 - lava_callback - INFO - Received callback for node 69c24ec94a2c5046eb880012 INFO: 172.20.0.1:52238 - "POST /node/69c24ec94a2c5046eb880012 HTTP/1.0" 202 Accepted 2026-03-24 08:50:48,850 - lava_callback - INFO - Starting async job submit for node 69c24ec94a2c5046eb880012 2026-03-24 08:50:48,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:50:48,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec94a2c5046eb880012 HTTP/1.1" 200 None 2026-03-24 08:50:48,890 - lava_callback - DEBUG - Processing node 69c24ec94a2c5046eb880012: baseline-arm-clabbe 2026-03-24 08:50:49,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:50:49,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:50:49,421 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec94a2c5046eb880012/log.txt.gz 2026-03-24 08:50:49,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:50:49,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:50:49,809 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec94a2c5046eb880012/lava-logs.yaml 2026-03-24 08:50:49,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:50:50,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:50:50,030 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c24ec94a2c5046eb880012/lava_callback.json.gz 2026-03-24 08:50:50,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:50:50,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ec94a2c5046eb880012 HTTP/1.1" 200 None 2026-03-24 08:50:50,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:50:50,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:50:50,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:50:50,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ec94a2c5046eb880012 HTTP/1.1" 200 None 2026-03-24 08:50:50,139 - lava_callback - INFO - Completed processing callback for node 69c24ec94a2c5046eb880012 2026-03-24 08:50:54,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:50:54,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36230 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 08:51:15,503 - lava_callback - INFO - Received callback for node 69bfad787544ab5cd4a7e347 INFO: 172.20.0.1:36236 - "POST /node/69bfad787544ab5cd4a7e347 HTTP/1.0" 202 Accepted 2026-03-24 08:51:15,512 - lava_callback - INFO - Starting async job submit for node 69bfad787544ab5cd4a7e347 2026-03-24 08:51:15,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:51:15,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad787544ab5cd4a7e347 HTTP/1.1" 200 None 2026-03-24 08:51:15,530 - lava_callback - DEBUG - Processing node 69bfad787544ab5cd4a7e347: baseline-arm64-clabbe 2026-03-24 08:51:15,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfad787544ab5cd4a7e347 2026-03-24 08:51:15,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:51:15,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfad787544ab5cd4a7e347 HTTP/1.1" 200 None 2026-03-24 08:51:15,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:51:15,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 08:51:15,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:51:15,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfad787544ab5cd4a7e347 HTTP/1.1" 200 None 2026-03-24 08:51:15,606 - lava_callback - ERROR - Error processing callback for node 69bfad787544ab5cd4a7e347: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37538 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:51:27,725 - lava_callback - INFO - Received callback for node 69c24bd94a2c5046eb87f90a INFO: 172.20.0.1:37546 - "POST /node/69c24bd94a2c5046eb87f90a HTTP/1.0" 202 Accepted 2026-03-24 08:51:27,736 - lava_callback - INFO - Starting async job submit for node 69c24bd94a2c5046eb87f90a 2026-03-24 08:51:27,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:51:27,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24bd94a2c5046eb87f90a HTTP/1.1" 200 None 2026-03-24 08:51:27,853 - lava_callback - DEBUG - Processing node 69c24bd94a2c5046eb87f90a: ltp-smoke 2026-03-24 08:51:28,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:51:29,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:51:29,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c24bd94a2c5046eb87f90a/log.txt.gz 2026-03-24 08:51:29,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:51:30,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:51:30,284 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c24bd94a2c5046eb87f90a/lava-logs.yaml 2026-03-24 08:51:30,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:51:31,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:51:31,015 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c24bd94a2c5046eb87f90a/lava_callback.json.gz 2026-03-24 08:51:31,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:51:31,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24bd94a2c5046eb87f90a HTTP/1.1" 200 None 2026-03-24 08:51:31,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:51:31,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248de4a2c5046eb87f896 HTTP/1.1" 200 None 2026-03-24 08:51:31,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:51:31,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24bd94a2c5046eb87f90a HTTP/1.1" 200 None 2026-03-24 08:51:31,439 - lava_callback - INFO - Completed processing callback for node 69c24bd94a2c5046eb87f90a INFO: 172.20.0.1:58368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42798 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:51:54,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:51:54,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42678 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:52:00,845 - lava_callback - INFO - Received callback for node 69c24ee74a2c5046eb880059 INFO: 172.20.0.1:48252 - "POST /node/69c24ee74a2c5046eb880059 HTTP/1.0" 202 Accepted 2026-03-24 08:52:00,853 - lava_callback - INFO - Starting async job submit for node 69c24ee74a2c5046eb880059 2026-03-24 08:52:00,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:00,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ee74a2c5046eb880059 HTTP/1.1" 200 None 2026-03-24 08:52:00,951 - lava_callback - DEBUG - Processing node 69c24ee74a2c5046eb880059: ltp-controllers 2026-03-24 08:52:01,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:52:02,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:52:02,140 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c24ee74a2c5046eb880059/log.txt.gz 2026-03-24 08:52:02,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:52:03,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:52:03,132 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c24ee74a2c5046eb880059/lava-logs.yaml 2026-03-24 08:52:03,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:52:03,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:52:03,649 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c24ee74a2c5046eb880059/lava_callback.json.gz 2026-03-24 08:52:03,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:03,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ee74a2c5046eb880059 HTTP/1.1" 200 None 2026-03-24 08:52:03,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:03,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:52:03,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:03,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ee74a2c5046eb880059 HTTP/1.1" 200 None 2026-03-24 08:52:03,803 - lava_callback - INFO - Completed processing callback for node 69c24ee74a2c5046eb880059 INFO: 172.20.0.1:48254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50318 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:52:24,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:24,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50282 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:52:35,558 - lava_callback - INFO - Received callback for node 69bfadc47544ab5cd4a7e376 INFO: 172.20.0.1:35796 - "POST /node/69bfadc47544ab5cd4a7e376 HTTP/1.0" 202 Accepted 2026-03-24 08:52:35,563 - lava_callback - INFO - Starting async job submit for node 69bfadc47544ab5cd4a7e376 2026-03-24 08:52:35,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:35,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfadc47544ab5cd4a7e376 HTTP/1.1" 200 None 2026-03-24 08:52:35,596 - lava_callback - DEBUG - Processing node 69bfadc47544ab5cd4a7e376: baseline-arm64-clabbe 2026-03-24 08:52:35,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfadc47544ab5cd4a7e376 2026-03-24 08:52:35,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:35,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfadc47544ab5cd4a7e376 HTTP/1.1" 200 None 2026-03-24 08:52:35,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:35,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 08:52:35,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:35,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfadc47544ab5cd4a7e376 HTTP/1.1" 200 None 2026-03-24 08:52:35,675 - lava_callback - ERROR - Error processing callback for node 69bfadc47544ab5cd4a7e376: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35804 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:52:39,687 - lava_callback - INFO - Received callback for node 69c24eea4a2c5046eb880062 INFO: 172.20.0.1:35806 - "POST /node/69c24eea4a2c5046eb880062 HTTP/1.0" 202 Accepted 2026-03-24 08:52:39,693 - lava_callback - INFO - Starting async job submit for node 69c24eea4a2c5046eb880062 2026-03-24 08:52:39,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:39,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eea4a2c5046eb880062 HTTP/1.1" 200 None 2026-03-24 08:52:39,737 - lava_callback - DEBUG - Processing node 69c24eea4a2c5046eb880062: ltp-cve 2026-03-24 08:52:40,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:52:40,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:52:40,349 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c24eea4a2c5046eb880062/log.txt.gz 2026-03-24 08:52:40,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:52:40,868 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:52:40,871 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c24eea4a2c5046eb880062/lava-logs.yaml 2026-03-24 08:52:40,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:52:41,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:52:41,043 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c24eea4a2c5046eb880062/lava_callback.json.gz 2026-03-24 08:52:41,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:41,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eea4a2c5046eb880062 HTTP/1.1" 200 None 2026-03-24 08:52:41,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:41,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:52:41,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:41,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eea4a2c5046eb880062 HTTP/1.1" 200 None 2026-03-24 08:52:41,148 - lava_callback - INFO - Completed processing callback for node 69c24eea4a2c5046eb880062 INFO: 172.20.0.1:41970 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:52:54,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:52:54,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46988 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41640 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:53:39,664 - lava_callback - INFO - Received callback for node 69c24eee4a2c5046eb88006b INFO: 172.20.0.1:41650 - "POST /node/69c24eee4a2c5046eb88006b HTTP/1.0" 202 Accepted 2026-03-24 08:53:39,671 - lava_callback - INFO - Starting async job submit for node 69c24eee4a2c5046eb88006b 2026-03-24 08:53:39,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:39,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eee4a2c5046eb88006b HTTP/1.1" 200 None 2026-03-24 08:53:39,718 - lava_callback - DEBUG - Processing node 69c24eee4a2c5046eb88006b: ltp-fsx 2026-03-24 08:53:40,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:53:40,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:53:40,350 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c24eee4a2c5046eb88006b/log.txt.gz 2026-03-24 08:53:40,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:53:40,998 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:53:41,000 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c24eee4a2c5046eb88006b/lava-logs.yaml 2026-03-24 08:53:41,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:53:41,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:53:41,191 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c24eee4a2c5046eb88006b/lava_callback.json.gz 2026-03-24 08:53:41,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:41,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eee4a2c5046eb88006b HTTP/1.1" 200 None 2026-03-24 08:53:41,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:41,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:53:41,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:41,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eee4a2c5046eb88006b HTTP/1.1" 200 None 2026-03-24 08:53:41,325 - lava_callback - INFO - Completed processing callback for node 69c24eee4a2c5046eb88006b INFO: 172.20.0.1:55948 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:53:54,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:54,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 08:53:55,676 - lava_callback - INFO - Received callback for node 69bfae117544ab5cd4a7e398 INFO: 172.20.0.1:39194 - "POST /node/69bfae117544ab5cd4a7e398 HTTP/1.0" 202 Accepted 2026-03-24 08:53:55,683 - lava_callback - INFO - Starting async job submit for node 69bfae117544ab5cd4a7e398 2026-03-24 08:53:55,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:55,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae117544ab5cd4a7e398 HTTP/1.1" 200 None 2026-03-24 08:53:55,704 - lava_callback - DEBUG - Processing node 69bfae117544ab5cd4a7e398: baseline-arm64-clabbe 2026-03-24 08:53:55,704 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfae117544ab5cd4a7e398 2026-03-24 08:53:55,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:55,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae117544ab5cd4a7e398 HTTP/1.1" 200 None 2026-03-24 08:53:55,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:55,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 08:53:55,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:55,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfae117544ab5cd4a7e398 HTTP/1.1" 200 None 2026-03-24 08:53:55,767 - lava_callback - ERROR - Error processing callback for node 69bfae117544ab5cd4a7e398: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39204 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:53:58,380 - lava_callback - INFO - Received callback for node 69c251144a2c5046eb8801f0 INFO: 172.20.0.1:39212 - "POST /node/69c251144a2c5046eb8801f0 HTTP/1.0" 202 Accepted 2026-03-24 08:53:58,386 - lava_callback - INFO - Starting async job submit for node 69c251144a2c5046eb8801f0 2026-03-24 08:53:58,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:58,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251144a2c5046eb8801f0 HTTP/1.1" 200 None 2026-03-24 08:53:58,411 - lava_callback - DEBUG - Processing node 69c251144a2c5046eb8801f0: baseline-arm 2026-03-24 08:53:58,418 - lava_callback - WARNING - Failed to upload log 2026-03-24 08:53:58,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:53:58,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:53:58,585 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c251144a2c5046eb8801f0/lava-logs.yaml 2026-03-24 08:53:58,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:53:58,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:53:58,748 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c251144a2c5046eb8801f0/lava_callback.json.gz 2026-03-24 08:53:58,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:58,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251144a2c5046eb8801f0 HTTP/1.1" 200 None 2026-03-24 08:53:58,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:58,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:53:58,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:53:58,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251144a2c5046eb8801f0 HTTP/1.1" 200 None 2026-03-24 08:53:58,822 - lava_callback - INFO - Completed processing callback for node 69c251144a2c5046eb8801f0 INFO: 172.20.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:54:24,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:54:24,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38594 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:54:59,528 - lava_callback - INFO - Received callback for node 69bfae567544ab5cd4a7e3d1 INFO: 172.20.0.1:38604 - "POST /node/69bfae567544ab5cd4a7e3d1 HTTP/1.0" 202 Accepted 2026-03-24 08:54:59,535 - lava_callback - INFO - Starting async job submit for node 69bfae567544ab5cd4a7e3d1 2026-03-24 08:54:59,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:54:59,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae567544ab5cd4a7e3d1 HTTP/1.1" 200 None 2026-03-24 08:54:59,599 - lava_callback - DEBUG - Processing node 69bfae567544ab5cd4a7e3d1: baseline-arm-baylibre 2026-03-24 08:54:59,600 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfae567544ab5cd4a7e3d1 2026-03-24 08:54:59,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:54:59,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae567544ab5cd4a7e3d1 HTTP/1.1" 200 None 2026-03-24 08:54:59,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:54:59,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 08:54:59,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:54:59,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfae567544ab5cd4a7e3d1 HTTP/1.1" 200 None 2026-03-24 08:54:59,690 - lava_callback - ERROR - Error processing callback for node 69bfae567544ab5cd4a7e3d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:55:00,196 - lava_callback - INFO - Received callback for node 69bfae597544ab5cd4a7e3dc INFO: 172.20.0.1:34604 - "POST /node/69bfae597544ab5cd4a7e3dc HTTP/1.0" 202 Accepted 2026-03-24 08:55:00,206 - lava_callback - INFO - Starting async job submit for node 69bfae597544ab5cd4a7e3dc 2026-03-24 08:55:00,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:00,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae597544ab5cd4a7e3dc HTTP/1.1" 200 None 2026-03-24 08:55:00,232 - lava_callback - DEBUG - Processing node 69bfae597544ab5cd4a7e3dc: baseline-arm-baylibre 2026-03-24 08:55:00,232 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfae597544ab5cd4a7e3dc 2026-03-24 08:55:00,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:00,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae597544ab5cd4a7e3dc HTTP/1.1" 200 None 2026-03-24 08:55:00,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:00,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 08:55:00,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:00,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfae597544ab5cd4a7e3dc HTTP/1.1" 200 None 2026-03-24 08:55:00,369 - lava_callback - ERROR - Error processing callback for node 69bfae597544ab5cd4a7e3dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:55:00,853 - lava_callback - INFO - Received callback for node 69bfae5a7544ab5cd4a7e3df INFO: 172.20.0.1:34620 - "POST /node/69bfae5a7544ab5cd4a7e3df HTTP/1.0" 202 Accepted 2026-03-24 08:55:00,860 - lava_callback - INFO - Starting async job submit for node 69bfae5a7544ab5cd4a7e3df 2026-03-24 08:55:00,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:00,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae5a7544ab5cd4a7e3df HTTP/1.1" 200 None 2026-03-24 08:55:00,877 - lava_callback - DEBUG - Processing node 69bfae5a7544ab5cd4a7e3df: baseline-arm-baylibre 2026-03-24 08:55:00,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfae5a7544ab5cd4a7e3df 2026-03-24 08:55:00,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:00,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae5a7544ab5cd4a7e3df HTTP/1.1" 200 None 2026-03-24 08:55:00,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:00,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 08:55:00,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:00,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfae5a7544ab5cd4a7e3df HTTP/1.1" 200 None 2026-03-24 08:55:00,942 - lava_callback - ERROR - Error processing callback for node 69bfae5a7544ab5cd4a7e3df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34636 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:55:11,365 - lava_callback - INFO - Received callback for node 69c251524a2c5046eb88021c INFO: 172.20.0.1:42822 - "POST /node/69c251524a2c5046eb88021c HTTP/1.0" 202 Accepted 2026-03-24 08:55:11,375 - lava_callback - INFO - Starting async job submit for node 69c251524a2c5046eb88021c 2026-03-24 08:55:11,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:11,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251524a2c5046eb88021c HTTP/1.1" 200 None 2026-03-24 08:55:11,426 - lava_callback - DEBUG - Processing node 69c251524a2c5046eb88021c: baseline-arm64 2026-03-24 08:55:11,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:55:11,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:55:11,882 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251524a2c5046eb88021c/log.txt.gz 2026-03-24 08:55:12,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:55:12,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:55:12,362 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251524a2c5046eb88021c/lava-logs.yaml 2026-03-24 08:55:12,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:55:12,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:55:12,493 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251524a2c5046eb88021c/lava_callback.json.gz 2026-03-24 08:55:12,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:12,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251524a2c5046eb88021c HTTP/1.1" 200 None 2026-03-24 08:55:12,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:12,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:55:12,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:12,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251524a2c5046eb88021c HTTP/1.1" 200 None 2026-03-24 08:55:12,593 - lava_callback - INFO - Completed processing callback for node 69c251524a2c5046eb88021c INFO: 172.20.0.1:42832 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 08:55:15,729 - lava_callback - INFO - Received callback for node 69bfae5e7544ab5cd4a7e3ea INFO: 172.20.0.1:42844 - "POST /node/69bfae5e7544ab5cd4a7e3ea HTTP/1.0" 202 Accepted 2026-03-24 08:55:15,738 - lava_callback - INFO - Starting async job submit for node 69bfae5e7544ab5cd4a7e3ea 2026-03-24 08:55:15,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:15,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae5e7544ab5cd4a7e3ea HTTP/1.1" 200 None 2026-03-24 08:55:15,765 - lava_callback - DEBUG - Processing node 69bfae5e7544ab5cd4a7e3ea: baseline-arm64-clabbe 2026-03-24 08:55:15,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfae5e7544ab5cd4a7e3ea 2026-03-24 08:55:15,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:15,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae5e7544ab5cd4a7e3ea HTTP/1.1" 200 None 2026-03-24 08:55:15,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:15,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 08:55:15,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:15,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfae5e7544ab5cd4a7e3ea HTTP/1.1" 200 None 2026-03-24 08:55:15,842 - lava_callback - ERROR - Error processing callback for node 69bfae5e7544ab5cd4a7e3ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42852 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:55:16,388 - lava_callback - INFO - Received callback for node 69bfae627544ab5cd4a7e3f9 INFO: 172.20.0.1:42856 - "POST /node/69bfae627544ab5cd4a7e3f9 HTTP/1.0" 202 Accepted 2026-03-24 08:55:16,397 - lava_callback - INFO - Starting async job submit for node 69bfae627544ab5cd4a7e3f9 2026-03-24 08:55:16,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:16,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae627544ab5cd4a7e3f9 HTTP/1.1" 200 None 2026-03-24 08:55:16,431 - lava_callback - DEBUG - Processing node 69bfae627544ab5cd4a7e3f9: baseline-arm-clabbe 2026-03-24 08:55:16,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfae627544ab5cd4a7e3f9 2026-03-24 08:55:16,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:16,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae627544ab5cd4a7e3f9 HTTP/1.1" 200 None 2026-03-24 08:55:16,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:16,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 08:55:16,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:16,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfae627544ab5cd4a7e3f9 HTTP/1.1" 200 None 2026-03-24 08:55:16,502 - lava_callback - ERROR - Error processing callback for node 69bfae627544ab5cd4a7e3f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42866 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:55:17,053 - lava_callback - INFO - Received callback for node 69bfae657544ab5cd4a7e408 INFO: 172.20.0.1:42880 - "POST /node/69bfae657544ab5cd4a7e408 HTTP/1.0" 202 Accepted 2026-03-24 08:55:17,065 - lava_callback - INFO - Starting async job submit for node 69bfae657544ab5cd4a7e408 2026-03-24 08:55:17,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:17,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae657544ab5cd4a7e408 HTTP/1.1" 200 None 2026-03-24 08:55:17,091 - lava_callback - DEBUG - Processing node 69bfae657544ab5cd4a7e408: baseline-arm-clabbe 2026-03-24 08:55:17,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfae657544ab5cd4a7e408 2026-03-24 08:55:17,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:17,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae657544ab5cd4a7e408 HTTP/1.1" 200 None 2026-03-24 08:55:17,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:17,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 08:55:17,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:17,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfae657544ab5cd4a7e408 HTTP/1.1" 200 None 2026-03-24 08:55:17,166 - lava_callback - ERROR - Error processing callback for node 69bfae657544ab5cd4a7e408: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:55:17,710 - lava_callback - INFO - Received callback for node 69bfae667544ab5cd4a7e40b INFO: 172.20.0.1:42894 - "POST /node/69bfae667544ab5cd4a7e40b HTTP/1.0" 202 Accepted 2026-03-24 08:55:17,721 - lava_callback - INFO - Starting async job submit for node 69bfae667544ab5cd4a7e40b 2026-03-24 08:55:17,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:17,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae667544ab5cd4a7e40b HTTP/1.1" 200 None 2026-03-24 08:55:17,780 - lava_callback - DEBUG - Processing node 69bfae667544ab5cd4a7e40b: baseline-arm-clabbe 2026-03-24 08:55:17,780 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfae667544ab5cd4a7e40b 2026-03-24 08:55:17,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:17,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae667544ab5cd4a7e40b HTTP/1.1" 200 None 2026-03-24 08:55:17,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:17,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 08:55:17,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:17,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfae667544ab5cd4a7e40b HTTP/1.1" 200 None 2026-03-24 08:55:17,891 - lava_callback - ERROR - Error processing callback for node 69bfae667544ab5cd4a7e40b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:55:18,356 - lava_callback - INFO - Received callback for node 69bfae677544ab5cd4a7e40e INFO: 172.20.0.1:42898 - "POST /node/69bfae677544ab5cd4a7e40e HTTP/1.0" 202 Accepted 2026-03-24 08:55:18,377 - lava_callback - INFO - Starting async job submit for node 69bfae677544ab5cd4a7e40e 2026-03-24 08:55:18,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:18,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae677544ab5cd4a7e40e HTTP/1.1" 200 None 2026-03-24 08:55:18,408 - lava_callback - DEBUG - Processing node 69bfae677544ab5cd4a7e40e: baseline-arm-clabbe 2026-03-24 08:55:18,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfae677544ab5cd4a7e40e 2026-03-24 08:55:18,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:18,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae677544ab5cd4a7e40e HTTP/1.1" 200 None 2026-03-24 08:55:18,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:18,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 08:55:18,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:18,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfae677544ab5cd4a7e40e HTTP/1.1" 200 None 2026-03-24 08:55:18,502 - lava_callback - ERROR - Error processing callback for node 69bfae677544ab5cd4a7e40e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:55:19,032 - lava_callback - INFO - Received callback for node 69bfae6a7544ab5cd4a7e417 INFO: 172.20.0.1:42914 - "POST /node/69bfae6a7544ab5cd4a7e417 HTTP/1.0" 202 Accepted 2026-03-24 08:55:19,041 - lava_callback - INFO - Starting async job submit for node 69bfae6a7544ab5cd4a7e417 2026-03-24 08:55:19,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:19,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae6a7544ab5cd4a7e417 HTTP/1.1" 200 None 2026-03-24 08:55:19,098 - lava_callback - DEBUG - Processing node 69bfae6a7544ab5cd4a7e417: baseline-arm-clabbe 2026-03-24 08:55:19,098 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfae6a7544ab5cd4a7e417 2026-03-24 08:55:19,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:19,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfae6a7544ab5cd4a7e417 HTTP/1.1" 200 None 2026-03-24 08:55:19,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:19,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 08:55:19,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:19,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfae6a7544ab5cd4a7e417 HTTP/1.1" 200 None 2026-03-24 08:55:19,173 - lava_callback - ERROR - Error processing callback for node 69bfae6a7544ab5cd4a7e417: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:55:24,184 - lava_callback - INFO - Received callback for node 69c2515f4a2c5046eb880252 INFO: 172.20.0.1:40474 - "POST /node/69c2515f4a2c5046eb880252 HTTP/1.0" 202 Accepted 2026-03-24 08:55:24,191 - lava_callback - INFO - Starting async job submit for node 69c2515f4a2c5046eb880252 2026-03-24 08:55:24,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:24,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2515f4a2c5046eb880252 HTTP/1.1" 200 None 2026-03-24 08:55:24,241 - lava_callback - DEBUG - Processing node 69c2515f4a2c5046eb880252: baseline-arm64 2026-03-24 08:55:24,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:55:24,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:24,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-24 08:55:24,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:55:24,773 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515f4a2c5046eb880252/log.txt.gz 2026-03-24 08:55:24,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:55:25,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:55:25,260 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515f4a2c5046eb880252/lava-logs.yaml 2026-03-24 08:55:25,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:55:25,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:55:25,463 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515f4a2c5046eb880252/lava_callback.json.gz 2026-03-24 08:55:25,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:25,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2515f4a2c5046eb880252 HTTP/1.1" 200 None 2026-03-24 08:55:25,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:25,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:55:25,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:25,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2515f4a2c5046eb880252 HTTP/1.1" 200 None 2026-03-24 08:55:25,617 - lava_callback - INFO - Completed processing callback for node 69c2515f4a2c5046eb880252 INFO: 172.20.0.1:40484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34020 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:55:37,880 - lava_callback - INFO - Received callback for node 69c2516f4a2c5046eb880284 INFO: 172.20.0.1:34030 - "POST /node/69c2516f4a2c5046eb880284 HTTP/1.0" 202 Accepted 2026-03-24 08:55:37,890 - lava_callback - INFO - Starting async job submit for node 69c2516f4a2c5046eb880284 2026-03-24 08:55:37,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:37,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2516f4a2c5046eb880284 HTTP/1.1" 200 None 2026-03-24 08:55:37,921 - lava_callback - DEBUG - Processing node 69c2516f4a2c5046eb880284: baseline-nfs-arm64 2026-03-24 08:55:37,923 - lava_callback - WARNING - Failed to upload log 2026-03-24 08:55:37,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:55:38,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:55:38,147 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516f4a2c5046eb880284/lava-logs.yaml 2026-03-24 08:55:38,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:55:38,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:55:38,287 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516f4a2c5046eb880284/lava_callback.json.gz 2026-03-24 08:55:38,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:38,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2516f4a2c5046eb880284 HTTP/1.1" 200 None 2026-03-24 08:55:38,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:38,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:55:38,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:38,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2516f4a2c5046eb880284 HTTP/1.1" 200 None 2026-03-24 08:55:38,391 - lava_callback - INFO - Completed processing callback for node 69c2516f4a2c5046eb880284 INFO: 172.20.0.1:32936 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:55:54,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:55:54,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48974 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:56:00,800 - lava_callback - INFO - Received callback for node 69c251544a2c5046eb880222 INFO: 172.20.0.1:41100 - "POST /node/69c251544a2c5046eb880222 HTTP/1.0" 202 Accepted 2026-03-24 08:56:00,807 - lava_callback - INFO - Starting async job submit for node 69c251544a2c5046eb880222 2026-03-24 08:56:00,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:00,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251544a2c5046eb880222 HTTP/1.1" 200 None 2026-03-24 08:56:00,909 - lava_callback - DEBUG - Processing node 69c251544a2c5046eb880222: baseline-arm64 2026-03-24 08:56:01,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:01,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:01,772 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251544a2c5046eb880222/log.txt.gz 2026-03-24 08:56:01,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:02,246 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:02,248 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251544a2c5046eb880222/lava-logs.yaml 2026-03-24 08:56:02,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:02,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:02,410 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251544a2c5046eb880222/lava_callback.json.gz 2026-03-24 08:56:02,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:02,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251544a2c5046eb880222 HTTP/1.1" 200 None 2026-03-24 08:56:02,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:02,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:02,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:02,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251544a2c5046eb880222 HTTP/1.1" 200 None 2026-03-24 08:56:02,549 - lava_callback - INFO - Completed processing callback for node 69c251544a2c5046eb880222 INFO: 172.20.0.1:41116 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:56:11,194 - lava_callback - INFO - Received callback for node 69c24eec4a2c5046eb880068 INFO: 172.20.0.1:49122 - "POST /node/69c24eec4a2c5046eb880068 HTTP/1.0" 202 Accepted 2026-03-24 08:56:11,202 - lava_callback - INFO - Starting async job submit for node 69c24eec4a2c5046eb880068 2026-03-24 08:56:11,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:11,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eec4a2c5046eb880068 HTTP/1.1" 200 None 2026-03-24 08:56:11,281 - lava_callback - DEBUG - Processing node 69c24eec4a2c5046eb880068: ltp-fs_bind 2026-03-24 08:56:11,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:12,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:12,022 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c24eec4a2c5046eb880068/log.txt.gz 2026-03-24 08:56:12,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:12,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:12,620 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c24eec4a2c5046eb880068/lava-logs.yaml 2026-03-24 08:56:12,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:12,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:12,880 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c24eec4a2c5046eb880068/lava_callback.json.gz 2026-03-24 08:56:12,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:12,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24eec4a2c5046eb880068 HTTP/1.1" 200 None 2026-03-24 08:56:12,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:12,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:56:12,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:12,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24eec4a2c5046eb880068 HTTP/1.1" 200 None 2026-03-24 08:56:12,997 - lava_callback - INFO - Completed processing callback for node 69c24eec4a2c5046eb880068 INFO: 172.20.0.1:49136 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 08:56:15,686 - lava_callback - INFO - Received callback for node 69bfaeaa7544ab5cd4a7e4b8 INFO: 172.20.0.1:49142 - "POST /node/69bfaeaa7544ab5cd4a7e4b8 HTTP/1.0" 202 Accepted 2026-03-24 08:56:15,692 - lava_callback - INFO - Starting async job submit for node 69bfaeaa7544ab5cd4a7e4b8 2026-03-24 08:56:15,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:15,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfaeaa7544ab5cd4a7e4b8 HTTP/1.1" 200 None 2026-03-24 08:56:15,709 - lava_callback - DEBUG - Processing node 69bfaeaa7544ab5cd4a7e4b8: baseline-arm64-clabbe 2026-03-24 08:56:15,710 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfaeaa7544ab5cd4a7e4b8 2026-03-24 08:56:15,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:15,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfaeaa7544ab5cd4a7e4b8 HTTP/1.1" 200 None 2026-03-24 08:56:15,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:15,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 08:56:15,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:15,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfaeaa7544ab5cd4a7e4b8 HTTP/1.1" 200 None 2026-03-24 08:56:15,821 - lava_callback - ERROR - Error processing callback for node 69bfaeaa7544ab5cd4a7e4b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49160 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:56:18,892 - lava_callback - INFO - Received callback for node 69c2515a4a2c5046eb880231 INFO: 172.20.0.1:49174 - "POST /node/69c2515a4a2c5046eb880231 HTTP/1.0" 202 Accepted 2026-03-24 08:56:18,902 - lava_callback - INFO - Starting async job submit for node 69c2515a4a2c5046eb880231 2026-03-24 08:56:18,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:18,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2515a4a2c5046eb880231 HTTP/1.1" 200 None 2026-03-24 08:56:18,942 - lava_callback - DEBUG - Processing node 69c2515a4a2c5046eb880231: baseline-arm64 2026-03-24 08:56:19,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:19,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:19,377 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515a4a2c5046eb880231/log.txt.gz 2026-03-24 08:56:19,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:19,963 - lava_callback - INFO - Received callback for node 69c24ef04a2c5046eb88006e INFO: 172.20.0.1:34084 - "POST /node/69c24ef04a2c5046eb88006e HTTP/1.0" 202 Accepted 2026-03-24 08:56:19,978 - lava_callback - INFO - Starting async job submit for node 69c24ef04a2c5046eb88006e 2026-03-24 08:56:20,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:20,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef04a2c5046eb88006e HTTP/1.1" 200 None 2026-03-24 08:56:20,073 - lava_callback - DEBUG - Processing node 69c24ef04a2c5046eb88006e: ltp-input 2026-03-24 08:56:20,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:20,170 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515a4a2c5046eb880231/lava-logs.yaml 2026-03-24 08:56:20,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:20,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:20,493 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515a4a2c5046eb880231/lava_callback.json.gz 2026-03-24 08:56:20,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:20,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:20,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2515a4a2c5046eb880231 HTTP/1.1" 200 None 2026-03-24 08:56:20,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:20,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:20,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:20,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2515a4a2c5046eb880231 HTTP/1.1" 200 None 2026-03-24 08:56:20,644 - lava_callback - INFO - Completed processing callback for node 69c2515a4a2c5046eb880231 2026-03-24 08:56:20,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:20,745 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c24ef04a2c5046eb88006e/log.txt.gz 2026-03-24 08:56:21,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:21,343 - lava_callback - INFO - Received callback for node 69c2515b4a2c5046eb880248 INFO: 172.20.0.1:34092 - "POST /node/69c2515b4a2c5046eb880248 HTTP/1.0" 202 Accepted 2026-03-24 08:56:21,353 - lava_callback - INFO - Starting async job submit for node 69c2515b4a2c5046eb880248 2026-03-24 08:56:21,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:21,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2515b4a2c5046eb880248 HTTP/1.1" 200 None 2026-03-24 08:56:21,421 - lava_callback - DEBUG - Processing node 69c2515b4a2c5046eb880248: baseline-arm64 2026-03-24 08:56:21,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:21,571 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c24ef04a2c5046eb88006e/lava-logs.yaml 2026-03-24 08:56:21,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:21,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:21,890 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:21,892 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c24ef04a2c5046eb88006e/lava_callback.json.gz 2026-03-24 08:56:21,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:21,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:21,924 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515b4a2c5046eb880248/log.txt.gz 2026-03-24 08:56:21,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef04a2c5046eb88006e HTTP/1.1" 200 None 2026-03-24 08:56:22,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:22,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:56:22,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:22,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:22,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ef04a2c5046eb88006e HTTP/1.1" 200 None 2026-03-24 08:56:22,181 - lava_callback - INFO - Completed processing callback for node 69c24ef04a2c5046eb88006e 2026-03-24 08:56:22,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:22,750 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515b4a2c5046eb880248/lava-logs.yaml 2026-03-24 08:56:22,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:22,946 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:22,948 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515b4a2c5046eb880248/lava_callback.json.gz 2026-03-24 08:56:22,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:22,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2515b4a2c5046eb880248 HTTP/1.1" 200 None 2026-03-24 08:56:22,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:23,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:23,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:23,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2515b4a2c5046eb880248 HTTP/1.1" 200 None 2026-03-24 08:56:23,060 - lava_callback - INFO - Completed processing callback for node 69c2515b4a2c5046eb880248 2026-03-24 08:56:24,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:24,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:34104 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:56:29,351 - lava_callback - INFO - Received callback for node 69c2515d4a2c5046eb88024f INFO: 172.20.0.1:34120 - "POST /node/69c2515d4a2c5046eb88024f HTTP/1.0" 202 Accepted 2026-03-24 08:56:29,358 - lava_callback - INFO - Starting async job submit for node 69c2515d4a2c5046eb88024f 2026-03-24 08:56:29,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:29,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2515d4a2c5046eb88024f HTTP/1.1" 200 None 2026-03-24 08:56:29,407 - lava_callback - DEBUG - Processing node 69c2515d4a2c5046eb88024f: baseline-arm64 2026-03-24 08:56:29,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:30,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:30,077 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515d4a2c5046eb88024f/log.txt.gz 2026-03-24 08:56:30,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:30,530 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:30,532 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515d4a2c5046eb88024f/lava-logs.yaml 2026-03-24 08:56:30,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:30,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:30,664 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515d4a2c5046eb88024f/lava_callback.json.gz 2026-03-24 08:56:30,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:30,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2515d4a2c5046eb88024f HTTP/1.1" 200 None 2026-03-24 08:56:30,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:30,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:30,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:30,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2515d4a2c5046eb88024f HTTP/1.1" 200 None 2026-03-24 08:56:30,776 - lava_callback - INFO - Completed processing callback for node 69c2515d4a2c5046eb88024f 2026-03-24 08:56:31,496 - lava_callback - INFO - Received callback for node 69c251624a2c5046eb88025b INFO: 172.20.0.1:37930 - "POST /node/69c251624a2c5046eb88025b HTTP/1.0" 202 Accepted 2026-03-24 08:56:31,503 - lava_callback - INFO - Starting async job submit for node 69c251624a2c5046eb88025b 2026-03-24 08:56:31,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:31,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251624a2c5046eb88025b HTTP/1.1" 200 None 2026-03-24 08:56:31,553 - lava_callback - DEBUG - Processing node 69c251624a2c5046eb88025b: baseline-arm64 2026-03-24 08:56:31,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:32,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:32,169 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251624a2c5046eb88025b/log.txt.gz 2026-03-24 08:56:32,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:32,588 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:32,590 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251624a2c5046eb88025b/lava-logs.yaml 2026-03-24 08:56:32,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:32,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:32,736 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251624a2c5046eb88025b/lava_callback.json.gz 2026-03-24 08:56:32,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:32,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251624a2c5046eb88025b HTTP/1.1" 200 None 2026-03-24 08:56:32,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:32,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:32,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:32,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251624a2c5046eb88025b HTTP/1.1" 200 None 2026-03-24 08:56:32,840 - lava_callback - INFO - Completed processing callback for node 69c251624a2c5046eb88025b 2026-03-24 08:56:34,503 - lava_callback - INFO - Received callback for node 69c251564a2c5046eb880228 INFO: 172.20.0.1:37934 - "POST /node/69c251564a2c5046eb880228 HTTP/1.0" 202 Accepted 2026-03-24 08:56:34,510 - lava_callback - INFO - Starting async job submit for node 69c251564a2c5046eb880228 2026-03-24 08:56:34,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:34,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251564a2c5046eb880228 HTTP/1.1" 200 None 2026-03-24 08:56:34,560 - lava_callback - DEBUG - Processing node 69c251564a2c5046eb880228: baseline-arm64 2026-03-24 08:56:34,707 - lava_callback - WARNING - Failed to upload log 2026-03-24 08:56:34,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:34,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:34,967 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251564a2c5046eb880228/lava-logs.yaml 2026-03-24 08:56:34,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:35,045 - lava_callback - INFO - Received callback for node 69c251534a2c5046eb88021f INFO: 172.20.0.1:37936 - "POST /node/69c251534a2c5046eb88021f HTTP/1.0" 202 Accepted 2026-03-24 08:56:35,052 - lava_callback - INFO - Starting async job submit for node 69c251534a2c5046eb88021f 2026-03-24 08:56:35,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:35,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:35,091 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251564a2c5046eb880228/lava_callback.json.gz 2026-03-24 08:56:35,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:35,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251534a2c5046eb88021f HTTP/1.1" 200 None 2026-03-24 08:56:35,133 - lava_callback - DEBUG - Processing node 69c251534a2c5046eb88021f: baseline-arm64 2026-03-24 08:56:35,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251564a2c5046eb880228 HTTP/1.1" 200 None 2026-03-24 08:56:35,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:35,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:35,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:35,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251564a2c5046eb880228 HTTP/1.1" 200 None 2026-03-24 08:56:35,576 - lava_callback - INFO - Completed processing callback for node 69c251564a2c5046eb880228 2026-03-24 08:56:35,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:35,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:35,778 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251534a2c5046eb88021f/log.txt.gz 2026-03-24 08:56:35,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:36,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:36,205 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251534a2c5046eb88021f/lava-logs.yaml 2026-03-24 08:56:36,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:36,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:36,396 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251534a2c5046eb88021f/lava_callback.json.gz 2026-03-24 08:56:36,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:36,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251534a2c5046eb88021f HTTP/1.1" 200 None 2026-03-24 08:56:36,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:36,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:36,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:37948 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:56:36,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251534a2c5046eb88021f HTTP/1.1" 200 None 2026-03-24 08:56:36,562 - lava_callback - INFO - Completed processing callback for node 69c251534a2c5046eb88021f 2026-03-24 08:56:39,858 - lava_callback - INFO - Received callback for node 69c24ef14a2c5046eb880071 INFO: 172.20.0.1:50064 - "POST /node/69c24ef14a2c5046eb880071 HTTP/1.0" 202 Accepted 2026-03-24 08:56:39,867 - lava_callback - INFO - Starting async job submit for node 69c24ef14a2c5046eb880071 2026-03-24 08:56:39,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:39,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef14a2c5046eb880071 HTTP/1.1" 200 None 2026-03-24 08:56:39,916 - lava_callback - DEBUG - Processing node 69c24ef14a2c5046eb880071: ltp-ipc 2026-03-24 08:56:40,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:40,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:40,575 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c24ef14a2c5046eb880071/log.txt.gz 2026-03-24 08:56:40,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:41,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:41,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c24ef14a2c5046eb880071/lava-logs.yaml 2026-03-24 08:56:41,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:41,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:41,275 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c24ef14a2c5046eb880071/lava_callback.json.gz 2026-03-24 08:56:41,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:41,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef14a2c5046eb880071 HTTP/1.1" 200 None 2026-03-24 08:56:41,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:41,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:56:41,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:41,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ef14a2c5046eb880071 HTTP/1.1" 200 None 2026-03-24 08:56:41,366 - lava_callback - INFO - Completed processing callback for node 69c24ef14a2c5046eb880071 2026-03-24 08:56:41,735 - lava_callback - INFO - Received callback for node 69c2515c4a2c5046eb88024c INFO: 172.20.0.1:50078 - "POST /node/69c2515c4a2c5046eb88024c HTTP/1.0" 202 Accepted 2026-03-24 08:56:41,747 - lava_callback - INFO - Starting async job submit for node 69c2515c4a2c5046eb88024c 2026-03-24 08:56:41,748 - lava_callback - INFO - Received callback for node 69c251604a2c5046eb880255 INFO: 172.20.0.1:50086 - "POST /node/69c251604a2c5046eb880255 HTTP/1.0" 202 Accepted 2026-03-24 08:56:41,796 - lava_callback - INFO - Starting async job submit for node 69c251604a2c5046eb880255 2026-03-24 08:56:41,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:41,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:41,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251604a2c5046eb880255 HTTP/1.1" 200 None 2026-03-24 08:56:41,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2515c4a2c5046eb88024c HTTP/1.1" 200 None 2026-03-24 08:56:41,904 - lava_callback - DEBUG - Processing node 69c2515c4a2c5046eb88024c: baseline-arm64 2026-03-24 08:56:41,910 - lava_callback - DEBUG - Processing node 69c251604a2c5046eb880255: baseline-arm64 2026-03-24 08:56:42,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:42,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:42,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:42,998 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251604a2c5046eb880255/log.txt.gz 2026-03-24 08:56:43,034 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:43,045 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515c4a2c5046eb88024c/log.txt.gz 2026-03-24 08:56:43,348 - lava_callback - INFO - Received callback for node 69c251734a2c5046eb880292 2026-03-24 08:56:43,366 - lava_callback - INFO - Starting async job submit for node 69c251734a2c5046eb880292 INFO: 172.20.0.1:50090 - "POST /node/69c251734a2c5046eb880292 HTTP/1.0" 202 Accepted 2026-03-24 08:56:43,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:43,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:43,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251734a2c5046eb880292 HTTP/1.1" 200 None 2026-03-24 08:56:43,504 - lava_callback - DEBUG - Processing node 69c251734a2c5046eb880292: baseline-nfs-arm64 2026-03-24 08:56:43,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:43,703 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:43,746 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251604a2c5046eb880255/lava-logs.yaml 2026-03-24 08:56:43,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:44,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:44,189 - lava_callback - INFO - Received callback for node 69c2516d4a2c5046eb88027e 2026-03-24 08:56:44,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:44,203 - lava_callback - INFO - Starting async job submit for node 69c2516d4a2c5046eb88027e INFO: 172.20.0.1:50106 - "POST /node/69c2516d4a2c5046eb88027e HTTP/1.0" 202 Accepted 2026-03-24 08:56:44,212 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515c4a2c5046eb88024c/lava-logs.yaml 2026-03-24 08:56:44,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:44,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:44,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:44,299 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251604a2c5046eb880255/lava_callback.json.gz 2026-03-24 08:56:44,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:44,324 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251734a2c5046eb880292/log.txt.gz 2026-03-24 08:56:44,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:44,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2516d4a2c5046eb88027e HTTP/1.1" 200 None 2026-03-24 08:56:44,356 - lava_callback - DEBUG - Processing node 69c2516d4a2c5046eb88027e: baseline-nfs-arm64 2026-03-24 08:56:44,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251604a2c5046eb880255 HTTP/1.1" 200 None 2026-03-24 08:56:44,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:44,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:44,641 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2515c4a2c5046eb88024c/lava_callback.json.gz 2026-03-24 08:56:44,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:44,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:44,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:44,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:45,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2515c4a2c5046eb88024c HTTP/1.1" 200 None 2026-03-24 08:56:45,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:45,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251604a2c5046eb880255 HTTP/1.1" 200 None 2026-03-24 08:56:45,054 - lava_callback - INFO - Completed processing callback for node 69c251604a2c5046eb880255 2026-03-24 08:56:45,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:45,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:45,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2515c4a2c5046eb88024c HTTP/1.1" 200 None 2026-03-24 08:56:45,477 - lava_callback - INFO - Completed processing callback for node 69c2515c4a2c5046eb88024c 2026-03-24 08:56:45,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:45,822 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:45,824 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516d4a2c5046eb88027e/log.txt.gz 2026-03-24 08:56:45,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:45,891 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251734a2c5046eb880292/lava-logs.yaml 2026-03-24 08:56:45,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:46,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:46,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:46,193 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251734a2c5046eb880292/lava_callback.json.gz 2026-03-24 08:56:46,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:46,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251734a2c5046eb880292 HTTP/1.1" 200 None 2026-03-24 08:56:46,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:46,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:46,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:46,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251734a2c5046eb880292 HTTP/1.1" 200 None 2026-03-24 08:56:46,283 - lava_callback - INFO - Completed processing callback for node 69c251734a2c5046eb880292 2026-03-24 08:56:46,358 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:46,360 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516d4a2c5046eb88027e/lava-logs.yaml 2026-03-24 08:56:46,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:50118 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:56:46,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:46,520 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516d4a2c5046eb88027e/lava_callback.json.gz 2026-03-24 08:56:46,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:46,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2516d4a2c5046eb88027e HTTP/1.1" 200 None 2026-03-24 08:56:46,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:46,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:46,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:46,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2516d4a2c5046eb88027e HTTP/1.1" 200 None 2026-03-24 08:56:46,623 - lava_callback - INFO - Completed processing callback for node 69c2516d4a2c5046eb88027e 2026-03-24 08:56:53,347 - lava_callback - INFO - Received callback for node 69c251a14a2c5046eb88033d INFO: 172.20.0.1:50908 - "POST /node/69c251a14a2c5046eb88033d HTTP/1.0" 202 Accepted 2026-03-24 08:56:53,354 - lava_callback - INFO - Starting async job submit for node 69c251a14a2c5046eb88033d 2026-03-24 08:56:53,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:53,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251a14a2c5046eb88033d HTTP/1.1" 200 None 2026-03-24 08:56:53,414 - lava_callback - DEBUG - Processing node 69c251a14a2c5046eb88033d: baseline-arm64-foundriesio 2026-03-24 08:56:53,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:53,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:53,910 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c251a14a2c5046eb88033d/log.txt.gz 2026-03-24 08:56:54,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:54,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:54,321 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c251a14a2c5046eb88033d/lava-logs.yaml 2026-03-24 08:56:54,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:54,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:54,536 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c251a14a2c5046eb88033d/lava_callback.json.gz 2026-03-24 08:56:54,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:54,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251a14a2c5046eb88033d HTTP/1.1" 200 None 2026-03-24 08:56:54,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:54,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:54,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:54,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251a14a2c5046eb88033d HTTP/1.1" 200 None 2026-03-24 08:56:54,636 - lava_callback - INFO - Completed processing callback for node 69c251a14a2c5046eb88033d 2026-03-24 08:56:54,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:54,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-24 08:56:54,985 - lava_callback - INFO - Received callback for node 69c251a24a2c5046eb880340 INFO: 172.20.0.1:50916 - "POST /node/69c251a24a2c5046eb880340 HTTP/1.0" 202 Accepted 2026-03-24 08:56:54,998 - lava_callback - INFO - Starting async job submit for node 69c251a24a2c5046eb880340 2026-03-24 08:56:55,070 - lava_callback - INFO - Received callback for node 69c251644a2c5046eb880266 2026-03-24 08:56:55,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:50920 - "POST /node/69c251644a2c5046eb880266 HTTP/1.0" 202 Accepted 2026-03-24 08:56:55,084 - lava_callback - INFO - Starting async job submit for node 69c251644a2c5046eb880266 2026-03-24 08:56:55,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:55,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251a24a2c5046eb880340 HTTP/1.1" 200 None 2026-03-24 08:56:55,130 - lava_callback - DEBUG - Processing node 69c251a24a2c5046eb880340: baseline-arm64-foundriesio 2026-03-24 08:56:55,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251644a2c5046eb880266 HTTP/1.1" 200 None 2026-03-24 08:56:55,205 - lava_callback - DEBUG - Processing node 69c251644a2c5046eb880266: baseline-arm64 2026-03-24 08:56:55,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:55,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:55,976 - lava_callback - INFO - Received callback for node 69c251764a2c5046eb8802a0 INFO: 172.20.0.1:50928 - "POST /node/69c251764a2c5046eb8802a0 HTTP/1.0" 202 Accepted 2026-03-24 08:56:55,986 - lava_callback - INFO - Starting async job submit for node 69c251764a2c5046eb8802a0 2026-03-24 08:56:56,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:56,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251764a2c5046eb8802a0 HTTP/1.1" 200 None 2026-03-24 08:56:56,072 - lava_callback - DEBUG - Processing node 69c251764a2c5046eb8802a0: baseline-nfs-arm64 2026-03-24 08:56:56,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:56,323 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251644a2c5046eb880266/log.txt.gz 2026-03-24 08:56:56,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:56,396 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c251a24a2c5046eb880340/log.txt.gz INFO: 172.20.0.1:50932 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:56:56,907 - lava_callback - INFO - Received callback for node 69c2519d4a2c5046eb880334 INFO: 172.20.0.1:50934 - "POST /node/69c2519d4a2c5046eb880334 HTTP/1.0" 202 Accepted 2026-03-24 08:56:56,965 - lava_callback - INFO - Starting async job submit for node 69c2519d4a2c5046eb880334 2026-03-24 08:56:56,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:57,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:57,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:57,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2519d4a2c5046eb880334 HTTP/1.1" 200 None 2026-03-24 08:56:57,134 - lava_callback - DEBUG - Processing node 69c2519d4a2c5046eb880334: baseline-arm64-pengutronix 2026-03-24 08:56:57,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:57,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:57,572 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:57,575 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251764a2c5046eb8802a0/log.txt.gz 2026-03-24 08:56:57,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:57,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:57,674 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251644a2c5046eb880266/lava-logs.yaml 2026-03-24 08:56:57,675 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c251a24a2c5046eb880340/lava-logs.yaml 2026-03-24 08:56:57,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:57,716 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2519d4a2c5046eb880334/log.txt.gz 2026-03-24 08:56:57,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:57,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:57,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:57,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:58,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:58,298 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251644a2c5046eb880266/lava_callback.json.gz 2026-03-24 08:56:58,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:58,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251644a2c5046eb880266 HTTP/1.1" 200 None 2026-03-24 08:56:58,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:58,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:58,366 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c251a24a2c5046eb880340/lava_callback.json.gz 2026-03-24 08:56:58,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:58,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:58,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:58,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251644a2c5046eb880266 HTTP/1.1" 200 None 2026-03-24 08:56:58,481 - lava_callback - INFO - Completed processing callback for node 69c251644a2c5046eb880266 2026-03-24 08:56:58,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251a24a2c5046eb880340 HTTP/1.1" 200 None 2026-03-24 08:56:58,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:58,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:58,514 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2519d4a2c5046eb880334/lava-logs.yaml 2026-03-24 08:56:58,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:58,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:58,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:58,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251a24a2c5046eb880340 HTTP/1.1" 200 None 2026-03-24 08:56:58,670 - lava_callback - INFO - Completed processing callback for node 69c251a24a2c5046eb880340 2026-03-24 08:56:58,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:58,701 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251764a2c5046eb8802a0/lava-logs.yaml 2026-03-24 08:56:58,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:56:58,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:58,906 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2519d4a2c5046eb880334/lava_callback.json.gz 2026-03-24 08:56:58,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:58,965 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:56:58,967 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251764a2c5046eb8802a0/lava_callback.json.gz 2026-03-24 08:56:58,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2519d4a2c5046eb880334 HTTP/1.1" 200 None 2026-03-24 08:56:58,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:59,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:59,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:59,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:59,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251764a2c5046eb8802a0 HTTP/1.1" 200 None 2026-03-24 08:56:59,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:59,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:56:59,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:56:59,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2519d4a2c5046eb880334 HTTP/1.1" 200 None 2026-03-24 08:56:59,081 - lava_callback - INFO - Completed processing callback for node 69c2519d4a2c5046eb880334 2026-03-24 08:56:59,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251764a2c5046eb8802a0 HTTP/1.1" 200 None 2026-03-24 08:56:59,205 - lava_callback - INFO - Completed processing callback for node 69c251764a2c5046eb8802a0 2026-03-24 08:57:03,168 - lava_callback - INFO - Received callback for node 69c251574a2c5046eb88022b INFO: 172.20.0.1:33396 - "POST /node/69c251574a2c5046eb88022b HTTP/1.0" 202 Accepted 2026-03-24 08:57:03,176 - lava_callback - INFO - Starting async job submit for node 69c251574a2c5046eb88022b 2026-03-24 08:57:03,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:03,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251574a2c5046eb88022b HTTP/1.1" 200 None 2026-03-24 08:57:03,245 - lava_callback - DEBUG - Processing node 69c251574a2c5046eb88022b: baseline-arm64 2026-03-24 08:57:03,675 - lava_callback - WARNING - Failed to upload log 2026-03-24 08:57:03,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:04,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:04,019 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251574a2c5046eb88022b/lava-logs.yaml 2026-03-24 08:57:04,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:04,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:04,198 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251574a2c5046eb88022b/lava_callback.json.gz 2026-03-24 08:57:04,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:04,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251574a2c5046eb88022b HTTP/1.1" 200 None 2026-03-24 08:57:04,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:04,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:04,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:04,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251574a2c5046eb88022b HTTP/1.1" 200 None 2026-03-24 08:57:04,478 - lava_callback - INFO - Completed processing callback for node 69c251574a2c5046eb88022b 2026-03-24 08:57:05,554 - lava_callback - INFO - Received callback for node 69c251674a2c5046eb88026e INFO: 172.20.0.1:33410 - "POST /node/69c251674a2c5046eb88026e HTTP/1.0" 202 Accepted 2026-03-24 08:57:05,593 - lava_callback - INFO - Starting async job submit for node 69c251674a2c5046eb88026e 2026-03-24 08:57:05,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:05,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251674a2c5046eb88026e HTTP/1.1" 200 None 2026-03-24 08:57:05,682 - lava_callback - DEBUG - Processing node 69c251674a2c5046eb88026e: baseline-nfs-arm64 2026-03-24 08:57:06,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:06,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:06,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251674a2c5046eb88026e/log.txt.gz 2026-03-24 08:57:06,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:06,539 - lava_callback - INFO - Received callback for node 69c2519e4a2c5046eb880337 INFO: 172.20.0.1:33416 - "POST /node/69c2519e4a2c5046eb880337 HTTP/1.0" 202 Accepted 2026-03-24 08:57:06,548 - lava_callback - INFO - Starting async job submit for node 69c2519e4a2c5046eb880337 INFO: 172.20.0.1:33418 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:57:06,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:06,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2519e4a2c5046eb880337 HTTP/1.1" 200 None 2026-03-24 08:57:06,622 - lava_callback - DEBUG - Processing node 69c2519e4a2c5046eb880337: baseline-arm64-pengutronix 2026-03-24 08:57:06,763 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:06,801 - lava_callback - INFO - Received callback for node 69c251554a2c5046eb880225 INFO: 172.20.0.1:33422 - "POST /node/69c251554a2c5046eb880225 HTTP/1.0" 202 Accepted 2026-03-24 08:57:06,813 - lava_callback - INFO - Starting async job submit for node 69c251554a2c5046eb880225 2026-03-24 08:57:06,854 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251674a2c5046eb88026e/lava-logs.yaml 2026-03-24 08:57:06,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:06,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:06,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:06,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251554a2c5046eb880225 HTTP/1.1" 200 None 2026-03-24 08:57:06,924 - lava_callback - DEBUG - Processing node 69c251554a2c5046eb880225: baseline-arm64 2026-03-24 08:57:07,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:07,090 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2519e4a2c5046eb880337/log.txt.gz 2026-03-24 08:57:07,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:07,167 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251674a2c5046eb88026e/lava_callback.json.gz 2026-03-24 08:57:07,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:07,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:07,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251674a2c5046eb88026e HTTP/1.1" 200 None 2026-03-24 08:57:07,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:07,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:07,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:07,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:07,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251674a2c5046eb88026e HTTP/1.1" 200 None 2026-03-24 08:57:07,621 - lava_callback - INFO - Completed processing callback for node 69c251674a2c5046eb88026e 2026-03-24 08:57:07,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:07,779 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251554a2c5046eb880225/log.txt.gz 2026-03-24 08:57:07,804 - lava_callback - INFO - Received callback for node 69c2518e4a2c5046eb8802f4 INFO: 172.20.0.1:33434 - "POST /node/69c2518e4a2c5046eb8802f4 HTTP/1.0" 202 Accepted 2026-03-24 08:57:07,826 - lava_callback - INFO - Starting async job submit for node 69c2518e4a2c5046eb8802f4 2026-03-24 08:57:07,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:07,866 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2519e4a2c5046eb880337/lava-logs.yaml 2026-03-24 08:57:07,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:07,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:08,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2518e4a2c5046eb8802f4 HTTP/1.1" 200 None 2026-03-24 08:57:08,034 - lava_callback - DEBUG - Processing node 69c2518e4a2c5046eb8802f4: baseline-arm64-broonie 2026-03-24 08:57:08,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:08,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:08,285 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2519e4a2c5046eb880337/lava_callback.json.gz 2026-03-24 08:57:08,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:08,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2519e4a2c5046eb880337 HTTP/1.1" 200 None 2026-03-24 08:57:08,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:08,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:08,510 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251554a2c5046eb880225/lava-logs.yaml 2026-03-24 08:57:08,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:08,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:08,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:08,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:08,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2519e4a2c5046eb880337 HTTP/1.1" 200 None 2026-03-24 08:57:08,572 - lava_callback - INFO - Completed processing callback for node 69c2519e4a2c5046eb880337 2026-03-24 08:57:08,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:08,717 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251554a2c5046eb880225/lava_callback.json.gz 2026-03-24 08:57:08,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:08,746 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:08,748 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2518e4a2c5046eb8802f4/log.txt.gz 2026-03-24 08:57:08,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251554a2c5046eb880225 HTTP/1.1" 200 None 2026-03-24 08:57:08,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:08,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:08,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:08,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:08,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251554a2c5046eb880225 HTTP/1.1" 200 None 2026-03-24 08:57:08,933 - lava_callback - INFO - Completed processing callback for node 69c251554a2c5046eb880225 2026-03-24 08:57:09,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:09,132 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2518e4a2c5046eb8802f4/lava-logs.yaml 2026-03-24 08:57:09,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:09,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:09,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2518e4a2c5046eb8802f4/lava_callback.json.gz 2026-03-24 08:57:09,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:09,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2518e4a2c5046eb8802f4 HTTP/1.1" 200 None 2026-03-24 08:57:09,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:09,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:09,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:09,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2518e4a2c5046eb8802f4 HTTP/1.1" 200 None 2026-03-24 08:57:09,354 - lava_callback - INFO - Completed processing callback for node 69c2518e4a2c5046eb8802f4 2026-03-24 08:57:13,796 - lava_callback - INFO - Received callback for node 69c2519f4a2c5046eb88033a INFO: 172.20.0.1:35450 - "POST /node/69c2519f4a2c5046eb88033a HTTP/1.0" 202 Accepted 2026-03-24 08:57:13,802 - lava_callback - INFO - Starting async job submit for node 69c2519f4a2c5046eb88033a 2026-03-24 08:57:13,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:13,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2519f4a2c5046eb88033a HTTP/1.1" 200 None 2026-03-24 08:57:13,837 - lava_callback - DEBUG - Processing node 69c2519f4a2c5046eb88033a: baseline-arm64-pengutronix 2026-03-24 08:57:14,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:14,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:14,235 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2519f4a2c5046eb88033a/log.txt.gz 2026-03-24 08:57:14,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:14,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:14,549 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2519f4a2c5046eb88033a/lava-logs.yaml 2026-03-24 08:57:14,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:14,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:14,689 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c2519f4a2c5046eb88033a/lava_callback.json.gz 2026-03-24 08:57:14,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:14,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2519f4a2c5046eb88033a HTTP/1.1" 200 None 2026-03-24 08:57:14,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:14,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:14,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:14,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2519f4a2c5046eb88033a HTTP/1.1" 200 None 2026-03-24 08:57:14,779 - lava_callback - INFO - Completed processing callback for node 69c2519f4a2c5046eb88033a INFO: 172.20.0.1:35462 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 08:57:15,430 - lava_callback - INFO - Received callback for node 69c2516a4a2c5046eb880278 INFO: 172.20.0.1:35472 - "POST /node/69c2516a4a2c5046eb880278 HTTP/1.0" 202 Accepted 2026-03-24 08:57:15,436 - lava_callback - INFO - Starting async job submit for node 69c2516a4a2c5046eb880278 2026-03-24 08:57:15,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:15,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2516a4a2c5046eb880278 HTTP/1.1" 200 None 2026-03-24 08:57:15,482 - lava_callback - DEBUG - Processing node 69c2516a4a2c5046eb880278: baseline-nfs-arm64 2026-03-24 08:57:15,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:16,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:16,001 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516a4a2c5046eb880278/log.txt.gz 2026-03-24 08:57:16,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:35484 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:57:16,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:16,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516a4a2c5046eb880278/lava-logs.yaml 2026-03-24 08:57:16,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:16,517 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:16,519 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516a4a2c5046eb880278/lava_callback.json.gz INFO: 172.20.0.1:35498 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:57:16,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:16,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2516a4a2c5046eb880278 HTTP/1.1" 200 None 2026-03-24 08:57:16,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:16,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:16,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:16,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2516a4a2c5046eb880278 HTTP/1.1" 200 None 2026-03-24 08:57:16,616 - lava_callback - INFO - Completed processing callback for node 69c2516a4a2c5046eb880278 2026-03-24 08:57:24,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:24,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-24 08:57:25,723 - lava_callback - INFO - Received callback for node 69c251714a2c5046eb88028b INFO: 172.20.0.1:51050 - "POST /node/69c251714a2c5046eb88028b HTTP/1.0" 202 Accepted 2026-03-24 08:57:25,730 - lava_callback - INFO - Starting async job submit for node 69c251714a2c5046eb88028b 2026-03-24 08:57:25,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:25,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251714a2c5046eb88028b HTTP/1.1" 200 None 2026-03-24 08:57:25,768 - lava_callback - DEBUG - Processing node 69c251714a2c5046eb88028b: baseline-nfs-arm64 2026-03-24 08:57:26,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:26,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:26,181 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251714a2c5046eb88028b/log.txt.gz 2026-03-24 08:57:26,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:51064 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:57:26,546 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:26,548 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251714a2c5046eb88028b/lava-logs.yaml 2026-03-24 08:57:26,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:26,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:26,698 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251714a2c5046eb88028b/lava_callback.json.gz 2026-03-24 08:57:26,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:26,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251714a2c5046eb88028b HTTP/1.1" 200 None 2026-03-24 08:57:26,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:26,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:26,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:26,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251714a2c5046eb88028b HTTP/1.1" 200 None 2026-03-24 08:57:26,778 - lava_callback - INFO - Completed processing callback for node 69c251714a2c5046eb88028b 2026-03-24 08:57:35,749 - lava_callback - INFO - Received callback for node 69bfaef57544ab5cd4a7e50e INFO: 172.20.0.1:39760 - "POST /node/69bfaef57544ab5cd4a7e50e HTTP/1.0" 202 Accepted 2026-03-24 08:57:35,755 - lava_callback - INFO - Starting async job submit for node 69bfaef57544ab5cd4a7e50e 2026-03-24 08:57:35,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:35,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfaef57544ab5cd4a7e50e HTTP/1.1" 200 None 2026-03-24 08:57:35,770 - lava_callback - DEBUG - Processing node 69bfaef57544ab5cd4a7e50e: baseline-arm64-clabbe 2026-03-24 08:57:35,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfaef57544ab5cd4a7e50e 2026-03-24 08:57:35,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:35,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfaef57544ab5cd4a7e50e HTTP/1.1" 200 None 2026-03-24 08:57:35,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:35,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 08:57:35,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:35,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfaef57544ab5cd4a7e50e HTTP/1.1" 200 None 2026-03-24 08:57:35,871 - lava_callback - ERROR - Error processing callback for node 69bfaef57544ab5cd4a7e50e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 08:57:36,260 - lava_callback - INFO - Received callback for node 69c251984a2c5046eb880327 INFO: 172.20.0.1:39768 - "POST /node/69c251984a2c5046eb880327 HTTP/1.0" 202 Accepted 2026-03-24 08:57:36,267 - lava_callback - INFO - Starting async job submit for node 69c251984a2c5046eb880327 2026-03-24 08:57:36,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:36,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251984a2c5046eb880327 HTTP/1.1" 200 None 2026-03-24 08:57:36,322 - lava_callback - DEBUG - Processing node 69c251984a2c5046eb880327: baseline-arm64-clabbe INFO: 172.20.0.1:39782 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:57:36,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:36,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:36,672 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c251984a2c5046eb880327/log.txt.gz 2026-03-24 08:57:36,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:37,011 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:37,013 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c251984a2c5046eb880327/lava-logs.yaml 2026-03-24 08:57:37,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:37,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:37,146 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c251984a2c5046eb880327/lava_callback.json.gz 2026-03-24 08:57:37,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:37,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251984a2c5046eb880327 HTTP/1.1" 200 None 2026-03-24 08:57:37,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:37,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:37,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:37,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251984a2c5046eb880327 HTTP/1.1" 200 None 2026-03-24 08:57:37,252 - lava_callback - INFO - Completed processing callback for node 69c251984a2c5046eb880327 2026-03-24 08:57:39,599 - lava_callback - INFO - Received callback for node 69c251884a2c5046eb8802e9 INFO: 172.20.0.1:39786 - "POST /node/69c251884a2c5046eb8802e9 HTTP/1.0" 202 Accepted 2026-03-24 08:57:39,606 - lava_callback - INFO - Starting async job submit for node 69c251884a2c5046eb8802e9 2026-03-24 08:57:39,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:39,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251884a2c5046eb8802e9 HTTP/1.1" 200 None 2026-03-24 08:57:39,649 - lava_callback - DEBUG - Processing node 69c251884a2c5046eb8802e9: baseline-arm64-broonie 2026-03-24 08:57:39,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:40,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:40,059 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251884a2c5046eb8802e9/log.txt.gz 2026-03-24 08:57:40,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:40,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:40,414 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251884a2c5046eb8802e9/lava-logs.yaml 2026-03-24 08:57:40,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:40,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:40,552 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251884a2c5046eb8802e9/lava_callback.json.gz 2026-03-24 08:57:40,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:40,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251884a2c5046eb8802e9 HTTP/1.1" 200 None 2026-03-24 08:57:40,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:40,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:40,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:40,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251884a2c5046eb8802e9 HTTP/1.1" 200 None 2026-03-24 08:57:40,643 - lava_callback - INFO - Completed processing callback for node 69c251884a2c5046eb8802e9 2026-03-24 08:57:41,023 - lava_callback - INFO - Received callback for node 69c2518a4a2c5046eb8802ee INFO: 172.20.0.1:33790 - "POST /node/69c2518a4a2c5046eb8802ee HTTP/1.0" 202 Accepted 2026-03-24 08:57:41,030 - lava_callback - INFO - Starting async job submit for node 69c2518a4a2c5046eb8802ee 2026-03-24 08:57:41,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:41,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2518a4a2c5046eb8802ee HTTP/1.1" 200 None 2026-03-24 08:57:41,088 - lava_callback - DEBUG - Processing node 69c2518a4a2c5046eb8802ee: baseline-arm64-broonie 2026-03-24 08:57:41,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:41,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:41,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2518a4a2c5046eb8802ee/log.txt.gz 2026-03-24 08:57:41,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:42,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:42,048 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2518a4a2c5046eb8802ee/lava-logs.yaml 2026-03-24 08:57:42,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:42,182 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:42,183 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2518a4a2c5046eb8802ee/lava_callback.json.gz 2026-03-24 08:57:42,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:42,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2518a4a2c5046eb8802ee HTTP/1.1" 200 None 2026-03-24 08:57:42,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:42,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:42,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:42,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2518a4a2c5046eb8802ee HTTP/1.1" 200 None 2026-03-24 08:57:42,285 - lava_callback - INFO - Completed processing callback for node 69c2518a4a2c5046eb8802ee 2026-03-24 08:57:42,483 - lava_callback - INFO - Received callback for node 69c251924a2c5046eb8802fa INFO: 172.20.0.1:33800 - "POST /node/69c251924a2c5046eb8802fa HTTP/1.0" 202 Accepted 2026-03-24 08:57:42,489 - lava_callback - INFO - Starting async job submit for node 69c251924a2c5046eb8802fa 2026-03-24 08:57:42,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:42,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251924a2c5046eb8802fa HTTP/1.1" 200 None 2026-03-24 08:57:42,532 - lava_callback - DEBUG - Processing node 69c251924a2c5046eb8802fa: baseline-arm64-broonie 2026-03-24 08:57:42,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:42,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:42,920 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251924a2c5046eb8802fa/log.txt.gz 2026-03-24 08:57:43,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:43,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:43,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251924a2c5046eb8802fa/lava-logs.yaml 2026-03-24 08:57:43,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:43,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:43,464 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251924a2c5046eb8802fa/lava_callback.json.gz 2026-03-24 08:57:43,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:43,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251924a2c5046eb8802fa HTTP/1.1" 200 None 2026-03-24 08:57:43,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:43,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:43,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:43,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251924a2c5046eb8802fa HTTP/1.1" 200 None 2026-03-24 08:57:43,551 - lava_callback - INFO - Completed processing callback for node 69c251924a2c5046eb8802fa INFO: 172.20.0.1:33802 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:57:48,132 - lava_callback - INFO - Received callback for node 69c251934a2c5046eb8802ff INFO: 172.20.0.1:33812 - "POST /node/69c251934a2c5046eb8802ff HTTP/1.0" 202 Accepted 2026-03-24 08:57:48,138 - lava_callback - INFO - Starting async job submit for node 69c251934a2c5046eb8802ff 2026-03-24 08:57:48,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:48,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251934a2c5046eb8802ff HTTP/1.1" 200 None 2026-03-24 08:57:48,175 - lava_callback - DEBUG - Processing node 69c251934a2c5046eb8802ff: baseline-arm64-broonie 2026-03-24 08:57:48,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:48,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:48,562 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251934a2c5046eb8802ff/log.txt.gz 2026-03-24 08:57:48,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:48,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:48,893 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251934a2c5046eb8802ff/lava-logs.yaml 2026-03-24 08:57:48,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:57:49,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:57:49,030 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251934a2c5046eb8802ff/lava_callback.json.gz 2026-03-24 08:57:49,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:49,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251934a2c5046eb8802ff HTTP/1.1" 200 None 2026-03-24 08:57:49,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:49,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:57:49,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:49,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251934a2c5046eb8802ff HTTP/1.1" 200 None 2026-03-24 08:57:49,120 - lava_callback - INFO - Completed processing callback for node 69c251934a2c5046eb8802ff 2026-03-24 08:57:54,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:57:54,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:60784 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:58:00,905 - lava_callback - INFO - Received callback for node 69c24ef34a2c5046eb880074 INFO: 172.20.0.1:45628 - "POST /node/69c24ef34a2c5046eb880074 HTTP/1.0" 202 Accepted 2026-03-24 08:58:00,912 - lava_callback - INFO - Starting async job submit for node 69c24ef34a2c5046eb880074 2026-03-24 08:58:00,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:00,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef34a2c5046eb880074 HTTP/1.1" 200 None 2026-03-24 08:58:00,955 - lava_callback - DEBUG - Processing node 69c24ef34a2c5046eb880074: ltp-pty 2026-03-24 08:58:01,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:58:01,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:58:01,458 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c24ef34a2c5046eb880074/log.txt.gz 2026-03-24 08:58:01,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:58:01,880 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:58:01,882 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c24ef34a2c5046eb880074/lava-logs.yaml 2026-03-24 08:58:01,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:58:02,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:58:02,069 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c24ef34a2c5046eb880074/lava_callback.json.gz 2026-03-24 08:58:02,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:02,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef34a2c5046eb880074 HTTP/1.1" 200 None 2026-03-24 08:58:02,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:02,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 08:58:02,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:02,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ef34a2c5046eb880074 HTTP/1.1" 200 None 2026-03-24 08:58:02,157 - lava_callback - INFO - Completed processing callback for node 69c24ef34a2c5046eb880074 INFO: 172.20.0.1:45642 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:58:07,708 - lava_callback - INFO - Received callback for node 69c251854a2c5046eb8802e3 INFO: 172.20.0.1:45650 - "POST /node/69c251854a2c5046eb8802e3 HTTP/1.0" 202 Accepted 2026-03-24 08:58:07,714 - lava_callback - INFO - Starting async job submit for node 69c251854a2c5046eb8802e3 2026-03-24 08:58:07,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:07,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251854a2c5046eb8802e3 HTTP/1.1" 200 None 2026-03-24 08:58:07,752 - lava_callback - DEBUG - Processing node 69c251854a2c5046eb8802e3: baseline-arm64-broonie 2026-03-24 08:58:07,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:58:08,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:58:08,142 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251854a2c5046eb8802e3/log.txt.gz 2026-03-24 08:58:08,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:58:08,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:58:08,499 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251854a2c5046eb8802e3/lava-logs.yaml 2026-03-24 08:58:08,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:58:08,714 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:58:08,716 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251854a2c5046eb8802e3/lava_callback.json.gz 2026-03-24 08:58:08,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:08,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251854a2c5046eb8802e3 HTTP/1.1" 200 None 2026-03-24 08:58:08,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:08,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:58:08,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:08,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251854a2c5046eb8802e3 HTTP/1.1" 200 None 2026-03-24 08:58:08,796 - lava_callback - INFO - Completed processing callback for node 69c251854a2c5046eb8802e3 INFO: 172.20.0.1:50712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50720 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:58:24,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:24,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56232 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:58:36,048 - lava_callback - INFO - Received callback for node 69c251584a2c5046eb88022e INFO: 172.20.0.1:38898 - "POST /node/69c251584a2c5046eb88022e HTTP/1.0" 202 Accepted 2026-03-24 08:58:36,058 - lava_callback - INFO - Starting async job submit for node 69c251584a2c5046eb88022e 2026-03-24 08:58:36,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:36,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251584a2c5046eb88022e HTTP/1.1" 200 None 2026-03-24 08:58:36,120 - lava_callback - DEBUG - Processing node 69c251584a2c5046eb88022e: baseline-arm64 INFO: 172.20.0.1:38900 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:58:36,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:58:37,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:58:37,031 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251584a2c5046eb88022e/log.txt.gz 2026-03-24 08:58:37,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:58:37,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:58:37,702 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251584a2c5046eb88022e/lava-logs.yaml 2026-03-24 08:58:37,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:58:37,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:58:37,895 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251584a2c5046eb88022e/lava_callback.json.gz 2026-03-24 08:58:37,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:37,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251584a2c5046eb88022e HTTP/1.1" 200 None 2026-03-24 08:58:37,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:37,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:58:37,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:37,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251584a2c5046eb88022e HTTP/1.1" 200 None 2026-03-24 08:58:37,994 - lava_callback - INFO - Completed processing callback for node 69c251584a2c5046eb88022e INFO: 172.20.0.1:44812 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:58:54,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:58:54,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58330 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:59:29,508 - lava_callback - INFO - Received callback for node 69c2518c4a2c5046eb8802f1 INFO: 172.20.0.1:58334 - "POST /node/69c2518c4a2c5046eb8802f1 HTTP/1.0" 202 Accepted 2026-03-24 08:59:29,519 - lava_callback - INFO - Starting async job submit for node 69c2518c4a2c5046eb8802f1 2026-03-24 08:59:29,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:59:29,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2518c4a2c5046eb8802f1 HTTP/1.1" 200 None 2026-03-24 08:59:29,591 - lava_callback - DEBUG - Processing node 69c2518c4a2c5046eb8802f1: baseline-arm64-broonie 2026-03-24 08:59:30,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:59:30,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:59:30,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2518c4a2c5046eb8802f1/log.txt.gz 2026-03-24 08:59:30,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:59:30,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:59:30,722 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2518c4a2c5046eb8802f1/lava-logs.yaml 2026-03-24 08:59:30,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:59:31,124 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:59:31,126 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c2518c4a2c5046eb8802f1/lava_callback.json.gz 2026-03-24 08:59:31,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:59:31,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2518c4a2c5046eb8802f1 HTTP/1.1" 200 None 2026-03-24 08:59:31,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:59:31,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:59:31,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:59:31,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2518c4a2c5046eb8802f1 HTTP/1.1" 200 None 2026-03-24 08:59:31,262 - lava_callback - INFO - Completed processing callback for node 69c2518c4a2c5046eb8802f1 INFO: 172.20.0.1:46516 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:59:41,516 - lava_callback - INFO - Received callback for node 69c251694a2c5046eb880271 INFO: 172.20.0.1:43626 - "POST /node/69c251694a2c5046eb880271 HTTP/1.0" 202 Accepted 2026-03-24 08:59:41,523 - lava_callback - INFO - Starting async job submit for node 69c251694a2c5046eb880271 2026-03-24 08:59:41,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:59:41,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251694a2c5046eb880271 HTTP/1.1" 200 None 2026-03-24 08:59:41,566 - lava_callback - DEBUG - Processing node 69c251694a2c5046eb880271: baseline-nfs-arm64 2026-03-24 08:59:42,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:59:42,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:59:42,436 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251694a2c5046eb880271/log.txt.gz 2026-03-24 08:59:42,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:59:43,106 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:59:43,108 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251694a2c5046eb880271/lava-logs.yaml 2026-03-24 08:59:43,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 08:59:43,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 08:59:43,448 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251694a2c5046eb880271/lava_callback.json.gz 2026-03-24 08:59:43,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:59:43,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251694a2c5046eb880271 HTTP/1.1" 200 None 2026-03-24 08:59:43,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:59:43,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 08:59:43,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:59:43,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251694a2c5046eb880271 HTTP/1.1" 200 None 2026-03-24 08:59:43,562 - lava_callback - INFO - Completed processing callback for node 69c251694a2c5046eb880271 INFO: 172.20.0.1:43628 - "GET / HTTP/1.0" 200 OK 2026-03-24 08:59:54,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 08:59:54,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43746 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:00:08,257 - lava_callback - INFO - Received callback for node 69c24ef44a2c5046eb880079 INFO: 172.20.0.1:43762 - "POST /node/69c24ef44a2c5046eb880079 HTTP/1.0" 202 Accepted 2026-03-24 09:00:08,263 - lava_callback - INFO - Starting async job submit for node 69c24ef44a2c5046eb880079 2026-03-24 09:00:08,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:08,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef44a2c5046eb880079 HTTP/1.1" 200 None 2026-03-24 09:00:08,305 - lava_callback - DEBUG - Processing node 69c24ef44a2c5046eb880079: ltp-sched 2026-03-24 09:00:08,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:08,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:08,798 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c24ef44a2c5046eb880079/log.txt.gz 2026-03-24 09:00:09,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:09,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:09,201 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c24ef44a2c5046eb880079/lava-logs.yaml 2026-03-24 09:00:09,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:09,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:09,377 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c24ef44a2c5046eb880079/lava_callback.json.gz 2026-03-24 09:00:09,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:09,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef44a2c5046eb880079 HTTP/1.1" 200 None 2026-03-24 09:00:09,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:09,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 09:00:09,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:09,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ef44a2c5046eb880079 HTTP/1.1" 200 None 2026-03-24 09:00:09,461 - lava_callback - INFO - Completed processing callback for node 69c24ef44a2c5046eb880079 INFO: 172.20.0.1:35700 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 09:00:15,530 - lava_callback - INFO - Received callback for node 69c24ea54a2c5046eb87ff94 INFO: 172.20.0.1:35708 - "POST /node/69c24ea54a2c5046eb87ff94 HTTP/1.0" 202 Accepted 2026-03-24 09:00:15,537 - lava_callback - INFO - Starting async job submit for node 69c24ea54a2c5046eb87ff94 2026-03-24 09:00:15,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:15,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea54a2c5046eb87ff94 HTTP/1.1" 200 None 2026-03-24 09:00:15,615 - lava_callback - DEBUG - Processing node 69c24ea54a2c5046eb87ff94: baseline-arm 2026-03-24 09:00:16,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:35712 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:00:16,389 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:16,391 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea54a2c5046eb87ff94/log.txt.gz INFO: 172.20.0.1:35720 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:00:16,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:16,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:16,953 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea54a2c5046eb87ff94/lava-logs.yaml 2026-03-24 09:00:16,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:17,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:17,132 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c24ea54a2c5046eb87ff94/lava_callback.json.gz DEBUG: Setup failed for 69c24ea54a2c5046eb87ff94. Transitting job node result: incomplete -> fail 2026-03-24 09:00:17,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:17,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ea54a2c5046eb87ff94 HTTP/1.1" 200 None 2026-03-24 09:00:17,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:17,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 09:00:17,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:17,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ea54a2c5046eb87ff94 HTTP/1.1" 200 None 2026-03-24 09:00:17,425 - lava_callback - INFO - Completed processing callback for node 69c24ea54a2c5046eb87ff94 2026-03-24 09:00:19,970 - lava_callback - INFO - Received callback for node 69c24ef54a2c5046eb88007c INFO: 172.20.0.1:44700 - "POST /node/69c24ef54a2c5046eb88007c HTTP/1.0" 202 Accepted 2026-03-24 09:00:19,981 - lava_callback - INFO - Starting async job submit for node 69c24ef54a2c5046eb88007c 2026-03-24 09:00:20,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:20,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef54a2c5046eb88007c HTTP/1.1" 200 None 2026-03-24 09:00:20,027 - lava_callback - DEBUG - Processing node 69c24ef54a2c5046eb88007c: ltp-smoke 2026-03-24 09:00:20,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:20,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:20,502 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c24ef54a2c5046eb88007c/log.txt.gz 2026-03-24 09:00:20,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:20,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:20,907 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c24ef54a2c5046eb88007c/lava-logs.yaml 2026-03-24 09:00:20,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:21,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:21,075 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c24ef54a2c5046eb88007c/lava_callback.json.gz 2026-03-24 09:00:21,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:21,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef54a2c5046eb88007c HTTP/1.1" 200 None 2026-03-24 09:00:21,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:21,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 09:00:21,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:21,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ef54a2c5046eb88007c HTTP/1.1" 200 None 2026-03-24 09:00:21,184 - lava_callback - INFO - Completed processing callback for node 69c24ef54a2c5046eb88007c 2026-03-24 09:00:24,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:24,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59720 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:00:40,505 - lava_callback - INFO - Received callback for node 69c24ef84a2c5046eb880082 INFO: 172.20.0.1:35006 - "POST /node/69c24ef84a2c5046eb880082 HTTP/1.0" 202 Accepted 2026-03-24 09:00:40,512 - lava_callback - INFO - Starting async job submit for node 69c24ef84a2c5046eb880082 2026-03-24 09:00:40,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:40,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef84a2c5046eb880082 HTTP/1.1" 200 None 2026-03-24 09:00:40,560 - lava_callback - DEBUG - Processing node 69c24ef84a2c5046eb880082: ltp-syscalls-ipc 2026-03-24 09:00:40,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:41,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:41,028 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c24ef84a2c5046eb880082/log.txt.gz 2026-03-24 09:00:41,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:41,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:41,454 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c24ef84a2c5046eb880082/lava-logs.yaml 2026-03-24 09:00:41,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:00:41,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:00:41,614 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c24ef84a2c5046eb880082/lava_callback.json.gz 2026-03-24 09:00:41,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:41,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ef84a2c5046eb880082 HTTP/1.1" 200 None 2026-03-24 09:00:41,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:41,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 09:00:41,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:41,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ef84a2c5046eb880082 HTTP/1.1" 200 None 2026-03-24 09:00:41,726 - lava_callback - INFO - Completed processing callback for node 69c24ef84a2c5046eb880082 INFO: 172.20.0.1:35008 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:00:54,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:00:55,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49774 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:02:31,552 - lava_callback - INFO - Received callback for node 69c251664a2c5046eb88026b INFO: 172.20.0.1:55288 - "POST /node/69c251664a2c5046eb88026b HTTP/1.0" 202 Accepted 2026-03-24 09:02:31,560 - lava_callback - INFO - Starting async job submit for node 69c251664a2c5046eb88026b 2026-03-24 09:02:31,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:02:31,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251664a2c5046eb88026b HTTP/1.1" 200 None 2026-03-24 09:02:31,613 - lava_callback - DEBUG - Processing node 69c251664a2c5046eb88026b: baseline-arm64 2026-03-24 09:02:32,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:02:32,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:02:32,339 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251664a2c5046eb88026b/log.txt.gz 2026-03-24 09:02:32,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:02:32,775 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:02:32,777 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251664a2c5046eb88026b/lava-logs.yaml 2026-03-24 09:02:32,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:02:32,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:02:32,957 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251664a2c5046eb88026b/lava_callback.json.gz DEBUG: Setup failed for 69c251664a2c5046eb88026b. Transitting job node result: incomplete -> fail 2026-03-24 09:02:33,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:02:33,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251664a2c5046eb88026b HTTP/1.1" 200 None 2026-03-24 09:02:33,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:02:33,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:02:33,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:02:33,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251664a2c5046eb88026b HTTP/1.1" 200 None 2026-03-24 09:02:33,184 - lava_callback - INFO - Completed processing callback for node 69c251664a2c5046eb88026b INFO: 172.20.0.1:55300 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:02:39,836 - lava_callback - INFO - Received callback for node 69c24efa4a2c5046eb880085 INFO: 172.20.0.1:58064 - "POST /node/69c24efa4a2c5046eb880085 HTTP/1.0" 202 Accepted 2026-03-24 09:02:39,843 - lava_callback - INFO - Starting async job submit for node 69c24efa4a2c5046eb880085 2026-03-24 09:02:39,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:02:39,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24efa4a2c5046eb880085 HTTP/1.1" 200 None 2026-03-24 09:02:39,884 - lava_callback - DEBUG - Processing node 69c24efa4a2c5046eb880085: ltp-timers 2026-03-24 09:02:40,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:02:40,413 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:02:40,414 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c24efa4a2c5046eb880085/log.txt.gz 2026-03-24 09:02:40,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:02:40,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:02:40,848 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c24efa4a2c5046eb880085/lava-logs.yaml 2026-03-24 09:02:40,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:02:41,012 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:02:41,014 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c24efa4a2c5046eb880085/lava_callback.json.gz 2026-03-24 09:02:41,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:02:41,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24efa4a2c5046eb880085 HTTP/1.1" 200 None 2026-03-24 09:02:41,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:02:41,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 09:02:41,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:02:41,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24efa4a2c5046eb880085 HTTP/1.1" 200 None 2026-03-24 09:02:41,120 - lava_callback - INFO - Completed processing callback for node 69c24efa4a2c5046eb880085 INFO: 172.20.0.1:58072 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:02:55,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:02:55,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57596 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:03:17,136 - lava_callback - INFO - Received callback for node 69c2516e4a2c5046eb880281 INFO: 172.20.0.1:57610 - "POST /node/69c2516e4a2c5046eb880281 HTTP/1.0" 202 Accepted 2026-03-24 09:03:17,144 - lava_callback - INFO - Starting async job submit for node 69c2516e4a2c5046eb880281 2026-03-24 09:03:17,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:03:17,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2516e4a2c5046eb880281 HTTP/1.1" 200 None 2026-03-24 09:03:17,197 - lava_callback - DEBUG - Processing node 69c2516e4a2c5046eb880281: baseline-nfs-arm64 2026-03-24 09:03:18,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:03:18,168 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:03:18,169 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516e4a2c5046eb880281/log.txt.gz 2026-03-24 09:03:18,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:03:18,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:03:18,880 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516e4a2c5046eb880281/lava-logs.yaml 2026-03-24 09:03:18,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:03:19,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:03:19,083 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516e4a2c5046eb880281/lava_callback.json.gz 2026-03-24 09:03:19,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:03:19,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2516e4a2c5046eb880281 HTTP/1.1" 200 None 2026-03-24 09:03:19,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:03:19,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:03:19,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:03:19,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2516e4a2c5046eb880281 HTTP/1.1" 200 None 2026-03-24 09:03:19,181 - lava_callback - INFO - Completed processing callback for node 69c2516e4a2c5046eb880281 2026-03-24 09:03:25,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:03:25,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57386 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:03:27,116 - lava_callback - INFO - Received callback for node 69c24ed14a2c5046eb88002d INFO: 172.20.0.1:57388 - "POST /node/69c24ed14a2c5046eb88002d HTTP/1.0" 202 Accepted 2026-03-24 09:03:27,161 - lava_callback - INFO - Starting async job submit for node 69c24ed14a2c5046eb88002d 2026-03-24 09:03:27,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:03:27,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ed14a2c5046eb88002d HTTP/1.1" 200 None 2026-03-24 09:03:27,201 - lava_callback - DEBUG - Processing node 69c24ed14a2c5046eb88002d: baseline-arm-pengutronix 2026-03-24 09:03:27,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:03:27,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:03:27,423 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ed14a2c5046eb88002d/log.txt.gz 2026-03-24 09:03:27,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:03:27,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:03:27,647 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ed14a2c5046eb88002d/lava-logs.yaml 2026-03-24 09:03:27,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:03:27,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:03:27,762 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c24ed14a2c5046eb88002d/lava_callback.json.gz DEBUG: Setup failed for 69c24ed14a2c5046eb88002d. Transitting job node result: incomplete -> fail 2026-03-24 09:03:27,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:03:27,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24ed14a2c5046eb88002d HTTP/1.1" 200 None 2026-03-24 09:03:27,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:03:27,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 09:03:27,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:03:27,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24ed14a2c5046eb88002d HTTP/1.1" 200 None 2026-03-24 09:03:27,953 - lava_callback - INFO - Completed processing callback for node 69c24ed14a2c5046eb88002d INFO: 172.20.0.1:55868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52794 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:03:55,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:03:55,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33452 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:04:08,301 - lava_callback - INFO - Received callback for node 69c24efb4a2c5046eb880088 INFO: 172.20.0.1:33462 - "POST /node/69c24efb4a2c5046eb880088 HTTP/1.0" 202 Accepted 2026-03-24 09:04:08,307 - lava_callback - INFO - Starting async job submit for node 69c24efb4a2c5046eb880088 2026-03-24 09:04:08,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:04:08,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24efb4a2c5046eb880088 HTTP/1.1" 200 None 2026-03-24 09:04:08,347 - lava_callback - DEBUG - Processing node 69c24efb4a2c5046eb880088: ltp-watchqueue 2026-03-24 09:04:08,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:04:08,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:04:08,840 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c24efb4a2c5046eb880088/log.txt.gz 2026-03-24 09:04:09,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:04:09,290 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:04:09,292 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c24efb4a2c5046eb880088/lava-logs.yaml 2026-03-24 09:04:09,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:04:09,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:04:09,477 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c24efb4a2c5046eb880088/lava_callback.json.gz 2026-03-24 09:04:09,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:04:09,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c24efb4a2c5046eb880088 HTTP/1.1" 200 None 2026-03-24 09:04:09,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:04:09,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 09:04:09,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:04:09,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c24efb4a2c5046eb880088 HTTP/1.1" 200 None 2026-03-24 09:04:09,561 - lava_callback - INFO - Completed processing callback for node 69c24efb4a2c5046eb880088 INFO: 172.20.0.1:35916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35932 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:04:25,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:04:25,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46748 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:05:59,777 - lava_callback - INFO - Received callback for node 69c17ecb2b2b51e9ad97a973 INFO: 172.20.0.1:34166 - "POST /node/69c17ecb2b2b51e9ad97a973 HTTP/1.0" 202 Accepted 2026-03-24 09:05:59,783 - lava_callback - INFO - Starting async job submit for node 69c17ecb2b2b51e9ad97a973 2026-03-24 09:05:59,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:05:59,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c17ecb2b2b51e9ad97a973 HTTP/1.1" 200 None 2026-03-24 09:05:59,820 - lava_callback - DEBUG - Processing node 69c17ecb2b2b51e9ad97a973: baseline-arm64-broonie 2026-03-24 09:06:00,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:06:00,238 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:06:00,240 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c17ecb2b2b51e9ad97a973/log.txt.gz 2026-03-24 09:06:00,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:06:00,637 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:06:00,639 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c17ecb2b2b51e9ad97a973/lava-logs.yaml 2026-03-24 09:06:00,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:06:00,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:06:00,822 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c17ecb2b2b51e9ad97a973/lava_callback.json.gz 2026-03-24 09:06:00,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:00,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c17ecb2b2b51e9ad97a973 HTTP/1.1" 200 None 2026-03-24 09:06:00,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:00,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-24 09:06:00,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:00,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c17ecb2b2b51e9ad97a973 HTTP/1.1" 200 None 2026-03-24 09:06:00,909 - lava_callback - INFO - Completed processing callback for node 69c17ecb2b2b51e9ad97a973 INFO: 172.20.0.1:34180 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:06:15,060 - lava_callback - INFO - Received callback for node 69c251614a2c5046eb880258 INFO: 172.20.0.1:33580 - "POST /node/69c251614a2c5046eb880258 HTTP/1.0" 202 Accepted 2026-03-24 09:06:15,066 - lava_callback - INFO - Starting async job submit for node 69c251614a2c5046eb880258 2026-03-24 09:06:15,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:15,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251614a2c5046eb880258 HTTP/1.1" 200 None 2026-03-24 09:06:15,105 - lava_callback - DEBUG - Processing node 69c251614a2c5046eb880258: baseline-arm64 INFO: 172.20.0.1:33594 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 09:06:15,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:06:15,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:06:15,753 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251614a2c5046eb880258/log.txt.gz 2026-03-24 09:06:16,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:06:16,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:06:16,331 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251614a2c5046eb880258/lava-logs.yaml 2026-03-24 09:06:16,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33600 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:06:16,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 INFO: 172.20.0.1:33608 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:06:16,517 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c251614a2c5046eb880258/lava_callback.json.gz 2026-03-24 09:06:16,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:16,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251614a2c5046eb880258 HTTP/1.1" 200 None 2026-03-24 09:06:16,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:16,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:06:16,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:16,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251614a2c5046eb880258 HTTP/1.1" 200 None 2026-03-24 09:06:16,668 - lava_callback - INFO - Completed processing callback for node 69c251614a2c5046eb880258 2026-03-24 09:06:20,967 - lava_callback - INFO - Received callback for node 69c251724a2c5046eb88028e INFO: 172.20.0.1:42098 - "POST /node/69c251724a2c5046eb88028e HTTP/1.0" 202 Accepted 2026-03-24 09:06:20,973 - lava_callback - INFO - Starting async job submit for node 69c251724a2c5046eb88028e 2026-03-24 09:06:20,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:21,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251724a2c5046eb88028e HTTP/1.1" 200 None 2026-03-24 09:06:21,016 - lava_callback - DEBUG - Processing node 69c251724a2c5046eb88028e: baseline-nfs-arm64 2026-03-24 09:06:21,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:06:21,662 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:06:21,664 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251724a2c5046eb88028e/log.txt.gz 2026-03-24 09:06:21,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:06:22,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:06:22,150 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251724a2c5046eb88028e/lava-logs.yaml 2026-03-24 09:06:22,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:06:22,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:06:22,353 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c251724a2c5046eb88028e/lava_callback.json.gz 2026-03-24 09:06:22,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:22,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251724a2c5046eb88028e HTTP/1.1" 200 None 2026-03-24 09:06:22,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:22,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:06:22,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:22,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251724a2c5046eb88028e HTTP/1.1" 200 None 2026-03-24 09:06:22,510 - lava_callback - INFO - Completed processing callback for node 69c251724a2c5046eb88028e 2026-03-24 09:06:25,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:06:25,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32922 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:07:59,873 - lava_callback - INFO - Received callback for node 69c1e3bc0efaf21d9310609d INFO: 172.20.0.1:49228 - "POST /node/69c1e3bc0efaf21d9310609d HTTP/1.0" 202 Accepted 2026-03-24 09:07:59,879 - lava_callback - INFO - Starting async job submit for node 69c1e3bc0efaf21d9310609d 2026-03-24 09:07:59,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:07:59,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3bc0efaf21d9310609d HTTP/1.1" 200 None 2026-03-24 09:07:59,948 - lava_callback - DEBUG - Processing node 69c1e3bc0efaf21d9310609d: baseline-arm64-broonie 2026-03-24 09:08:00,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:08:00,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:08:00,335 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3bc0efaf21d9310609d/log.txt.gz 2026-03-24 09:08:00,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:08:00,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:08:00,686 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3bc0efaf21d9310609d/lava-logs.yaml 2026-03-24 09:08:00,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:08:00,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:08:00,832 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c1e3bc0efaf21d9310609d/lava_callback.json.gz 2026-03-24 09:08:00,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:08:00,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1e3bc0efaf21d9310609d HTTP/1.1" 200 None 2026-03-24 09:08:00,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:08:00,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 09:08:00,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:08:00,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c1e3bc0efaf21d9310609d HTTP/1.1" 200 None 2026-03-24 09:08:00,970 - lava_callback - INFO - Completed processing callback for node 69c1e3bc0efaf21d9310609d INFO: 172.20.0.1:49232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51294 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:08:25,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:08:25,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:09:59,944 - lava_callback - INFO - Received callback for node 69c23c850efaf21d93107f5f INFO: 172.20.0.1:35618 - "POST /node/69c23c850efaf21d93107f5f HTTP/1.0" 202 Accepted 2026-03-24 09:09:59,950 - lava_callback - INFO - Starting async job submit for node 69c23c850efaf21d93107f5f 2026-03-24 09:09:59,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:09:59,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23c850efaf21d93107f5f HTTP/1.1" 200 None 2026-03-24 09:09:59,988 - lava_callback - DEBUG - Processing node 69c23c850efaf21d93107f5f: baseline-arm64-broonie 2026-03-24 09:10:00,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:10:00,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:10:00,404 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c23c850efaf21d93107f5f/log.txt.gz 2026-03-24 09:10:00,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:10:00,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:10:00,761 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c23c850efaf21d93107f5f/lava-logs.yaml 2026-03-24 09:10:00,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:10:00,941 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:10:00,943 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c23c850efaf21d93107f5f/lava_callback.json.gz 2026-03-24 09:10:00,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:10:00,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c23c850efaf21d93107f5f HTTP/1.1" 200 None 2026-03-24 09:10:00,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:10:00,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-24 09:10:00,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:10:01,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c23c850efaf21d93107f5f HTTP/1.1" 200 None 2026-03-24 09:10:01,046 - lava_callback - INFO - Completed processing callback for node 69c23c850efaf21d93107f5f INFO: 172.20.0.1:35620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37854 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:10:25,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:10:25,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60010 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:12:00,074 - lava_callback - INFO - Received callback for node 69c251904a2c5046eb8802f7 INFO: 172.20.0.1:58544 - "POST /node/69c251904a2c5046eb8802f7 HTTP/1.0" 202 Accepted 2026-03-24 09:12:00,080 - lava_callback - INFO - Starting async job submit for node 69c251904a2c5046eb8802f7 2026-03-24 09:12:00,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:12:00,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251904a2c5046eb8802f7 HTTP/1.1" 200 None 2026-03-24 09:12:00,129 - lava_callback - DEBUG - Processing node 69c251904a2c5046eb8802f7: baseline-arm64-broonie 2026-03-24 09:12:00,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:12:00,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:12:00,545 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251904a2c5046eb8802f7/log.txt.gz 2026-03-24 09:12:00,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:12:00,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:12:00,926 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251904a2c5046eb8802f7/lava-logs.yaml 2026-03-24 09:12:00,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:12:01,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:12:01,101 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251904a2c5046eb8802f7/lava_callback.json.gz 2026-03-24 09:12:01,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:12:01,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251904a2c5046eb8802f7 HTTP/1.1" 200 None 2026-03-24 09:12:01,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:12:01,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:12:01,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:12:01,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251904a2c5046eb8802f7 HTTP/1.1" 200 None 2026-03-24 09:12:01,183 - lava_callback - INFO - Completed processing callback for node 69c251904a2c5046eb8802f7 INFO: 172.20.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47342 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:12:25,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:12:25,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48726 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:15:27,881 - lava_callback - INFO - Received callback for node 69c2516c4a2c5046eb88027b INFO: 172.20.0.1:48742 - "POST /node/69c2516c4a2c5046eb88027b HTTP/1.0" 202 Accepted 2026-03-24 09:15:27,889 - lava_callback - INFO - Starting async job submit for node 69c2516c4a2c5046eb88027b 2026-03-24 09:15:27,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:15:27,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2516c4a2c5046eb88027b HTTP/1.1" 200 None 2026-03-24 09:15:27,949 - lava_callback - DEBUG - Processing node 69c2516c4a2c5046eb88027b: baseline-nfs-arm64 2026-03-24 09:15:28,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:15:29,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:15:29,132 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516c4a2c5046eb88027b/log.txt.gz 2026-03-24 09:15:29,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:15:30,076 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:15:30,079 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516c4a2c5046eb88027b/lava-logs.yaml 2026-03-24 09:15:30,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:15:30,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:15:30,290 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2516c4a2c5046eb88027b/lava_callback.json.gz DEBUG: Setup failed for 69c2516c4a2c5046eb88027b. Transitting job node result: incomplete -> fail 2026-03-24 09:15:30,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:15:30,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2516c4a2c5046eb88027b HTTP/1.1" 200 None 2026-03-24 09:15:30,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:15:30,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:15:30,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:15:30,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2516c4a2c5046eb88027b HTTP/1.1" 200 None 2026-03-24 09:15:30,543 - lava_callback - INFO - Completed processing callback for node 69c2516c4a2c5046eb88027b INFO: 172.20.0.1:40806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38088 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:15:55,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:15:55,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51820 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:16:29,490 - lava_callback - INFO - Received callback for node 69c2565f4a2c5046eb88059f INFO: 172.20.0.1:51830 - "POST /node/69c2565f4a2c5046eb88059f HTTP/1.0" 202 Accepted 2026-03-24 09:16:29,496 - lava_callback - INFO - Starting async job submit for node 69c2565f4a2c5046eb88059f 2026-03-24 09:16:29,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:16:29,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2565f4a2c5046eb88059f HTTP/1.1" 200 None 2026-03-24 09:16:29,514 - lava_callback - DEBUG - Processing node 69c2565f4a2c5046eb88059f: baseline-arm 2026-03-24 09:16:29,519 - lava_callback - WARNING - Failed to upload log 2026-03-24 09:16:29,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:16:29,783 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:16:29,785 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2565f4a2c5046eb88059f/lava-logs.yaml 2026-03-24 09:16:29,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:16:29,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:16:29,920 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2565f4a2c5046eb88059f/lava_callback.json.gz 2026-03-24 09:16:29,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:16:29,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2565f4a2c5046eb88059f HTTP/1.1" 200 None 2026-03-24 09:16:29,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:16:29,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 09:16:29,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:16:29,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2565f4a2c5046eb88059f HTTP/1.1" 200 None 2026-03-24 09:16:29,990 - lava_callback - INFO - Completed processing callback for node 69c2565f4a2c5046eb88059f INFO: 172.20.0.1:60720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54016 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:16:55,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:16:55,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55326 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:27:49,829 - lava_callback - INFO - Received callback for node 69c259104a2c5046eb880636 INFO: 172.20.0.1:43230 - "POST /node/69c259104a2c5046eb880636 HTTP/1.0" 202 Accepted 2026-03-24 09:27:49,835 - lava_callback - INFO - Starting async job submit for node 69c259104a2c5046eb880636 2026-03-24 09:27:49,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:27:49,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c259104a2c5046eb880636 HTTP/1.1" 200 None 2026-03-24 09:27:49,853 - lava_callback - DEBUG - Processing node 69c259104a2c5046eb880636: baseline-nfs-arm64 2026-03-24 09:27:49,854 - lava_callback - WARNING - Failed to upload log 2026-03-24 09:27:49,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:27:50,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:27:50,003 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c259104a2c5046eb880636/lava-logs.yaml 2026-03-24 09:27:50,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:27:50,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:27:50,145 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c259104a2c5046eb880636/lava_callback.json.gz 2026-03-24 09:27:50,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:27:50,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c259104a2c5046eb880636 HTTP/1.1" 200 None 2026-03-24 09:27:50,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:27:50,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:27:50,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:27:50,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c259104a2c5046eb880636 HTTP/1.1" 200 None 2026-03-24 09:27:50,209 - lava_callback - INFO - Completed processing callback for node 69c259104a2c5046eb880636 2026-03-24 09:27:55,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:27:55,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33788 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:31:09,397 - lava_callback - INFO - Received callback for node 69c255bb4a2c5046eb880580 INFO: 172.20.0.1:33792 - "POST /node/69c255bb4a2c5046eb880580 HTTP/1.0" 202 Accepted 2026-03-24 09:31:09,404 - lava_callback - INFO - Starting async job submit for node 69c255bb4a2c5046eb880580 2026-03-24 09:31:09,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:31:09,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c255bb4a2c5046eb880580 HTTP/1.1" 200 None 2026-03-24 09:31:09,485 - lava_callback - DEBUG - Processing node 69c255bb4a2c5046eb880580: ltp-smoke 2026-03-24 09:31:09,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:31:10,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:31:10,107 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c255bb4a2c5046eb880580/log.txt.gz 2026-03-24 09:31:10,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:31:10,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:31:10,649 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c255bb4a2c5046eb880580/lava-logs.yaml 2026-03-24 09:31:10,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:31:10,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:31:10,859 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c255bb4a2c5046eb880580/lava_callback.json.gz 2026-03-24 09:31:10,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:31:10,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c255bb4a2c5046eb880580 HTTP/1.1" 200 None 2026-03-24 09:31:10,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:31:11,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248de4a2c5046eb87f896 HTTP/1.1" 200 None 2026-03-24 09:31:11,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:31:11,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c255bb4a2c5046eb880580 HTTP/1.1" 200 None 2026-03-24 09:31:11,177 - lava_callback - INFO - Completed processing callback for node 69c255bb4a2c5046eb880580 INFO: 172.20.0.1:39382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39404 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:31:25,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:31:25,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34060 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:31:53,156 - lava_callback - INFO - Received callback for node 69c259cc4a2c5046eb88065e INFO: 172.20.0.1:53600 - "POST /node/69c259cc4a2c5046eb88065e HTTP/1.0" 202 Accepted 2026-03-24 09:31:53,163 - lava_callback - INFO - Starting async job submit for node 69c259cc4a2c5046eb88065e 2026-03-24 09:31:53,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:31:53,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c259cc4a2c5046eb88065e HTTP/1.1" 200 None 2026-03-24 09:31:53,210 - lava_callback - DEBUG - Processing node 69c259cc4a2c5046eb88065e: baseline-arm64 2026-03-24 09:31:53,349 - lava_callback - WARNING - Failed to upload log 2026-03-24 09:31:53,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:31:53,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:31:53,623 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c259cc4a2c5046eb88065e/lava-logs.yaml 2026-03-24 09:31:53,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:31:53,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:31:53,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c259cc4a2c5046eb88065e/lava_callback.json.gz 2026-03-24 09:31:53,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:31:53,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c259cc4a2c5046eb88065e HTTP/1.1" 200 None 2026-03-24 09:31:53,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:31:53,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:31:53,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:31:53,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c259cc4a2c5046eb88065e HTTP/1.1" 200 None 2026-03-24 09:31:53,931 - lava_callback - INFO - Completed processing callback for node 69c259cc4a2c5046eb88065e 2026-03-24 09:31:55,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:31:55,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49292 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:33:14,098 - lava_callback - INFO - Received callback for node 69c25a204a2c5046eb880672 INFO: 172.20.0.1:42062 - "POST /node/69c25a204a2c5046eb880672 HTTP/1.0" 202 Accepted 2026-03-24 09:33:14,105 - lava_callback - INFO - Starting async job submit for node 69c25a204a2c5046eb880672 2026-03-24 09:33:14,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:33:14,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25a204a2c5046eb880672 HTTP/1.1" 200 None 2026-03-24 09:33:14,144 - lava_callback - DEBUG - Processing node 69c25a204a2c5046eb880672: baseline-arm64 2026-03-24 09:33:14,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:33:14,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:33:14,641 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25a204a2c5046eb880672/log.txt.gz 2026-03-24 09:33:14,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:33:15,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:33:15,077 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25a204a2c5046eb880672/lava-logs.yaml 2026-03-24 09:33:15,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:42066 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-24 09:33:15,227 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:33:15,228 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25a204a2c5046eb880672/lava_callback.json.gz 2026-03-24 09:33:15,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:33:15,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25a204a2c5046eb880672 HTTP/1.1" 200 None 2026-03-24 09:33:15,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:33:15,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:33:15,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:33:15,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25a204a2c5046eb880672 HTTP/1.1" 200 None 2026-03-24 09:33:15,328 - lava_callback - INFO - Completed processing callback for node 69c25a204a2c5046eb880672 INFO: 172.20.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42084 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:33:25,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:33:25,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33222 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:35:18,254 - lava_callback - INFO - Received callback for node 69c25a6f4a2c5046eb880686 INFO: 172.20.0.1:33224 - "POST /node/69c25a6f4a2c5046eb880686 HTTP/1.0" 202 Accepted 2026-03-24 09:35:18,261 - lava_callback - INFO - Starting async job submit for node 69c25a6f4a2c5046eb880686 2026-03-24 09:35:18,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:35:18,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25a6f4a2c5046eb880686 HTTP/1.1" 200 None 2026-03-24 09:35:18,325 - lava_callback - DEBUG - Processing node 69c25a6f4a2c5046eb880686: baseline-arm64 2026-03-24 09:35:18,526 - lava_callback - WARNING - Failed to upload log 2026-03-24 09:35:18,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:35:18,860 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:35:18,862 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25a6f4a2c5046eb880686/lava-logs.yaml 2026-03-24 09:35:18,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:35:19,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:35:19,005 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25a6f4a2c5046eb880686/lava_callback.json.gz 2026-03-24 09:35:19,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:35:19,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25a6f4a2c5046eb880686 HTTP/1.1" 200 None 2026-03-24 09:35:19,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:35:19,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:35:19,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:35:19,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25a6f4a2c5046eb880686 HTTP/1.1" 200 None 2026-03-24 09:35:19,201 - lava_callback - INFO - Completed processing callback for node 69c25a6f4a2c5046eb880686 2026-03-24 09:35:21,676 - lava_callback - INFO - Received callback for node 69bfb7c87544ab5cd4a7e85d INFO: 172.20.0.1:39492 - "POST /node/69bfb7c87544ab5cd4a7e85d HTTP/1.0" 202 Accepted 2026-03-24 09:35:21,683 - lava_callback - INFO - Starting async job submit for node 69bfb7c87544ab5cd4a7e85d 2026-03-24 09:35:21,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:35:21,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfb7c87544ab5cd4a7e85d HTTP/1.1" 200 None 2026-03-24 09:35:21,699 - lava_callback - DEBUG - Processing node 69bfb7c87544ab5cd4a7e85d: baseline-arm64-baylibre 2026-03-24 09:35:21,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfb7c87544ab5cd4a7e85d 2026-03-24 09:35:21,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:35:21,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfb7c87544ab5cd4a7e85d HTTP/1.1" 200 None 2026-03-24 09:35:21,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:35:21,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:35:21,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:35:21,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfb7c87544ab5cd4a7e85d HTTP/1.1" 200 None 2026-03-24 09:35:21,765 - lava_callback - ERROR - Error processing callback for node 69bfb7c87544ab5cd4a7e85d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 09:35:25,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:35:25,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36152 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:37:41,806 - lava_callback - INFO - Received callback for node 69bfb85e7544ab5cd4a7e884 INFO: 172.20.0.1:59252 - "POST /node/69bfb85e7544ab5cd4a7e884 HTTP/1.0" 202 Accepted 2026-03-24 09:37:41,811 - lava_callback - INFO - Starting async job submit for node 69bfb85e7544ab5cd4a7e884 2026-03-24 09:37:41,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:37:41,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfb85e7544ab5cd4a7e884 HTTP/1.1" 200 None 2026-03-24 09:37:41,828 - lava_callback - DEBUG - Processing node 69bfb85e7544ab5cd4a7e884: baseline-arm64-baylibre 2026-03-24 09:37:41,829 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfb85e7544ab5cd4a7e884 2026-03-24 09:37:41,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:37:41,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfb85e7544ab5cd4a7e884 HTTP/1.1" 200 None 2026-03-24 09:37:41,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:37:41,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:37:41,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:37:41,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfb85e7544ab5cd4a7e884 HTTP/1.1" 200 None 2026-03-24 09:37:41,895 - lava_callback - ERROR - Error processing callback for node 69bfb85e7544ab5cd4a7e884: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57234 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:41:22,151 - lava_callback - INFO - Received callback for node 69bfb9397544ab5cd4a7e8aa INFO: 172.20.0.1:50772 - "POST /node/69bfb9397544ab5cd4a7e8aa HTTP/1.0" 202 Accepted 2026-03-24 09:41:22,156 - lava_callback - INFO - Starting async job submit for node 69bfb9397544ab5cd4a7e8aa 2026-03-24 09:41:22,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:41:22,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfb9397544ab5cd4a7e8aa HTTP/1.1" 200 None 2026-03-24 09:41:22,172 - lava_callback - DEBUG - Processing node 69bfb9397544ab5cd4a7e8aa: baseline-arm64-baylibre 2026-03-24 09:41:22,172 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfb9397544ab5cd4a7e8aa 2026-03-24 09:41:22,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:41:22,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfb9397544ab5cd4a7e8aa HTTP/1.1" 200 None 2026-03-24 09:41:22,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:41:22,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:41:22,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:41:22,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfb9397544ab5cd4a7e8aa HTTP/1.1" 200 None 2026-03-24 09:41:22,231 - lava_callback - ERROR - Error processing callback for node 69bfb9397544ab5cd4a7e8aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:43:02,093 - lava_callback - INFO - Received callback for node 69c25b5f4a2c5046eb8806b7 INFO: 172.20.0.1:51122 - "POST /node/69c25b5f4a2c5046eb8806b7 HTTP/1.0" 202 Accepted 2026-03-24 09:43:02,100 - lava_callback - INFO - Starting async job submit for node 69c25b5f4a2c5046eb8806b7 2026-03-24 09:43:02,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:43:02,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25b5f4a2c5046eb8806b7 HTTP/1.1" 200 None 2026-03-24 09:43:02,151 - lava_callback - DEBUG - Processing node 69c25b5f4a2c5046eb8806b7: baseline-arm64 2026-03-24 09:43:02,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:43:02,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:43:02,724 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25b5f4a2c5046eb8806b7/log.txt.gz 2026-03-24 09:43:02,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:43:03,177 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:43:03,179 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25b5f4a2c5046eb8806b7/lava-logs.yaml 2026-03-24 09:43:03,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:43:03,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:43:03,385 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25b5f4a2c5046eb8806b7/lava_callback.json.gz DEBUG: Setup failed for 69c25b5f4a2c5046eb8806b7. Transitting job node result: incomplete -> fail 2026-03-24 09:43:03,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:43:03,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25b5f4a2c5046eb8806b7 HTTP/1.1" 200 None 2026-03-24 09:43:03,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:43:03,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:43:03,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:43:03,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25b5f4a2c5046eb8806b7 HTTP/1.1" 200 None 2026-03-24 09:43:03,594 - lava_callback - INFO - Completed processing callback for node 69c25b5f4a2c5046eb8806b7 INFO: 172.20.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33380 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:43:25,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:43:25,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44484 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:44:02,391 - lava_callback - INFO - Received callback for node 69bfb9ce7544ab5cd4a7e8bf INFO: 172.20.0.1:51912 - "POST /node/69bfb9ce7544ab5cd4a7e8bf HTTP/1.0" 202 Accepted 2026-03-24 09:44:02,397 - lava_callback - INFO - Starting async job submit for node 69bfb9ce7544ab5cd4a7e8bf 2026-03-24 09:44:02,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:44:02,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfb9ce7544ab5cd4a7e8bf HTTP/1.1" 200 None 2026-03-24 09:44:02,414 - lava_callback - DEBUG - Processing node 69bfb9ce7544ab5cd4a7e8bf: baseline-arm64-baylibre 2026-03-24 09:44:02,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfb9ce7544ab5cd4a7e8bf 2026-03-24 09:44:02,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:44:02,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfb9ce7544ab5cd4a7e8bf HTTP/1.1" 200 None 2026-03-24 09:44:02,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:44:02,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:44:02,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:44:02,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfb9ce7544ab5cd4a7e8bf HTTP/1.1" 200 None 2026-03-24 09:44:02,468 - lava_callback - ERROR - Error processing callback for node 69bfb9ce7544ab5cd4a7e8bf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48180 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:47:42,347 - lava_callback - INFO - Received callback for node 69bfbaad7544ab5cd4a7e8eb INFO: 172.20.0.1:38264 - "POST /node/69bfbaad7544ab5cd4a7e8eb HTTP/1.0" 202 Accepted 2026-03-24 09:47:42,353 - lava_callback - INFO - Starting async job submit for node 69bfbaad7544ab5cd4a7e8eb 2026-03-24 09:47:42,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:47:42,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbaad7544ab5cd4a7e8eb HTTP/1.1" 200 None 2026-03-24 09:47:42,378 - lava_callback - DEBUG - Processing node 69bfbaad7544ab5cd4a7e8eb: baseline-arm64-baylibre 2026-03-24 09:47:42,378 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfbaad7544ab5cd4a7e8eb 2026-03-24 09:47:42,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:47:42,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbaad7544ab5cd4a7e8eb HTTP/1.1" 200 None 2026-03-24 09:47:42,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:47:42,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:47:42,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:47:42,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfbaad7544ab5cd4a7e8eb HTTP/1.1" 200 None 2026-03-24 09:47:42,436 - lava_callback - ERROR - Error processing callback for node 69bfbaad7544ab5cd4a7e8eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36794 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:48:42,362 - lava_callback - INFO - Received callback for node 69bfbaf67544ab5cd4a7e8fa INFO: 172.20.0.1:37774 - "POST /node/69bfbaf67544ab5cd4a7e8fa HTTP/1.0" 202 Accepted 2026-03-24 09:48:42,368 - lava_callback - INFO - Starting async job submit for node 69bfbaf67544ab5cd4a7e8fa 2026-03-24 09:48:42,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:48:42,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbaf67544ab5cd4a7e8fa HTTP/1.1" 200 None 2026-03-24 09:48:42,389 - lava_callback - DEBUG - Processing node 69bfbaf67544ab5cd4a7e8fa: baseline-arm64-baylibre 2026-03-24 09:48:42,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfbaf67544ab5cd4a7e8fa 2026-03-24 09:48:42,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:48:42,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbaf67544ab5cd4a7e8fa HTTP/1.1" 200 None 2026-03-24 09:48:42,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:48:42,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:48:42,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:48:42,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfbaf67544ab5cd4a7e8fa HTTP/1.1" 200 None 2026-03-24 09:48:42,447 - lava_callback - ERROR - Error processing callback for node 69bfbaf67544ab5cd4a7e8fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45514 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:49:30,487 - lava_callback - INFO - Received callback for node 69c25e264a2c5046eb88073a INFO: 172.20.0.1:56096 - "POST /node/69c25e264a2c5046eb88073a HTTP/1.0" 202 Accepted 2026-03-24 09:49:30,493 - lava_callback - INFO - Starting async job submit for node 69c25e264a2c5046eb88073a 2026-03-24 09:49:30,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:49:30,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25e264a2c5046eb88073a HTTP/1.1" 200 None 2026-03-24 09:49:30,510 - lava_callback - DEBUG - Processing node 69c25e264a2c5046eb88073a: baseline-nfs-arm64 2026-03-24 09:49:30,512 - lava_callback - WARNING - Failed to upload log 2026-03-24 09:49:30,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:49:30,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:49:30,672 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c25e264a2c5046eb88073a/lava-logs.yaml 2026-03-24 09:49:30,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:49:30,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:49:30,821 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c25e264a2c5046eb88073a/lava_callback.json.gz 2026-03-24 09:49:30,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:49:30,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25e264a2c5046eb88073a HTTP/1.1" 200 None 2026-03-24 09:49:30,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:49:30,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:49:30,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:49:30,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25e264a2c5046eb88073a HTTP/1.1" 200 None 2026-03-24 09:49:30,882 - lava_callback - INFO - Completed processing callback for node 69c25e264a2c5046eb88073a INFO: 172.20.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58466 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:49:55,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:49:55,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42506 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:51:18,096 - lava_callback - INFO - Received callback for node 69bfbb917544ab5cd4a7e91d INFO: 172.20.0.1:42508 - "POST /node/69bfbb917544ab5cd4a7e91d HTTP/1.0" 202 Accepted 2026-03-24 09:51:18,101 - lava_callback - INFO - Starting async job submit for node 69bfbb917544ab5cd4a7e91d 2026-03-24 09:51:18,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:51:18,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbb917544ab5cd4a7e91d HTTP/1.1" 200 None 2026-03-24 09:51:18,117 - lava_callback - DEBUG - Processing node 69bfbb917544ab5cd4a7e91d: baseline-arm64-clabbe 2026-03-24 09:51:18,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfbb917544ab5cd4a7e91d 2026-03-24 09:51:18,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:51:18,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbb917544ab5cd4a7e91d HTTP/1.1" 200 None 2026-03-24 09:51:18,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:51:18,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:51:18,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:51:18,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfbb917544ab5cd4a7e91d HTTP/1.1" 200 None 2026-03-24 09:51:18,176 - lava_callback - ERROR - Error processing callback for node 69bfbb917544ab5cd4a7e91d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36326 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:52:35,835 - lava_callback - INFO - Received callback for node 69c25e704a2c5046eb880747 INFO: 172.20.0.1:56264 - "POST /node/69c25e704a2c5046eb880747 HTTP/1.0" 202 Accepted 2026-03-24 09:52:35,842 - lava_callback - INFO - Starting async job submit for node 69c25e704a2c5046eb880747 2026-03-24 09:52:35,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:52:35,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25e704a2c5046eb880747 HTTP/1.1" 200 None 2026-03-24 09:52:35,887 - lava_callback - DEBUG - Processing node 69c25e704a2c5046eb880747: baseline-arm64 2026-03-24 09:52:36,020 - lava_callback - WARNING - Failed to upload log 2026-03-24 09:52:36,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:52:36,310 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:52:36,311 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25e704a2c5046eb880747/lava-logs.yaml 2026-03-24 09:52:36,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:52:36,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:52:36,445 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25e704a2c5046eb880747/lava_callback.json.gz 2026-03-24 09:52:36,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:52:36,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25e704a2c5046eb880747 HTTP/1.1" 200 None 2026-03-24 09:52:36,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:52:36,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:52:36,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:56274 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:52:36,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25e704a2c5046eb880747 HTTP/1.1" 200 None 2026-03-24 09:52:36,605 - lava_callback - INFO - Completed processing callback for node 69c25e704a2c5046eb880747 INFO: 172.20.0.1:43230 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:52:55,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:52:55,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56752 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:53:00,377 - lava_callback - INFO - Received callback for node 69c25ec24a2c5046eb880753 INFO: 172.20.0.1:41824 - "POST /node/69c25ec24a2c5046eb880753 HTTP/1.0" 202 Accepted 2026-03-24 09:53:00,383 - lava_callback - INFO - Starting async job submit for node 69c25ec24a2c5046eb880753 2026-03-24 09:53:00,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:00,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25ec24a2c5046eb880753 HTTP/1.1" 200 None 2026-03-24 09:53:00,428 - lava_callback - DEBUG - Processing node 69c25ec24a2c5046eb880753: baseline-arm64 2026-03-24 09:53:00,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:53:00,721 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:53:00,723 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25ec24a2c5046eb880753/log.txt.gz 2026-03-24 09:53:00,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:53:00,968 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:53:00,970 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25ec24a2c5046eb880753/lava-logs.yaml 2026-03-24 09:53:00,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:53:01,106 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:53:01,108 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25ec24a2c5046eb880753/lava_callback.json.gz 2026-03-24 09:53:01,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:01,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25ec24a2c5046eb880753 HTTP/1.1" 200 None 2026-03-24 09:53:01,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:01,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:53:01,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:01,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25ec24a2c5046eb880753 HTTP/1.1" 200 None 2026-03-24 09:53:01,334 - lava_callback - INFO - Completed processing callback for node 69c25ec24a2c5046eb880753 INFO: 172.20.0.1:41836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46398 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:53:18,125 - lava_callback - INFO - Received callback for node 69bfbc007544ab5cd4a7e944 INFO: 172.20.0.1:46412 - "POST /node/69bfbc007544ab5cd4a7e944 HTTP/1.0" 202 Accepted 2026-03-24 09:53:18,131 - lava_callback - INFO - Starting async job submit for node 69bfbc007544ab5cd4a7e944 2026-03-24 09:53:18,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:18,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbc007544ab5cd4a7e944 HTTP/1.1" 200 None 2026-03-24 09:53:18,147 - lava_callback - DEBUG - Processing node 69bfbc007544ab5cd4a7e944: baseline-arm64-clabbe 2026-03-24 09:53:18,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfbc007544ab5cd4a7e944 2026-03-24 09:53:18,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:18,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbc007544ab5cd4a7e944 HTTP/1.1" 200 None 2026-03-24 09:53:18,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:18,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:53:18,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:18,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfbc007544ab5cd4a7e944 HTTP/1.1" 200 None 2026-03-24 09:53:18,216 - lava_callback - ERROR - Error processing callback for node 69bfbc007544ab5cd4a7e944: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 09:53:25,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:25,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33662 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:53:43,993 - lava_callback - INFO - Received callback for node 69c25bd24a2c5046eb8806ce INFO: 172.20.0.1:34444 - "POST /node/69c25bd24a2c5046eb8806ce HTTP/1.0" 202 Accepted 2026-03-24 09:53:44,000 - lava_callback - INFO - Starting async job submit for node 69c25bd24a2c5046eb8806ce 2026-03-24 09:53:44,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:44,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25bd24a2c5046eb8806ce HTTP/1.1" 200 None 2026-03-24 09:53:44,039 - lava_callback - DEBUG - Processing node 69c25bd24a2c5046eb8806ce: baseline-arm64 2026-03-24 09:53:44,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:53:44,702 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:53:44,704 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25bd24a2c5046eb8806ce/log.txt.gz 2026-03-24 09:53:44,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:53:45,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:53:45,166 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25bd24a2c5046eb8806ce/lava-logs.yaml 2026-03-24 09:53:45,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:53:45,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:53:45,381 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25bd24a2c5046eb8806ce/lava_callback.json.gz 2026-03-24 09:53:45,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:45,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25bd24a2c5046eb8806ce HTTP/1.1" 200 None 2026-03-24 09:53:45,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:45,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:53:45,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:45,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25bd24a2c5046eb8806ce HTTP/1.1" 200 None 2026-03-24 09:53:45,518 - lava_callback - INFO - Completed processing callback for node 69c25bd24a2c5046eb8806ce INFO: 172.20.0.1:34448 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:53:55,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:53:55,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46984 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:54:27,907 - lava_callback - INFO - Received callback for node 69c25b364a2c5046eb8806ac INFO: 172.20.0.1:47000 - "POST /node/69c25b364a2c5046eb8806ac HTTP/1.0" 202 Accepted 2026-03-24 09:54:27,914 - lava_callback - INFO - Starting async job submit for node 69c25b364a2c5046eb8806ac 2026-03-24 09:54:27,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:54:27,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25b364a2c5046eb8806ac HTTP/1.1" 200 None 2026-03-24 09:54:28,000 - lava_callback - DEBUG - Processing node 69c25b364a2c5046eb8806ac: baseline-arm 2026-03-24 09:54:28,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:54:28,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:54:28,718 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c25b364a2c5046eb8806ac/log.txt.gz 2026-03-24 09:54:29,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:54:29,340 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:54:29,342 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c25b364a2c5046eb8806ac/lava-logs.yaml 2026-03-24 09:54:29,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:54:29,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:54:29,538 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c25b364a2c5046eb8806ac/lava_callback.json.gz DEBUG: Setup failed for 69c25b364a2c5046eb8806ac. Transitting job node result: incomplete -> fail 2026-03-24 09:54:29,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:54:29,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25b364a2c5046eb8806ac HTTP/1.1" 200 None 2026-03-24 09:54:29,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:54:29,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 09:54:29,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:54:29,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25b364a2c5046eb8806ac HTTP/1.1" 200 None 2026-03-24 09:54:29,825 - lava_callback - INFO - Completed processing callback for node 69c25b364a2c5046eb8806ac INFO: 172.20.0.1:41572 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:54:38,226 - lava_callback - INFO - Received callback for node 69bfbc4b7544ab5cd4a7e955 INFO: 172.20.0.1:41584 - "POST /node/69bfbc4b7544ab5cd4a7e955 HTTP/1.0" 202 Accepted 2026-03-24 09:54:38,232 - lava_callback - INFO - Starting async job submit for node 69bfbc4b7544ab5cd4a7e955 2026-03-24 09:54:38,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:54:38,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbc4b7544ab5cd4a7e955 HTTP/1.1" 200 None 2026-03-24 09:54:38,247 - lava_callback - DEBUG - Processing node 69bfbc4b7544ab5cd4a7e955: baseline-arm64-clabbe 2026-03-24 09:54:38,248 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfbc4b7544ab5cd4a7e955 2026-03-24 09:54:38,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:54:38,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbc4b7544ab5cd4a7e955 HTTP/1.1" 200 None 2026-03-24 09:54:38,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:54:38,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:54:38,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:54:38,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfbc4b7544ab5cd4a7e955 HTTP/1.1" 200 None 2026-03-24 09:54:38,303 - lava_callback - ERROR - Error processing callback for node 69bfbc4b7544ab5cd4a7e955: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54580 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:54:55,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:54:55,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36160 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:56:58,924 - lava_callback - INFO - Received callback for node 69c25c274a2c5046eb8806dc INFO: 172.20.0.1:36172 - "POST /node/69c25c274a2c5046eb8806dc HTTP/1.0" 202 Accepted 2026-03-24 09:56:58,931 - lava_callback - INFO - Starting async job submit for node 69c25c274a2c5046eb8806dc 2026-03-24 09:56:58,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:56:58,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25c274a2c5046eb8806dc HTTP/1.1" 200 None 2026-03-24 09:56:58,974 - lava_callback - DEBUG - Processing node 69c25c274a2c5046eb8806dc: baseline-nfs-arm64 2026-03-24 09:56:59,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:56:59,669 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:56:59,670 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c25c274a2c5046eb8806dc/log.txt.gz 2026-03-24 09:56:59,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:57:00,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:57:00,194 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c25c274a2c5046eb8806dc/lava-logs.yaml 2026-03-24 09:57:00,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:57:00,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:57:00,368 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c25c274a2c5046eb8806dc/lava_callback.json.gz 2026-03-24 09:57:00,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:57:00,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25c274a2c5046eb8806dc HTTP/1.1" 200 None 2026-03-24 09:57:00,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:57:00,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:57:00,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:57:00,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25c274a2c5046eb8806dc HTTP/1.1" 200 None 2026-03-24 09:57:00,526 - lava_callback - INFO - Completed processing callback for node 69c25c274a2c5046eb8806dc INFO: 172.20.0.1:46580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41282 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:57:25,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:57:25,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42678 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:58:18,361 - lava_callback - INFO - Received callback for node 69bfbd267544ab5cd4a7e990 INFO: 172.20.0.1:42688 - "POST /node/69bfbd267544ab5cd4a7e990 HTTP/1.0" 202 Accepted 2026-03-24 09:58:18,367 - lava_callback - INFO - Starting async job submit for node 69bfbd267544ab5cd4a7e990 2026-03-24 09:58:18,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:58:18,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbd267544ab5cd4a7e990 HTTP/1.1" 200 None 2026-03-24 09:58:18,386 - lava_callback - DEBUG - Processing node 69bfbd267544ab5cd4a7e990: baseline-arm64-clabbe 2026-03-24 09:58:18,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfbd267544ab5cd4a7e990 2026-03-24 09:58:18,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:58:18,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbd267544ab5cd4a7e990 HTTP/1.1" 200 None 2026-03-24 09:58:18,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:58:18,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:58:18,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:58:18,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfbd267544ab5cd4a7e990 HTTP/1.1" 200 None 2026-03-24 09:58:18,443 - lava_callback - ERROR - Error processing callback for node 69bfbd267544ab5cd4a7e990: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 09:58:22,664 - lava_callback - INFO - Received callback for node 69c25bb54a2c5046eb8806c6 INFO: 172.20.0.1:54840 - "POST /node/69c25bb54a2c5046eb8806c6 HTTP/1.0" 202 Accepted 2026-03-24 09:58:22,709 - lava_callback - INFO - Starting async job submit for node 69c25bb54a2c5046eb8806c6 2026-03-24 09:58:22,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:58:22,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25bb54a2c5046eb8806c6 HTTP/1.1" 200 None 2026-03-24 09:58:22,748 - lava_callback - DEBUG - Processing node 69c25bb54a2c5046eb8806c6: baseline-arm-pengutronix 2026-03-24 09:58:22,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:58:23,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:58:23,006 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c25bb54a2c5046eb8806c6/log.txt.gz 2026-03-24 09:58:23,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:58:23,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:58:23,277 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c25bb54a2c5046eb8806c6/lava-logs.yaml 2026-03-24 09:58:23,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:58:23,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:58:23,417 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c25bb54a2c5046eb8806c6/lava_callback.json.gz DEBUG: Setup failed for 69c25bb54a2c5046eb8806c6. Transitting job node result: incomplete -> fail 2026-03-24 09:58:23,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:58:23,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25bb54a2c5046eb8806c6 HTTP/1.1" 200 None 2026-03-24 09:58:23,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:58:23,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 09:58:23,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:58:23,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25bb54a2c5046eb8806c6 HTTP/1.1" 200 None 2026-03-24 09:58:23,596 - lava_callback - INFO - Completed processing callback for node 69c25bb54a2c5046eb8806c6 2026-03-24 09:58:25,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:58:25,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47538 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:59:09,357 - lava_callback - INFO - Received callback for node 69c25f174a2c5046eb880765 INFO: 172.20.0.1:47546 - "POST /node/69c25f174a2c5046eb880765 HTTP/1.0" 202 Accepted 2026-03-24 09:59:09,365 - lava_callback - INFO - Starting async job submit for node 69c25f174a2c5046eb880765 2026-03-24 09:59:09,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:09,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25f174a2c5046eb880765 HTTP/1.1" 200 None 2026-03-24 09:59:09,418 - lava_callback - DEBUG - Processing node 69c25f174a2c5046eb880765: baseline-arm64 2026-03-24 09:59:09,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:59:09,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:59:09,945 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25f174a2c5046eb880765/log.txt.gz 2026-03-24 09:59:10,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:59:10,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:59:10,426 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25f174a2c5046eb880765/lava-logs.yaml 2026-03-24 09:59:10,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:59:10,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:59:10,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c25f174a2c5046eb880765/lava_callback.json.gz DEBUG: Setup failed for 69c25f174a2c5046eb880765. Transitting job node result: incomplete -> fail 2026-03-24 09:59:10,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:10,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25f174a2c5046eb880765 HTTP/1.1" 200 None 2026-03-24 09:59:10,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:10,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:59:10,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:10,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25f174a2c5046eb880765 HTTP/1.1" 200 None 2026-03-24 09:59:10,836 - lava_callback - INFO - Completed processing callback for node 69c25f174a2c5046eb880765 2026-03-24 09:59:10,860 - lava_callback - INFO - Received callback for node 69c2605d4a2c5046eb880799 INFO: 172.20.0.1:60212 - "POST /node/69c2605d4a2c5046eb880799 HTTP/1.0" 202 Accepted 2026-03-24 09:59:10,870 - lava_callback - INFO - Starting async job submit for node 69c2605d4a2c5046eb880799 2026-03-24 09:59:10,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:10,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2605d4a2c5046eb880799 HTTP/1.1" 200 None 2026-03-24 09:59:10,907 - lava_callback - DEBUG - Processing node 69c2605d4a2c5046eb880799: baseline-nfs-arm64 2026-03-24 09:59:10,909 - lava_callback - WARNING - Failed to upload log 2026-03-24 09:59:10,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:59:11,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:59:11,056 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2605d4a2c5046eb880799/lava-logs.yaml 2026-03-24 09:59:11,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 09:59:11,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 09:59:11,189 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2605d4a2c5046eb880799/lava_callback.json.gz 2026-03-24 09:59:11,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:11,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2605d4a2c5046eb880799 HTTP/1.1" 200 None 2026-03-24 09:59:11,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:11,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 09:59:11,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:11,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2605d4a2c5046eb880799 HTTP/1.1" 200 None 2026-03-24 09:59:11,256 - lava_callback - INFO - Completed processing callback for node 69c2605d4a2c5046eb880799 INFO: 172.20.0.1:60222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60234 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:59:25,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:25,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46562 - "GET / HTTP/1.0" 200 OK 2026-03-24 09:59:58,411 - lava_callback - INFO - Received callback for node 69bfbd927544ab5cd4a7e9ab INFO: 172.20.0.1:46572 - "POST /node/69bfbd927544ab5cd4a7e9ab HTTP/1.0" 202 Accepted 2026-03-24 09:59:58,419 - lava_callback - INFO - Starting async job submit for node 69bfbd927544ab5cd4a7e9ab 2026-03-24 09:59:58,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:58,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbd927544ab5cd4a7e9ab HTTP/1.1" 200 None 2026-03-24 09:59:58,457 - lava_callback - DEBUG - Processing node 69bfbd927544ab5cd4a7e9ab: baseline-arm64-clabbe 2026-03-24 09:59:58,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfbd927544ab5cd4a7e9ab 2026-03-24 09:59:58,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:58,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbd927544ab5cd4a7e9ab HTTP/1.1" 200 None 2026-03-24 09:59:58,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:58,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 09:59:58,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 09:59:58,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfbd927544ab5cd4a7e9ab HTTP/1.1" 200 None 2026-03-24 09:59:58,524 - lava_callback - ERROR - Error processing callback for node 69bfbd927544ab5cd4a7e9ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35854 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:01:21,293 - lava_callback - INFO - Received callback for node 69c260a84a2c5046eb8807aa INFO: 172.20.0.1:39744 - "POST /node/69c260a84a2c5046eb8807aa HTTP/1.0" 202 Accepted 2026-03-24 10:01:21,300 - lava_callback - INFO - Starting async job submit for node 69c260a84a2c5046eb8807aa 2026-03-24 10:01:21,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:01:21,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c260a84a2c5046eb8807aa HTTP/1.1" 200 None 2026-03-24 10:01:21,346 - lava_callback - DEBUG - Processing node 69c260a84a2c5046eb8807aa: baseline-arm64 2026-03-24 10:01:21,496 - lava_callback - WARNING - Failed to upload log 2026-03-24 10:01:21,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:01:21,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:01:21,794 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c260a84a2c5046eb8807aa/lava-logs.yaml 2026-03-24 10:01:21,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:01:21,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:01:21,936 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c260a84a2c5046eb8807aa/lava_callback.json.gz 2026-03-24 10:01:21,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:01:21,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c260a84a2c5046eb8807aa HTTP/1.1" 200 None 2026-03-24 10:01:21,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:01:21,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 10:01:21,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:01:22,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c260a84a2c5046eb8807aa HTTP/1.1" 200 None 2026-03-24 10:01:22,093 - lava_callback - INFO - Completed processing callback for node 69c260a84a2c5046eb8807aa 2026-03-24 10:01:25,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:01:25,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55968 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:02:18,528 - lava_callback - INFO - Received callback for node 69bfbe217544ab5cd4a7e9ca INFO: 172.20.0.1:55978 - "POST /node/69bfbe217544ab5cd4a7e9ca HTTP/1.0" 202 Accepted 2026-03-24 10:02:18,533 - lava_callback - INFO - Starting async job submit for node 69bfbe217544ab5cd4a7e9ca 2026-03-24 10:02:18,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:02:18,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbe217544ab5cd4a7e9ca HTTP/1.1" 200 None 2026-03-24 10:02:18,550 - lava_callback - DEBUG - Processing node 69bfbe217544ab5cd4a7e9ca: baseline-arm64-clabbe 2026-03-24 10:02:18,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfbe217544ab5cd4a7e9ca 2026-03-24 10:02:18,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:02:18,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbe217544ab5cd4a7e9ca HTTP/1.1" 200 None 2026-03-24 10:02:18,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:02:18,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:02:18,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:02:18,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfbe217544ab5cd4a7e9ca HTTP/1.1" 200 None 2026-03-24 10:02:18,624 - lava_callback - ERROR - Error processing callback for node 69bfbe217544ab5cd4a7e9ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45618 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:02:32,542 - lava_callback - INFO - Received callback for node 69c260f94a2c5046eb8807b5 INFO: 172.20.0.1:52772 - "POST /node/69c260f94a2c5046eb8807b5 HTTP/1.0" 202 Accepted 2026-03-24 10:02:32,549 - lava_callback - INFO - Starting async job submit for node 69c260f94a2c5046eb8807b5 2026-03-24 10:02:32,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:02:32,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c260f94a2c5046eb8807b5 HTTP/1.1" 200 None 2026-03-24 10:02:32,593 - lava_callback - DEBUG - Processing node 69c260f94a2c5046eb8807b5: baseline-arm64 2026-03-24 10:02:32,733 - lava_callback - WARNING - Failed to upload log 2026-03-24 10:02:32,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:02:33,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:02:33,016 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c260f94a2c5046eb8807b5/lava-logs.yaml 2026-03-24 10:02:33,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:02:33,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:02:33,180 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c260f94a2c5046eb8807b5/lava_callback.json.gz 2026-03-24 10:02:33,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:02:33,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c260f94a2c5046eb8807b5 HTTP/1.1" 200 None 2026-03-24 10:02:33,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:02:33,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 10:02:33,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:02:33,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c260f94a2c5046eb8807b5 HTTP/1.1" 200 None 2026-03-24 10:02:33,339 - lava_callback - INFO - Completed processing callback for node 69c260f94a2c5046eb8807b5 INFO: 172.20.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36930 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:02:55,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:02:55,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48740 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:04:00,354 - lava_callback - INFO - Received callback for node 69c25ce24a2c5046eb88070b INFO: 172.20.0.1:51242 - "POST /node/69c25ce24a2c5046eb88070b HTTP/1.0" 202 Accepted 2026-03-24 10:04:00,362 - lava_callback - INFO - Starting async job submit for node 69c25ce24a2c5046eb88070b 2026-03-24 10:04:00,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:04:00,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25ce24a2c5046eb88070b HTTP/1.1" 200 None 2026-03-24 10:04:00,421 - lava_callback - DEBUG - Processing node 69c25ce24a2c5046eb88070b: baseline-nfs-arm64 2026-03-24 10:04:01,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:04:01,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:04:01,619 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c25ce24a2c5046eb88070b/log.txt.gz 2026-03-24 10:04:02,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:04:02,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:04:02,446 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c25ce24a2c5046eb88070b/lava-logs.yaml 2026-03-24 10:04:02,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:04:02,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:04:02,687 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c25ce24a2c5046eb88070b/lava_callback.json.gz DEBUG: Setup failed for 69c25ce24a2c5046eb88070b. Transitting job node result: incomplete -> fail 2026-03-24 10:04:02,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:04:02,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25ce24a2c5046eb88070b HTTP/1.1" 200 None 2026-03-24 10:04:02,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:04:02,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 10:04:02,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:04:02,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25ce24a2c5046eb88070b HTTP/1.1" 200 None 2026-03-24 10:04:02,942 - lava_callback - INFO - Completed processing callback for node 69c25ce24a2c5046eb88070b INFO: 172.20.0.1:51254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51856 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:04:25,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:04:25,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34908 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:06:23,557 - lava_callback - INFO - Received callback for node 69bfbf157544ab5cd4a7ea09 INFO: 172.20.0.1:35228 - "POST /node/69bfbf157544ab5cd4a7ea09 HTTP/1.0" 202 Accepted 2026-03-24 10:06:23,562 - lava_callback - INFO - Starting async job submit for node 69bfbf157544ab5cd4a7ea09 2026-03-24 10:06:23,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:06:23,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbf157544ab5cd4a7ea09 HTTP/1.1" 200 None 2026-03-24 10:06:23,583 - lava_callback - DEBUG - Processing node 69bfbf157544ab5cd4a7ea09: baseline-arm64-baylibre 2026-03-24 10:06:23,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfbf157544ab5cd4a7ea09 2026-03-24 10:06:23,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:06:23,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbf157544ab5cd4a7ea09 HTTP/1.1" 200 None 2026-03-24 10:06:23,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:06:23,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:06:23,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:06:23,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfbf157544ab5cd4a7ea09 HTTP/1.1" 200 None 2026-03-24 10:06:23,641 - lava_callback - ERROR - Error processing callback for node 69bfbf157544ab5cd4a7ea09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52884 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:08:10,910 - lava_callback - INFO - Received callback for node 69c25e6e4a2c5046eb880744 INFO: 172.20.0.1:45678 - "POST /node/69c25e6e4a2c5046eb880744 HTTP/1.0" 202 Accepted 2026-03-24 10:08:10,917 - lava_callback - INFO - Starting async job submit for node 69c25e6e4a2c5046eb880744 2026-03-24 10:08:10,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:08:10,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25e6e4a2c5046eb880744 HTTP/1.1" 200 None 2026-03-24 10:08:10,996 - lava_callback - DEBUG - Processing node 69c25e6e4a2c5046eb880744: ltp-smoke 2026-03-24 10:08:11,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:08:11,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:08:11,669 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c25e6e4a2c5046eb880744/log.txt.gz 2026-03-24 10:08:11,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:08:12,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:08:12,216 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c25e6e4a2c5046eb880744/lava-logs.yaml 2026-03-24 10:08:12,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:08:12,386 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:08:12,388 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c25e6e4a2c5046eb880744/lava_callback.json.gz 2026-03-24 10:08:12,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:08:12,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c25e6e4a2c5046eb880744 HTTP/1.1" 200 None 2026-03-24 10:08:12,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:08:12,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248de4a2c5046eb87f896 HTTP/1.1" 200 None 2026-03-24 10:08:12,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:08:12,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c25e6e4a2c5046eb880744 HTTP/1.1" 200 None 2026-03-24 10:08:12,684 - lava_callback - INFO - Completed processing callback for node 69c25e6e4a2c5046eb880744 INFO: 172.20.0.1:45684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45702 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:08:23,497 - lava_callback - INFO - Received callback for node 69bfbf827544ab5cd4a7ea25 INFO: 172.20.0.1:53538 - "POST /node/69bfbf827544ab5cd4a7ea25 HTTP/1.0" 202 Accepted 2026-03-24 10:08:23,502 - lava_callback - INFO - Starting async job submit for node 69bfbf827544ab5cd4a7ea25 2026-03-24 10:08:23,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:08:23,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbf827544ab5cd4a7ea25 HTTP/1.1" 200 None 2026-03-24 10:08:23,519 - lava_callback - DEBUG - Processing node 69bfbf827544ab5cd4a7ea25: baseline-arm64-baylibre 2026-03-24 10:08:23,520 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfbf827544ab5cd4a7ea25 2026-03-24 10:08:23,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:08:23,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfbf827544ab5cd4a7ea25 HTTP/1.1" 200 None 2026-03-24 10:08:23,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:08:23,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:08:23,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:08:23,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfbf827544ab5cd4a7ea25 HTTP/1.1" 200 None 2026-03-24 10:08:23,580 - lava_callback - ERROR - Error processing callback for node 69bfbf827544ab5cd4a7ea25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 10:08:25,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:08:25,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40928 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:12:03,743 - lava_callback - INFO - Received callback for node 69bfc05e7544ab5cd4a7ea50 INFO: 172.20.0.1:45176 - "POST /node/69bfc05e7544ab5cd4a7ea50 HTTP/1.0" 202 Accepted 2026-03-24 10:12:03,749 - lava_callback - INFO - Starting async job submit for node 69bfc05e7544ab5cd4a7ea50 2026-03-24 10:12:03,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:12:03,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc05e7544ab5cd4a7ea50 HTTP/1.1" 200 None 2026-03-24 10:12:03,774 - lava_callback - DEBUG - Processing node 69bfc05e7544ab5cd4a7ea50: baseline-arm64-baylibre 2026-03-24 10:12:03,775 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc05e7544ab5cd4a7ea50 2026-03-24 10:12:03,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:12:03,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc05e7544ab5cd4a7ea50 HTTP/1.1" 200 None 2026-03-24 10:12:03,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:12:03,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:12:03,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:12:03,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc05e7544ab5cd4a7ea50 HTTP/1.1" 200 None 2026-03-24 10:12:03,856 - lava_callback - ERROR - Error processing callback for node 69bfc05e7544ab5cd4a7ea50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55006 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:13:03,691 - lava_callback - INFO - Received callback for node 69bfc0aa7544ab5cd4a7ea61 INFO: 172.20.0.1:54124 - "POST /node/69bfc0aa7544ab5cd4a7ea61 HTTP/1.0" 202 Accepted 2026-03-24 10:13:03,696 - lava_callback - INFO - Starting async job submit for node 69bfc0aa7544ab5cd4a7ea61 2026-03-24 10:13:03,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:13:03,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc0aa7544ab5cd4a7ea61 HTTP/1.1" 200 None 2026-03-24 10:13:03,724 - lava_callback - DEBUG - Processing node 69bfc0aa7544ab5cd4a7ea61: baseline-arm64-baylibre 2026-03-24 10:13:03,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc0aa7544ab5cd4a7ea61 2026-03-24 10:13:03,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:13:03,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc0aa7544ab5cd4a7ea61 HTTP/1.1" 200 None 2026-03-24 10:13:03,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:13:03,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:13:03,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:13:03,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc0aa7544ab5cd4a7ea61 HTTP/1.1" 200 None 2026-03-24 10:13:03,781 - lava_callback - ERROR - Error processing callback for node 69bfc0aa7544ab5cd4a7ea61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46852 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:13:23,253 - lava_callback - INFO - Received callback for node 69c261514a2c5046eb8807c2 INFO: 172.20.0.1:43420 - "POST /node/69c261514a2c5046eb8807c2 HTTP/1.0" 202 Accepted 2026-03-24 10:13:23,259 - lava_callback - INFO - Starting async job submit for node 69c261514a2c5046eb8807c2 2026-03-24 10:13:23,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:13:23,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c261514a2c5046eb8807c2 HTTP/1.1" 200 None 2026-03-24 10:13:23,310 - lava_callback - DEBUG - Processing node 69c261514a2c5046eb8807c2: baseline-arm64 2026-03-24 10:13:23,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:13:23,974 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:13:23,976 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c261514a2c5046eb8807c2/log.txt.gz 2026-03-24 10:13:24,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:13:24,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:13:24,469 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c261514a2c5046eb8807c2/lava-logs.yaml 2026-03-24 10:13:24,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:13:24,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:13:24,662 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c261514a2c5046eb8807c2/lava_callback.json.gz 2026-03-24 10:13:24,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:13:24,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c261514a2c5046eb8807c2 HTTP/1.1" 200 None 2026-03-24 10:13:24,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:13:24,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 10:13:24,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:13:24,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c261514a2c5046eb8807c2 HTTP/1.1" 200 None 2026-03-24 10:13:24,810 - lava_callback - INFO - Completed processing callback for node 69c261514a2c5046eb8807c2 2026-03-24 10:13:25,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:13:25,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34748 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:14:20,340 - lava_callback - INFO - Received callback for node 69c2623f4a2c5046eb8807ec INFO: 172.20.0.1:51846 - "POST /node/69c2623f4a2c5046eb8807ec HTTP/1.0" 202 Accepted 2026-03-24 10:14:20,347 - lava_callback - INFO - Starting async job submit for node 69c2623f4a2c5046eb8807ec 2026-03-24 10:14:20,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:14:20,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2623f4a2c5046eb8807ec HTTP/1.1" 200 None 2026-03-24 10:14:20,421 - lava_callback - DEBUG - Processing node 69c2623f4a2c5046eb8807ec: baseline-arm64 2026-03-24 10:14:20,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:14:20,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:14:20,997 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2623f4a2c5046eb8807ec/log.txt.gz 2026-03-24 10:14:21,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:14:21,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:14:21,404 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2623f4a2c5046eb8807ec/lava-logs.yaml 2026-03-24 10:14:21,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:14:21,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:14:21,585 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c2623f4a2c5046eb8807ec/lava_callback.json.gz DEBUG: Setup failed for 69c2623f4a2c5046eb8807ec. Transitting job node result: incomplete -> fail 2026-03-24 10:14:21,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:14:21,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2623f4a2c5046eb8807ec HTTP/1.1" 200 None 2026-03-24 10:14:21,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:14:21,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 10:14:21,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:14:21,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2623f4a2c5046eb8807ec HTTP/1.1" 200 None 2026-03-24 10:14:21,811 - lava_callback - INFO - Completed processing callback for node 69c2623f4a2c5046eb8807ec 2026-03-24 10:14:23,749 - lava_callback - INFO - Received callback for node 69bfc0f47544ab5cd4a7ea75 INFO: 172.20.0.1:51852 - "POST /node/69bfc0f47544ab5cd4a7ea75 HTTP/1.0" 202 Accepted 2026-03-24 10:14:23,755 - lava_callback - INFO - Starting async job submit for node 69bfc0f47544ab5cd4a7ea75 2026-03-24 10:14:23,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:14:23,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc0f47544ab5cd4a7ea75 HTTP/1.1" 200 None 2026-03-24 10:14:23,771 - lava_callback - DEBUG - Processing node 69bfc0f47544ab5cd4a7ea75: baseline-arm64-baylibre 2026-03-24 10:14:23,771 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc0f47544ab5cd4a7ea75 2026-03-24 10:14:23,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:14:23,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc0f47544ab5cd4a7ea75 HTTP/1.1" 200 None 2026-03-24 10:14:23,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:14:23,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:14:23,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:14:23,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc0f47544ab5cd4a7ea75 HTTP/1.1" 200 None 2026-03-24 10:14:23,836 - lava_callback - ERROR - Error processing callback for node 69bfc0f47544ab5cd4a7ea75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 10:14:25,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:14:25,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33554 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:15:19,133 - lava_callback - INFO - Received callback for node 69c261cd4a2c5046eb8807d9 INFO: 172.20.0.1:33568 - "POST /node/69c261cd4a2c5046eb8807d9 HTTP/1.0" 202 Accepted 2026-03-24 10:15:19,139 - lava_callback - INFO - Starting async job submit for node 69c261cd4a2c5046eb8807d9 2026-03-24 10:15:19,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:15:19,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c261cd4a2c5046eb8807d9 HTTP/1.1" 200 None 2026-03-24 10:15:19,180 - lava_callback - DEBUG - Processing node 69c261cd4a2c5046eb8807d9: baseline-nfs-arm64 2026-03-24 10:15:19,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:15:19,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:15:19,860 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c261cd4a2c5046eb8807d9/log.txt.gz 2026-03-24 10:15:20,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:15:20,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:15:20,374 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c261cd4a2c5046eb8807d9/lava-logs.yaml 2026-03-24 10:15:20,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:15:20,540 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:15:20,542 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c261cd4a2c5046eb8807d9/lava_callback.json.gz 2026-03-24 10:15:20,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:15:20,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c261cd4a2c5046eb8807d9 HTTP/1.1" 200 None 2026-03-24 10:15:20,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:15:20,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 10:15:20,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:15:20,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c261cd4a2c5046eb8807d9 HTTP/1.1" 200 None 2026-03-24 10:15:20,738 - lava_callback - INFO - Completed processing callback for node 69c261cd4a2c5046eb8807d9 2026-03-24 10:15:25,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:15:25,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34650 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:15:43,848 - lava_callback - INFO - Received callback for node 69bfc13c7544ab5cd4a7ea83 INFO: 172.20.0.1:51048 - "POST /node/69bfc13c7544ab5cd4a7ea83 HTTP/1.0" 202 Accepted 2026-03-24 10:15:43,853 - lava_callback - INFO - Starting async job submit for node 69bfc13c7544ab5cd4a7ea83 2026-03-24 10:15:43,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:15:43,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc13c7544ab5cd4a7ea83 HTTP/1.1" 200 None 2026-03-24 10:15:43,871 - lava_callback - DEBUG - Processing node 69bfc13c7544ab5cd4a7ea83: baseline-arm64-baylibre 2026-03-24 10:15:43,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc13c7544ab5cd4a7ea83 2026-03-24 10:15:43,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:15:43,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc13c7544ab5cd4a7ea83 HTTP/1.1" 200 None 2026-03-24 10:15:43,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:15:43,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:15:43,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:15:43,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc13c7544ab5cd4a7ea83 HTTP/1.1" 200 None 2026-03-24 10:15:43,955 - lava_callback - ERROR - Error processing callback for node 69bfc13c7544ab5cd4a7ea83: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56018 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:18:44,120 - lava_callback - INFO - Received callback for node 69bfc1f57544ab5cd4a7eaac INFO: 172.20.0.1:43642 - "POST /node/69bfc1f57544ab5cd4a7eaac HTTP/1.0" 202 Accepted 2026-03-24 10:18:44,125 - lava_callback - INFO - Starting async job submit for node 69bfc1f57544ab5cd4a7eaac 2026-03-24 10:18:44,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:18:44,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc1f57544ab5cd4a7eaac HTTP/1.1" 200 None 2026-03-24 10:18:44,142 - lava_callback - DEBUG - Processing node 69bfc1f57544ab5cd4a7eaac: baseline-arm64-baylibre 2026-03-24 10:18:44,142 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc1f57544ab5cd4a7eaac 2026-03-24 10:18:44,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:18:44,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc1f57544ab5cd4a7eaac HTTP/1.1" 200 None 2026-03-24 10:18:44,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:18:44,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:18:44,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:18:44,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc1f57544ab5cd4a7eaac HTTP/1.1" 200 None 2026-03-24 10:18:44,214 - lava_callback - ERROR - Error processing callback for node 69bfc1f57544ab5cd4a7eaac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50192 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:19:59,174 - lava_callback - INFO - Received callback for node 69bfc2477544ab5cd4a7eabd INFO: 172.20.0.1:50206 - "POST /node/69bfc2477544ab5cd4a7eabd HTTP/1.0" 202 Accepted 2026-03-24 10:19:59,180 - lava_callback - INFO - Starting async job submit for node 69bfc2477544ab5cd4a7eabd 2026-03-24 10:19:59,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:19:59,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc2477544ab5cd4a7eabd HTTP/1.1" 200 None 2026-03-24 10:19:59,196 - lava_callback - DEBUG - Processing node 69bfc2477544ab5cd4a7eabd: baseline-arm64-clabbe 2026-03-24 10:19:59,196 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc2477544ab5cd4a7eabd 2026-03-24 10:19:59,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:19:59,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc2477544ab5cd4a7eabd HTTP/1.1" 200 None 2026-03-24 10:19:59,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:19:59,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:19:59,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:19:59,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc2477544ab5cd4a7eabd HTTP/1.1" 200 None 2026-03-24 10:19:59,283 - lava_callback - ERROR - Error processing callback for node 69bfc2477544ab5cd4a7eabd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59126 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:21:06,247 - lava_callback - INFO - Received callback for node 69c2619f4a2c5046eb8807cc INFO: 172.20.0.1:51774 - "POST /node/69c2619f4a2c5046eb8807cc HTTP/1.0" 202 Accepted 2026-03-24 10:21:06,254 - lava_callback - INFO - Starting async job submit for node 69c2619f4a2c5046eb8807cc 2026-03-24 10:21:06,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:21:06,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2619f4a2c5046eb8807cc HTTP/1.1" 200 None 2026-03-24 10:21:06,336 - lava_callback - DEBUG - Processing node 69c2619f4a2c5046eb8807cc: baseline-arm INFO: 172.20.0.1:51778 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:21:06,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:21:07,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:21:07,112 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c2619f4a2c5046eb8807cc/log.txt.gz 2026-03-24 10:21:07,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:21:07,796 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:21:07,798 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c2619f4a2c5046eb8807cc/lava-logs.yaml 2026-03-24 10:21:07,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:21:07,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:21:07,996 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c2619f4a2c5046eb8807cc/lava_callback.json.gz DEBUG: Setup failed for 69c2619f4a2c5046eb8807cc. Transitting job node result: incomplete -> fail 2026-03-24 10:21:08,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:21:08,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2619f4a2c5046eb8807cc HTTP/1.1" 200 None 2026-03-24 10:21:08,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:21:08,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 10:21:08,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:21:08,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2619f4a2c5046eb8807cc HTTP/1.1" 200 None 2026-03-24 10:21:08,271 - lava_callback - INFO - Completed processing callback for node 69c2619f4a2c5046eb8807cc INFO: 172.20.0.1:44818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44838 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:21:19,285 - lava_callback - INFO - Received callback for node 69bfc2937544ab5cd4a7eac7 INFO: 172.20.0.1:44844 - "POST /node/69bfc2937544ab5cd4a7eac7 HTTP/1.0" 202 Accepted 2026-03-24 10:21:19,290 - lava_callback - INFO - Starting async job submit for node 69bfc2937544ab5cd4a7eac7 2026-03-24 10:21:19,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:21:19,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc2937544ab5cd4a7eac7 HTTP/1.1" 200 None 2026-03-24 10:21:19,306 - lava_callback - DEBUG - Processing node 69bfc2937544ab5cd4a7eac7: baseline-arm64-clabbe 2026-03-24 10:21:19,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc2937544ab5cd4a7eac7 2026-03-24 10:21:19,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:21:19,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc2937544ab5cd4a7eac7 HTTP/1.1" 200 None 2026-03-24 10:21:19,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:21:19,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:21:19,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:21:19,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc2937544ab5cd4a7eac7 HTTP/1.1" 200 None 2026-03-24 10:21:19,366 - lava_callback - ERROR - Error processing callback for node 69bfc2937544ab5cd4a7eac7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 10:21:25,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:21:25,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39284 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:23:44,338 - lava_callback - INFO - Received callback for node 69bfc3227544ab5cd4a7eadf INFO: 172.20.0.1:50996 - "POST /node/69bfc3227544ab5cd4a7eadf HTTP/1.0" 202 Accepted 2026-03-24 10:23:44,344 - lava_callback - INFO - Starting async job submit for node 69bfc3227544ab5cd4a7eadf 2026-03-24 10:23:44,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:23:44,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc3227544ab5cd4a7eadf HTTP/1.1" 200 None 2026-03-24 10:23:44,361 - lava_callback - DEBUG - Processing node 69bfc3227544ab5cd4a7eadf: baseline-arm64-baylibre 2026-03-24 10:23:44,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc3227544ab5cd4a7eadf 2026-03-24 10:23:44,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:23:44,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc3227544ab5cd4a7eadf HTTP/1.1" 200 None 2026-03-24 10:23:44,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:23:44,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:23:44,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:23:44,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc3227544ab5cd4a7eadf HTTP/1.1" 200 None 2026-03-24 10:23:44,470 - lava_callback - ERROR - Error processing callback for node 69bfc3227544ab5cd4a7eadf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52496 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:24:59,382 - lava_callback - INFO - Received callback for node 69bfc3737544ab5cd4a7eaf0 INFO: 172.20.0.1:52502 - "POST /node/69bfc3737544ab5cd4a7eaf0 HTTP/1.0" 202 Accepted 2026-03-24 10:24:59,387 - lava_callback - INFO - Starting async job submit for node 69bfc3737544ab5cd4a7eaf0 2026-03-24 10:24:59,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:24:59,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc3737544ab5cd4a7eaf0 HTTP/1.1" 200 None 2026-03-24 10:24:59,414 - lava_callback - DEBUG - Processing node 69bfc3737544ab5cd4a7eaf0: baseline-arm64-clabbe 2026-03-24 10:24:59,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc3737544ab5cd4a7eaf0 2026-03-24 10:24:59,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:24:59,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc3737544ab5cd4a7eaf0 HTTP/1.1" 200 None 2026-03-24 10:24:59,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:24:59,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:24:59,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:24:59,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc3737544ab5cd4a7eaf0 HTTP/1.1" 200 None 2026-03-24 10:24:59,475 - lava_callback - ERROR - Error processing callback for node 69bfc3737544ab5cd4a7eaf0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33224 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:25:23,628 - lava_callback - INFO - Received callback for node 69c263ca4a2c5046eb88080c INFO: 172.20.0.1:50562 - "POST /node/69c263ca4a2c5046eb88080c HTTP/1.0" 202 Accepted 2026-03-24 10:25:23,635 - lava_callback - INFO - Starting async job submit for node 69c263ca4a2c5046eb88080c 2026-03-24 10:25:23,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:25:23,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c263ca4a2c5046eb88080c HTTP/1.1" 200 None 2026-03-24 10:25:23,676 - lava_callback - DEBUG - Processing node 69c263ca4a2c5046eb88080c: baseline-arm64 2026-03-24 10:25:24,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:25:24,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:25:24,352 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c263ca4a2c5046eb88080c/log.txt.gz 2026-03-24 10:25:24,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:25:24,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:25:24,839 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c263ca4a2c5046eb88080c/lava-logs.yaml 2026-03-24 10:25:24,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:25:25,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:25:25,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c263ca4a2c5046eb88080c/lava_callback.json.gz 2026-03-24 10:25:25,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:25:25,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c263ca4a2c5046eb88080c HTTP/1.1" 200 None 2026-03-24 10:25:25,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:25:25,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 10:25:25,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:25:25,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c263ca4a2c5046eb88080c HTTP/1.1" 200 None 2026-03-24 10:25:25,175 - lava_callback - INFO - Completed processing callback for node 69c263ca4a2c5046eb88080c 2026-03-24 10:25:25,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:25:25,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58734 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:25:43,709 - lava_callback - INFO - Received callback for node 69c262204a2c5046eb8807e5 INFO: 172.20.0.1:57636 - "POST /node/69c262204a2c5046eb8807e5 HTTP/1.0" 202 Accepted 2026-03-24 10:25:43,757 - lava_callback - INFO - Starting async job submit for node 69c262204a2c5046eb8807e5 2026-03-24 10:25:43,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:25:43,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c262204a2c5046eb8807e5 HTTP/1.1" 200 None 2026-03-24 10:25:43,797 - lava_callback - DEBUG - Processing node 69c262204a2c5046eb8807e5: baseline-arm-pengutronix 2026-03-24 10:25:43,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:25:44,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:25:44,047 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c262204a2c5046eb8807e5/log.txt.gz 2026-03-24 10:25:44,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:25:44,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:25:44,257 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c262204a2c5046eb8807e5/lava-logs.yaml 2026-03-24 10:25:44,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:25:44,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:25:44,370 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c262204a2c5046eb8807e5/lava_callback.json.gz DEBUG: Setup failed for 69c262204a2c5046eb8807e5. Transitting job node result: incomplete -> fail 2026-03-24 10:25:44,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:25:44,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c262204a2c5046eb8807e5 HTTP/1.1" 200 None 2026-03-24 10:25:44,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:25:44,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 10:25:44,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:25:44,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c262204a2c5046eb8807e5 HTTP/1.1" 200 None 2026-03-24 10:25:44,552 - lava_callback - INFO - Completed processing callback for node 69c262204a2c5046eb8807e5 INFO: 172.20.0.1:57646 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:25:55,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:25:55,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54986 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:26:04,445 - lava_callback - INFO - Received callback for node 69bfc3ba7544ab5cd4a7eafb INFO: 172.20.0.1:54406 - "POST /node/69bfc3ba7544ab5cd4a7eafb HTTP/1.0" 202 Accepted 2026-03-24 10:26:04,451 - lava_callback - INFO - Starting async job submit for node 69bfc3ba7544ab5cd4a7eafb 2026-03-24 10:26:04,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:26:04,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc3ba7544ab5cd4a7eafb HTTP/1.1" 200 None 2026-03-24 10:26:04,474 - lava_callback - DEBUG - Processing node 69bfc3ba7544ab5cd4a7eafb: baseline-arm64-baylibre 2026-03-24 10:26:04,474 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc3ba7544ab5cd4a7eafb 2026-03-24 10:26:04,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:26:04,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc3ba7544ab5cd4a7eafb HTTP/1.1" 200 None 2026-03-24 10:26:04,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:26:04,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:26:04,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:26:04,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc3ba7544ab5cd4a7eafb HTTP/1.1" 200 None 2026-03-24 10:26:04,541 - lava_callback - ERROR - Error processing callback for node 69bfc3ba7544ab5cd4a7eafb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41956 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:26:51,533 - lava_callback - INFO - Received callback for node 69c262db4a2c5046eb880803 INFO: 172.20.0.1:60730 - "POST /node/69c262db4a2c5046eb880803 HTTP/1.0" 202 Accepted 2026-03-24 10:26:51,539 - lava_callback - INFO - Starting async job submit for node 69c262db4a2c5046eb880803 2026-03-24 10:26:51,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:26:51,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c262db4a2c5046eb880803 HTTP/1.1" 200 None 2026-03-24 10:26:51,618 - lava_callback - DEBUG - Processing node 69c262db4a2c5046eb880803: ltp-smoke 2026-03-24 10:26:52,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:26:52,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:26:52,253 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c262db4a2c5046eb880803/log.txt.gz 2026-03-24 10:26:52,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:26:52,744 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:26:52,746 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c262db4a2c5046eb880803/lava-logs.yaml 2026-03-24 10:26:52,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:26:52,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:26:52,921 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c262db4a2c5046eb880803/lava_callback.json.gz 2026-03-24 10:26:53,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:26:53,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c262db4a2c5046eb880803 HTTP/1.1" 200 None 2026-03-24 10:26:53,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:26:53,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248de4a2c5046eb87f896 HTTP/1.1" 200 None 2026-03-24 10:26:53,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:26:53,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c262db4a2c5046eb880803 HTTP/1.1" 200 None 2026-03-24 10:26:53,217 - lava_callback - INFO - Completed processing callback for node 69c262db4a2c5046eb880803 2026-03-24 10:26:55,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:26:55,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41926 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:27:39,509 - lava_callback - INFO - Received callback for node 69bfc40c7544ab5cd4a7eb0c INFO: 172.20.0.1:41932 - "POST /node/69bfc40c7544ab5cd4a7eb0c HTTP/1.0" 202 Accepted 2026-03-24 10:27:39,514 - lava_callback - INFO - Starting async job submit for node 69bfc40c7544ab5cd4a7eb0c 2026-03-24 10:27:39,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:27:39,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc40c7544ab5cd4a7eb0c HTTP/1.1" 200 None 2026-03-24 10:27:39,531 - lava_callback - DEBUG - Processing node 69bfc40c7544ab5cd4a7eb0c: baseline-arm64-clabbe 2026-03-24 10:27:39,531 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc40c7544ab5cd4a7eb0c 2026-03-24 10:27:39,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:27:39,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc40c7544ab5cd4a7eb0c HTTP/1.1" 200 None 2026-03-24 10:27:39,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:27:39,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:27:39,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:27:39,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc40c7544ab5cd4a7eb0c HTTP/1.1" 200 None 2026-03-24 10:27:39,612 - lava_callback - ERROR - Error processing callback for node 69bfc40c7544ab5cd4a7eb0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50230 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:28:20,796 - lava_callback - INFO - Received callback for node 69c262904a2c5046eb8807f6 INFO: 172.20.0.1:47596 - "POST /node/69c262904a2c5046eb8807f6 HTTP/1.0" 202 Accepted 2026-03-24 10:28:20,803 - lava_callback - INFO - Starting async job submit for node 69c262904a2c5046eb8807f6 2026-03-24 10:28:20,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:28:20,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c262904a2c5046eb8807f6 HTTP/1.1" 200 None 2026-03-24 10:28:20,861 - lava_callback - DEBUG - Processing node 69c262904a2c5046eb8807f6: baseline-nfs-arm64 2026-03-24 10:28:21,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:28:22,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:28:22,023 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c262904a2c5046eb8807f6/log.txt.gz 2026-03-24 10:28:22,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:28:22,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:28:22,870 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c262904a2c5046eb8807f6/lava-logs.yaml 2026-03-24 10:28:22,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:28:23,072 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:28:23,074 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c262904a2c5046eb8807f6/lava_callback.json.gz DEBUG: Setup failed for 69c262904a2c5046eb8807f6. Transitting job node result: incomplete -> fail 2026-03-24 10:28:23,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:28:23,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c262904a2c5046eb8807f6 HTTP/1.1" 200 None 2026-03-24 10:28:23,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:28:23,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 10:28:23,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:28:23,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c262904a2c5046eb8807f6 HTTP/1.1" 200 None 2026-03-24 10:28:23,306 - lava_callback - INFO - Completed processing callback for node 69c262904a2c5046eb8807f6 2026-03-24 10:28:25,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:28:25,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42236 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:28:44,678 - lava_callback - INFO - Received callback for node 69bfc4527544ab5cd4a7eb15 INFO: 172.20.0.1:57008 - "POST /node/69bfc4527544ab5cd4a7eb15 HTTP/1.0" 202 Accepted 2026-03-24 10:28:44,684 - lava_callback - INFO - Starting async job submit for node 69bfc4527544ab5cd4a7eb15 2026-03-24 10:28:44,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:28:44,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc4527544ab5cd4a7eb15 HTTP/1.1" 200 None 2026-03-24 10:28:44,700 - lava_callback - DEBUG - Processing node 69bfc4527544ab5cd4a7eb15: baseline-arm64-baylibre 2026-03-24 10:28:44,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc4527544ab5cd4a7eb15 2026-03-24 10:28:44,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:28:44,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc4527544ab5cd4a7eb15 HTTP/1.1" 200 None 2026-03-24 10:28:44,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:28:44,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:28:44,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:28:44,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc4527544ab5cd4a7eb15 HTTP/1.1" 200 None 2026-03-24 10:28:44,761 - lava_callback - ERROR - Error processing callback for node 69bfc4527544ab5cd4a7eb15: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44808 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:29:59,580 - lava_callback - INFO - Received callback for node 69bfc4a17544ab5cd4a7eb29 INFO: 172.20.0.1:44824 - "POST /node/69bfc4a17544ab5cd4a7eb29 HTTP/1.0" 202 Accepted 2026-03-24 10:29:59,585 - lava_callback - INFO - Starting async job submit for node 69bfc4a17544ab5cd4a7eb29 2026-03-24 10:29:59,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:29:59,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc4a17544ab5cd4a7eb29 HTTP/1.1" 200 None 2026-03-24 10:29:59,602 - lava_callback - DEBUG - Processing node 69bfc4a17544ab5cd4a7eb29: baseline-arm64-clabbe 2026-03-24 10:29:59,602 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc4a17544ab5cd4a7eb29 2026-03-24 10:29:59,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:29:59,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc4a17544ab5cd4a7eb29 HTTP/1.1" 200 None 2026-03-24 10:29:59,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:29:59,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:29:59,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:29:59,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc4a17544ab5cd4a7eb29 HTTP/1.1" 200 None 2026-03-24 10:29:59,660 - lava_callback - ERROR - Error processing callback for node 69bfc4a17544ab5cd4a7eb29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57162 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:31:19,638 - lava_callback - INFO - Received callback for node 69bfc4ed7544ab5cd4a7eb36 INFO: 172.20.0.1:57168 - "POST /node/69bfc4ed7544ab5cd4a7eb36 HTTP/1.0" 202 Accepted 2026-03-24 10:31:19,644 - lava_callback - INFO - Starting async job submit for node 69bfc4ed7544ab5cd4a7eb36 2026-03-24 10:31:19,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:31:19,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc4ed7544ab5cd4a7eb36 HTTP/1.1" 200 None 2026-03-24 10:31:19,669 - lava_callback - DEBUG - Processing node 69bfc4ed7544ab5cd4a7eb36: baseline-arm64-clabbe 2026-03-24 10:31:19,670 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc4ed7544ab5cd4a7eb36 2026-03-24 10:31:19,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:31:19,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc4ed7544ab5cd4a7eb36 HTTP/1.1" 200 None 2026-03-24 10:31:19,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:31:19,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:31:19,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:31:19,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc4ed7544ab5cd4a7eb36 HTTP/1.1" 200 None 2026-03-24 10:31:19,759 - lava_callback - ERROR - Error processing callback for node 69bfc4ed7544ab5cd4a7eb36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60630 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:32:39,690 - lava_callback - INFO - Received callback for node 69bfc5397544ab5cd4a7eb47 INFO: 172.20.0.1:60636 - "POST /node/69bfc5397544ab5cd4a7eb47 HTTP/1.0" 202 Accepted 2026-03-24 10:32:39,696 - lava_callback - INFO - Starting async job submit for node 69bfc5397544ab5cd4a7eb47 2026-03-24 10:32:39,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:32:39,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc5397544ab5cd4a7eb47 HTTP/1.1" 200 None 2026-03-24 10:32:39,730 - lava_callback - DEBUG - Processing node 69bfc5397544ab5cd4a7eb47: baseline-arm64-clabbe 2026-03-24 10:32:39,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc5397544ab5cd4a7eb47 2026-03-24 10:32:39,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:32:39,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc5397544ab5cd4a7eb47 HTTP/1.1" 200 None 2026-03-24 10:32:39,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:32:39,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:32:39,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:32:39,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc5397544ab5cd4a7eb47 HTTP/1.1" 200 None 2026-03-24 10:32:39,816 - lava_callback - ERROR - Error processing callback for node 69bfc5397544ab5cd4a7eb47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39262 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:33:59,747 - lava_callback - INFO - Received callback for node 69bfc5837544ab5cd4a7eb54 INFO: 172.20.0.1:57374 - "POST /node/69bfc5837544ab5cd4a7eb54 HTTP/1.0" 202 Accepted 2026-03-24 10:33:59,752 - lava_callback - INFO - Starting async job submit for node 69bfc5837544ab5cd4a7eb54 2026-03-24 10:33:59,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:33:59,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc5837544ab5cd4a7eb54 HTTP/1.1" 200 None 2026-03-24 10:33:59,786 - lava_callback - DEBUG - Processing node 69bfc5837544ab5cd4a7eb54: baseline-arm64-clabbe 2026-03-24 10:33:59,786 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc5837544ab5cd4a7eb54 2026-03-24 10:33:59,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:33:59,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc5837544ab5cd4a7eb54 HTTP/1.1" 200 None 2026-03-24 10:33:59,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:33:59,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:33:59,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:33:59,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc5837544ab5cd4a7eb54 HTTP/1.1" 200 None 2026-03-24 10:33:59,883 - lava_callback - ERROR - Error processing callback for node 69bfc5837544ab5cd4a7eb54: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40572 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:34:59,871 - lava_callback - INFO - Received callback for node 69bfc5ce7544ab5cd4a7eb6f INFO: 172.20.0.1:57562 - "POST /node/69bfc5ce7544ab5cd4a7eb6f HTTP/1.0" 202 Accepted 2026-03-24 10:34:59,876 - lava_callback - INFO - Starting async job submit for node 69bfc5ce7544ab5cd4a7eb6f 2026-03-24 10:34:59,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:34:59,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc5ce7544ab5cd4a7eb6f HTTP/1.1" 200 None 2026-03-24 10:34:59,893 - lava_callback - DEBUG - Processing node 69bfc5ce7544ab5cd4a7eb6f: baseline-arm64-clabbe 2026-03-24 10:34:59,893 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc5ce7544ab5cd4a7eb6f 2026-03-24 10:34:59,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:34:59,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc5ce7544ab5cd4a7eb6f HTTP/1.1" 200 None 2026-03-24 10:34:59,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:34:59,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:34:59,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:34:59,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc5ce7544ab5cd4a7eb6f HTTP/1.1" 200 None 2026-03-24 10:34:59,983 - lava_callback - ERROR - Error processing callback for node 69bfc5ce7544ab5cd4a7eb6f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56452 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:36:19,897 - lava_callback - INFO - Received callback for node 69bfc61a7544ab5cd4a7eb7c INFO: 172.20.0.1:57250 - "POST /node/69bfc61a7544ab5cd4a7eb7c HTTP/1.0" 202 Accepted 2026-03-24 10:36:19,903 - lava_callback - INFO - Starting async job submit for node 69bfc61a7544ab5cd4a7eb7c 2026-03-24 10:36:19,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:36:19,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc61a7544ab5cd4a7eb7c HTTP/1.1" 200 None 2026-03-24 10:36:19,924 - lava_callback - DEBUG - Processing node 69bfc61a7544ab5cd4a7eb7c: baseline-arm64-clabbe 2026-03-24 10:36:19,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc61a7544ab5cd4a7eb7c 2026-03-24 10:36:19,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:36:19,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc61a7544ab5cd4a7eb7c HTTP/1.1" 200 None 2026-03-24 10:36:19,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:36:19,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:36:19,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:36:20,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc61a7544ab5cd4a7eb7c HTTP/1.1" 200 None 2026-03-24 10:36:20,030 - lava_callback - ERROR - Error processing callback for node 69bfc61a7544ab5cd4a7eb7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35354 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:37:39,953 - lava_callback - INFO - Received callback for node 69bfc6657544ab5cd4a7eb95 INFO: 172.20.0.1:40930 - "POST /node/69bfc6657544ab5cd4a7eb95 HTTP/1.0" 202 Accepted 2026-03-24 10:37:39,958 - lava_callback - INFO - Starting async job submit for node 69bfc6657544ab5cd4a7eb95 2026-03-24 10:37:39,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:37:39,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc6657544ab5cd4a7eb95 HTTP/1.1" 200 None 2026-03-24 10:37:39,975 - lava_callback - DEBUG - Processing node 69bfc6657544ab5cd4a7eb95: baseline-arm64-clabbe 2026-03-24 10:37:39,976 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc6657544ab5cd4a7eb95 2026-03-24 10:37:39,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:37:39,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc6657544ab5cd4a7eb95 HTTP/1.1" 200 None 2026-03-24 10:37:39,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:37:40,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:37:40,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:37:40,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc6657544ab5cd4a7eb95 HTTP/1.1" 200 None 2026-03-24 10:37:40,062 - lava_callback - ERROR - Error processing callback for node 69bfc6657544ab5cd4a7eb95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56094 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:38:46,715 - lava_callback - INFO - Received callback for node 69c265954a2c5046eb88082d INFO: 172.20.0.1:56110 - "POST /node/69c265954a2c5046eb88082d HTTP/1.0" 202 Accepted 2026-03-24 10:38:46,722 - lava_callback - INFO - Starting async job submit for node 69c265954a2c5046eb88082d 2026-03-24 10:38:46,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:38:46,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c265954a2c5046eb88082d HTTP/1.1" 200 None 2026-03-24 10:38:46,813 - lava_callback - DEBUG - Processing node 69c265954a2c5046eb88082d: baseline-arm 2026-03-24 10:38:47,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:38:47,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:38:47,619 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c265954a2c5046eb88082d/log.txt.gz 2026-03-24 10:38:47,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:38:48,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:38:48,225 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c265954a2c5046eb88082d/lava-logs.yaml 2026-03-24 10:38:48,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:38:48,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:38:48,438 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c265954a2c5046eb88082d/lava_callback.json.gz DEBUG: Setup failed for 69c265954a2c5046eb88082d. Transitting job node result: incomplete -> fail 2026-03-24 10:38:48,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:38:48,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c265954a2c5046eb88082d HTTP/1.1" 200 None 2026-03-24 10:38:48,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:38:48,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 10:38:48,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:38:48,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c265954a2c5046eb88082d HTTP/1.1" 200 None 2026-03-24 10:38:48,769 - lava_callback - INFO - Completed processing callback for node 69c265954a2c5046eb88082d 2026-03-24 10:38:55,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:38:55,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48630 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:38:59,998 - lava_callback - INFO - Received callback for node 69bfc6b07544ab5cd4a7eb9e INFO: 172.20.0.1:36062 - "POST /node/69bfc6b07544ab5cd4a7eb9e HTTP/1.0" 202 Accepted 2026-03-24 10:39:00,006 - lava_callback - INFO - Starting async job submit for node 69bfc6b07544ab5cd4a7eb9e 2026-03-24 10:39:00,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:39:00,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc6b07544ab5cd4a7eb9e HTTP/1.1" 200 None 2026-03-24 10:39:00,022 - lava_callback - DEBUG - Processing node 69bfc6b07544ab5cd4a7eb9e: baseline-arm64-clabbe 2026-03-24 10:39:00,023 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc6b07544ab5cd4a7eb9e 2026-03-24 10:39:00,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:39:00,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc6b07544ab5cd4a7eb9e HTTP/1.1" 200 None 2026-03-24 10:39:00,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:39:00,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b9ef90d5731cbd74145fce HTTP/1.1" 200 None 2026-03-24 10:39:00,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:39:00,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc6b07544ab5cd4a7eb9e HTTP/1.1" 200 None 2026-03-24 10:39:00,087 - lava_callback - ERROR - Error processing callback for node 69bfc6b07544ab5cd4a7eb9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41198 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:40:25,287 - lava_callback - INFO - Received callback for node 69bfc7157544ab5cd4a7ebb5 INFO: 172.20.0.1:45670 - "POST /node/69bfc7157544ab5cd4a7ebb5 HTTP/1.0" 202 Accepted 2026-03-24 10:40:25,292 - lava_callback - INFO - Starting async job submit for node 69bfc7157544ab5cd4a7ebb5 2026-03-24 10:40:25,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:40:25,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc7157544ab5cd4a7ebb5 HTTP/1.1" 200 None 2026-03-24 10:40:25,314 - lava_callback - DEBUG - Processing node 69bfc7157544ab5cd4a7ebb5: baseline-arm64-baylibre 2026-03-24 10:40:25,314 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc7157544ab5cd4a7ebb5 2026-03-24 10:40:25,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:40:25,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc7157544ab5cd4a7ebb5 HTTP/1.1" 200 None 2026-03-24 10:40:25,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:40:25,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 10:40:25,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:40:25,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc7157544ab5cd4a7ebb5 HTTP/1.1" 200 None 2026-03-24 10:40:25,366 - lava_callback - ERROR - Error processing callback for node 69bfc7157544ab5cd4a7ebb5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44022 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:45:04,200 - lava_callback - INFO - Received callback for node 69c266b44a2c5046eb880848 INFO: 172.20.0.1:36016 - "POST /node/69c266b44a2c5046eb880848 HTTP/1.0" 202 Accepted 2026-03-24 10:45:04,245 - lava_callback - INFO - Starting async job submit for node 69c266b44a2c5046eb880848 2026-03-24 10:45:04,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:04,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c266b44a2c5046eb880848 HTTP/1.1" 200 None 2026-03-24 10:45:04,287 - lava_callback - DEBUG - Processing node 69c266b44a2c5046eb880848: baseline-arm-pengutronix 2026-03-24 10:45:04,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:45:04,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:45:04,535 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c266b44a2c5046eb880848/log.txt.gz 2026-03-24 10:45:04,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:45:04,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:45:04,809 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c266b44a2c5046eb880848/lava-logs.yaml 2026-03-24 10:45:04,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:45:04,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:45:04,957 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c266b44a2c5046eb880848/lava_callback.json.gz DEBUG: Setup failed for 69c266b44a2c5046eb880848. Transitting job node result: incomplete -> fail 2026-03-24 10:45:05,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:05,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c266b44a2c5046eb880848 HTTP/1.1" 200 None 2026-03-24 10:45:05,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:05,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248d74a2c5046eb87f890 HTTP/1.1" 200 None 2026-03-24 10:45:05,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:05,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c266b44a2c5046eb880848 HTTP/1.1" 200 None 2026-03-24 10:45:05,129 - lava_callback - INFO - Completed processing callback for node 69c266b44a2c5046eb880848 INFO: 172.20.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43188 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:45:25,493 - lava_callback - INFO - Received callback for node 69bfc8407544ab5cd4a7ebe8 INFO: 172.20.0.1:38722 - "POST /node/69bfc8407544ab5cd4a7ebe8 HTTP/1.0" 202 Accepted 2026-03-24 10:45:25,498 - lava_callback - INFO - Starting async job submit for node 69bfc8407544ab5cd4a7ebe8 2026-03-24 10:45:25,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:25,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc8407544ab5cd4a7ebe8 HTTP/1.1" 200 None 2026-03-24 10:45:25,520 - lava_callback - DEBUG - Processing node 69bfc8407544ab5cd4a7ebe8: baseline-arm64-baylibre 2026-03-24 10:45:25,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc8407544ab5cd4a7ebe8 2026-03-24 10:45:25,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:25,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc8407544ab5cd4a7ebe8 HTTP/1.1" 200 None 2026-03-24 10:45:25,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:25,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 10:45:25,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:25,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc8407544ab5cd4a7ebe8 HTTP/1.1" 200 None 2026-03-24 10:45:25,604 - lava_callback - ERROR - Error processing callback for node 69bfc8407544ab5cd4a7ebe8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-24 10:45:25,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:25,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38730 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:45:35,998 - lava_callback - INFO - Received callback for node 69c264434a2c5046eb88081e INFO: 172.20.0.1:60982 - "POST /node/69c264434a2c5046eb88081e HTTP/1.0" 202 Accepted 2026-03-24 10:45:36,005 - lava_callback - INFO - Starting async job submit for node 69c264434a2c5046eb88081e 2026-03-24 10:45:36,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:36,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c264434a2c5046eb88081e HTTP/1.1" 200 None 2026-03-24 10:45:36,048 - lava_callback - DEBUG - Processing node 69c264434a2c5046eb88081e: baseline-nfs-arm64 INFO: 172.20.0.1:60986 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:45:36,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:45:36,722 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:45:36,724 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c264434a2c5046eb88081e/log.txt.gz 2026-03-24 10:45:36,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:45:37,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:45:37,188 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c264434a2c5046eb88081e/lava-logs.yaml 2026-03-24 10:45:37,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:45:37,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:45:37,362 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c264434a2c5046eb88081e/lava_callback.json.gz 2026-03-24 10:45:37,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:37,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c264434a2c5046eb88081e HTTP/1.1" 200 None 2026-03-24 10:45:37,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:37,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 10:45:37,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:37,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c264434a2c5046eb88081e HTTP/1.1" 200 None 2026-03-24 10:45:37,539 - lava_callback - INFO - Completed processing callback for node 69c264434a2c5046eb88081e INFO: 172.20.0.1:53916 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:45:55,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:45:55,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60022 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:48:41,691 - lava_callback - INFO - Received callback for node 69c2674f4a2c5046eb88085d INFO: 172.20.0.1:43410 - "POST /node/69c2674f4a2c5046eb88085d HTTP/1.0" 202 Accepted 2026-03-24 10:48:41,700 - lava_callback - INFO - Starting async job submit for node 69c2674f4a2c5046eb88085d 2026-03-24 10:48:41,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:48:41,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2674f4a2c5046eb88085d HTTP/1.1" 200 None 2026-03-24 10:48:41,759 - lava_callback - DEBUG - Processing node 69c2674f4a2c5046eb88085d: baseline-nfs-arm64 2026-03-24 10:48:42,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:48:42,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:48:42,940 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2674f4a2c5046eb88085d/log.txt.gz 2026-03-24 10:48:43,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:48:43,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:48:43,753 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2674f4a2c5046eb88085d/lava-logs.yaml 2026-03-24 10:48:43,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 10:48:43,945 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 10:48:43,946 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c2674f4a2c5046eb88085d/lava_callback.json.gz DEBUG: Setup failed for 69c2674f4a2c5046eb88085d. Transitting job node result: incomplete -> fail 2026-03-24 10:48:44,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:48:44,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2674f4a2c5046eb88085d HTTP/1.1" 200 None 2026-03-24 10:48:44,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:48:44,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 10:48:44,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:48:44,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c2674f4a2c5046eb88085d HTTP/1.1" 200 None 2026-03-24 10:48:44,184 - lava_callback - INFO - Completed processing callback for node 69c2674f4a2c5046eb88085d INFO: 172.20.0.1:43420 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:48:55,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:48:55,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43516 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:50:20,841 - lava_callback - INFO - Received callback for node 69bfc9687544ab5cd4a7ec1c INFO: 172.20.0.1:37130 - "POST /node/69bfc9687544ab5cd4a7ec1c HTTP/1.0" 202 Accepted 2026-03-24 10:50:20,848 - lava_callback - INFO - Starting async job submit for node 69bfc9687544ab5cd4a7ec1c 2026-03-24 10:50:20,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:50:20,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc9687544ab5cd4a7ec1c HTTP/1.1" 200 None 2026-03-24 10:50:20,865 - lava_callback - DEBUG - Processing node 69bfc9687544ab5cd4a7ec1c: baseline-arm64-clabbe 2026-03-24 10:50:20,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc9687544ab5cd4a7ec1c 2026-03-24 10:50:20,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:50:20,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc9687544ab5cd4a7ec1c HTTP/1.1" 200 None 2026-03-24 10:50:20,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:50:20,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 10:50:20,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:50:20,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc9687544ab5cd4a7ec1c HTTP/1.1" 200 None 2026-03-24 10:50:20,924 - lava_callback - ERROR - Error processing callback for node 69bfc9687544ab5cd4a7ec1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46024 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:51:40,624 - lava_callback - INFO - Received callback for node 69bfc9b37544ab5cd4a7ec25 INFO: 172.20.0.1:48462 - "POST /node/69bfc9b37544ab5cd4a7ec25 HTTP/1.0" 202 Accepted 2026-03-24 10:51:40,630 - lava_callback - INFO - Starting async job submit for node 69bfc9b37544ab5cd4a7ec25 2026-03-24 10:51:40,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:51:40,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc9b37544ab5cd4a7ec25 HTTP/1.1" 200 None 2026-03-24 10:51:40,645 - lava_callback - DEBUG - Processing node 69bfc9b37544ab5cd4a7ec25: baseline-arm64-clabbe 2026-03-24 10:51:40,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc9b37544ab5cd4a7ec25 2026-03-24 10:51:40,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:51:40,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc9b37544ab5cd4a7ec25 HTTP/1.1" 200 None 2026-03-24 10:51:40,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:51:40,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 10:51:40,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:51:40,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc9b37544ab5cd4a7ec25 HTTP/1.1" 200 None 2026-03-24 10:51:40,710 - lava_callback - ERROR - Error processing callback for node 69bfc9b37544ab5cd4a7ec25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48078 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38090 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:53:00,677 - lava_callback - INFO - Received callback for node 69bfc9ff7544ab5cd4a7ec3c INFO: 172.20.0.1:46954 - "POST /node/69bfc9ff7544ab5cd4a7ec3c HTTP/1.0" 202 Accepted 2026-03-24 10:53:00,682 - lava_callback - INFO - Starting async job submit for node 69bfc9ff7544ab5cd4a7ec3c 2026-03-24 10:53:00,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:53:00,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc9ff7544ab5cd4a7ec3c HTTP/1.1" 200 None 2026-03-24 10:53:00,699 - lava_callback - DEBUG - Processing node 69bfc9ff7544ab5cd4a7ec3c: baseline-arm64-clabbe 2026-03-24 10:53:00,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfc9ff7544ab5cd4a7ec3c 2026-03-24 10:53:00,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:53:00,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfc9ff7544ab5cd4a7ec3c HTTP/1.1" 200 None 2026-03-24 10:53:00,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:53:00,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 10:53:00,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:53:00,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfc9ff7544ab5cd4a7ec3c HTTP/1.1" 200 None 2026-03-24 10:53:00,755 - lava_callback - ERROR - Error processing callback for node 69bfc9ff7544ab5cd4a7ec3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47416 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:54:20,769 - lava_callback - INFO - Received callback for node 69bfca4a7544ab5cd4a7ec45 INFO: 172.20.0.1:52726 - "POST /node/69bfca4a7544ab5cd4a7ec45 HTTP/1.0" 202 Accepted 2026-03-24 10:54:20,775 - lava_callback - INFO - Starting async job submit for node 69bfca4a7544ab5cd4a7ec45 2026-03-24 10:54:20,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:54:20,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfca4a7544ab5cd4a7ec45 HTTP/1.1" 200 None 2026-03-24 10:54:20,798 - lava_callback - DEBUG - Processing node 69bfca4a7544ab5cd4a7ec45: baseline-arm64-clabbe 2026-03-24 10:54:20,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfca4a7544ab5cd4a7ec45 2026-03-24 10:54:20,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:54:20,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfca4a7544ab5cd4a7ec45 HTTP/1.1" 200 None 2026-03-24 10:54:20,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:54:20,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 10:54:20,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:54:20,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfca4a7544ab5cd4a7ec45 HTTP/1.1" 200 None 2026-03-24 10:54:20,859 - lava_callback - ERROR - Error processing callback for node 69bfca4a7544ab5cd4a7ec45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44630 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:55:20,807 - lava_callback - INFO - Received callback for node 69bfca967544ab5cd4a7ec5b INFO: 172.20.0.1:36508 - "POST /node/69bfca967544ab5cd4a7ec5b HTTP/1.0" 202 Accepted 2026-03-24 10:55:20,812 - lava_callback - INFO - Starting async job submit for node 69bfca967544ab5cd4a7ec5b 2026-03-24 10:55:20,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:55:20,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfca967544ab5cd4a7ec5b HTTP/1.1" 200 None 2026-03-24 10:55:20,828 - lava_callback - DEBUG - Processing node 69bfca967544ab5cd4a7ec5b: baseline-arm64-clabbe 2026-03-24 10:55:20,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfca967544ab5cd4a7ec5b 2026-03-24 10:55:20,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:55:20,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfca967544ab5cd4a7ec5b HTTP/1.1" 200 None 2026-03-24 10:55:20,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:55:20,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 10:55:20,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:55:20,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfca967544ab5cd4a7ec5b HTTP/1.1" 200 None 2026-03-24 10:55:20,882 - lava_callback - ERROR - Error processing callback for node 69bfca967544ab5cd4a7ec5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56788 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:56:40,860 - lava_callback - INFO - Received callback for node 69bfcae07544ab5cd4a7ec65 INFO: 172.20.0.1:50052 - "POST /node/69bfcae07544ab5cd4a7ec65 HTTP/1.0" 202 Accepted 2026-03-24 10:56:40,866 - lava_callback - INFO - Starting async job submit for node 69bfcae07544ab5cd4a7ec65 2026-03-24 10:56:40,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:56:40,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcae07544ab5cd4a7ec65 HTTP/1.1" 200 None 2026-03-24 10:56:40,882 - lava_callback - DEBUG - Processing node 69bfcae07544ab5cd4a7ec65: baseline-arm64-clabbe 2026-03-24 10:56:40,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcae07544ab5cd4a7ec65 2026-03-24 10:56:40,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:56:40,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcae07544ab5cd4a7ec65 HTTP/1.1" 200 None 2026-03-24 10:56:40,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:56:40,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 10:56:40,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:56:40,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcae07544ab5cd4a7ec65 HTTP/1.1" 200 None 2026-03-24 10:56:40,976 - lava_callback - ERROR - Error processing callback for node 69bfcae07544ab5cd4a7ec65: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37630 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:57:46,291 - lava_callback - INFO - Received callback for node 69bfcb197544ab5cd4a7ec71 INFO: 172.20.0.1:45452 - "POST /node/69bfcb197544ab5cd4a7ec71 HTTP/1.0" 202 Accepted 2026-03-24 10:57:46,297 - lava_callback - INFO - Starting async job submit for node 69bfcb197544ab5cd4a7ec71 2026-03-24 10:57:46,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:57:46,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcb197544ab5cd4a7ec71 HTTP/1.1" 200 None 2026-03-24 10:57:46,312 - lava_callback - DEBUG - Processing node 69bfcb197544ab5cd4a7ec71: baseline-arm-baylibre 2026-03-24 10:57:46,313 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcb197544ab5cd4a7ec71 2026-03-24 10:57:46,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:57:46,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcb197544ab5cd4a7ec71 HTTP/1.1" 200 None 2026-03-24 10:57:46,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:57:46,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 10:57:46,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:57:46,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcb197544ab5cd4a7ec71 HTTP/1.1" 200 None 2026-03-24 10:57:46,399 - lava_callback - ERROR - Error processing callback for node 69bfcb197544ab5cd4a7ec71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57486 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:58:26,223 - lava_callback - INFO - Received callback for node 69bfcb3d7544ab5cd4a7ec7c INFO: 172.20.0.1:45074 - "POST /node/69bfcb3d7544ab5cd4a7ec7c HTTP/1.0" 202 Accepted 2026-03-24 10:58:26,229 - lava_callback - INFO - Starting async job submit for node 69bfcb3d7544ab5cd4a7ec7c 2026-03-24 10:58:26,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:58:26,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcb3d7544ab5cd4a7ec7c HTTP/1.1" 200 None 2026-03-24 10:58:26,244 - lava_callback - DEBUG - Processing node 69bfcb3d7544ab5cd4a7ec7c: baseline-arm-baylibre 2026-03-24 10:58:26,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcb3d7544ab5cd4a7ec7c 2026-03-24 10:58:26,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:58:26,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcb3d7544ab5cd4a7ec7c HTTP/1.1" 200 None 2026-03-24 10:58:26,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:58:26,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 10:58:26,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:58:26,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcb3d7544ab5cd4a7ec7c HTTP/1.1" 200 None 2026-03-24 10:58:26,305 - lava_callback - ERROR - Error processing callback for node 69bfcb3d7544ab5cd4a7ec7c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52866 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:58:46,232 - lava_callback - INFO - Received callback for node 69bfcb617544ab5cd4a7ec86 INFO: 172.20.0.1:58490 - "POST /node/69bfcb617544ab5cd4a7ec86 HTTP/1.0" 202 Accepted 2026-03-24 10:58:46,239 - lava_callback - INFO - Starting async job submit for node 69bfcb617544ab5cd4a7ec86 2026-03-24 10:58:46,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:58:46,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcb617544ab5cd4a7ec86 HTTP/1.1" 200 None 2026-03-24 10:58:46,273 - lava_callback - DEBUG - Processing node 69bfcb617544ab5cd4a7ec86: baseline-arm-baylibre 2026-03-24 10:58:46,274 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcb617544ab5cd4a7ec86 2026-03-24 10:58:46,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:58:46,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcb617544ab5cd4a7ec86 HTTP/1.1" 200 None 2026-03-24 10:58:46,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:58:46,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 10:58:46,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:58:46,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcb617544ab5cd4a7ec86 HTTP/1.1" 200 None 2026-03-24 10:58:46,355 - lava_callback - ERROR - Error processing callback for node 69bfcb617544ab5cd4a7ec86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54314 - "GET / HTTP/1.0" 200 OK 2026-03-24 10:59:26,298 - lava_callback - INFO - Received callback for node 69bfcb857544ab5cd4a7ec99 INFO: 172.20.0.1:42210 - "POST /node/69bfcb857544ab5cd4a7ec99 HTTP/1.0" 202 Accepted 2026-03-24 10:59:26,304 - lava_callback - INFO - Starting async job submit for node 69bfcb857544ab5cd4a7ec99 2026-03-24 10:59:26,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:59:26,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcb857544ab5cd4a7ec99 HTTP/1.1" 200 None 2026-03-24 10:59:26,320 - lava_callback - DEBUG - Processing node 69bfcb857544ab5cd4a7ec99: baseline-arm-baylibre 2026-03-24 10:59:26,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcb857544ab5cd4a7ec99 2026-03-24 10:59:26,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:59:26,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcb857544ab5cd4a7ec99 HTTP/1.1" 200 None 2026-03-24 10:59:26,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:59:26,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 10:59:26,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 10:59:26,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcb857544ab5cd4a7ec99 HTTP/1.1" 200 None 2026-03-24 10:59:26,384 - lava_callback - ERROR - Error processing callback for node 69bfcb857544ab5cd4a7ec99: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52904 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:00:06,320 - lava_callback - INFO - Received callback for node 69bfcbaa7544ab5cd4a7eca5 INFO: 172.20.0.1:59390 - "POST /node/69bfcbaa7544ab5cd4a7eca5 HTTP/1.0" 202 Accepted 2026-03-24 11:00:06,326 - lava_callback - INFO - Starting async job submit for node 69bfcbaa7544ab5cd4a7eca5 2026-03-24 11:00:06,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:00:06,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcbaa7544ab5cd4a7eca5 HTTP/1.1" 200 None 2026-03-24 11:00:06,362 - lava_callback - DEBUG - Processing node 69bfcbaa7544ab5cd4a7eca5: baseline-arm-baylibre 2026-03-24 11:00:06,362 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcbaa7544ab5cd4a7eca5 2026-03-24 11:00:06,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:00:06,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcbaa7544ab5cd4a7eca5 HTTP/1.1" 200 None 2026-03-24 11:00:06,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:59400 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:00:06,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 11:00:06,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:00:06,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcbaa7544ab5cd4a7eca5 HTTP/1.1" 200 None 2026-03-24 11:00:06,596 - lava_callback - ERROR - Error processing callback for node 69bfcbaa7544ab5cd4a7eca5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46446 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:00:46,389 - lava_callback - INFO - Received callback for node 69bfcbcd7544ab5cd4a7ecb5 INFO: 172.20.0.1:52404 - "POST /node/69bfcbcd7544ab5cd4a7ecb5 HTTP/1.0" 202 Accepted 2026-03-24 11:00:46,394 - lava_callback - INFO - Starting async job submit for node 69bfcbcd7544ab5cd4a7ecb5 2026-03-24 11:00:46,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:00:46,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcbcd7544ab5cd4a7ecb5 HTTP/1.1" 200 None 2026-03-24 11:00:46,413 - lava_callback - DEBUG - Processing node 69bfcbcd7544ab5cd4a7ecb5: baseline-arm-baylibre 2026-03-24 11:00:46,413 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcbcd7544ab5cd4a7ecb5 2026-03-24 11:00:46,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:00:46,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcbcd7544ab5cd4a7ecb5 HTTP/1.1" 200 None 2026-03-24 11:00:46,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:00:46,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 11:00:46,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:00:46,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcbcd7544ab5cd4a7ecb5 HTTP/1.1" 200 None 2026-03-24 11:00:46,477 - lava_callback - ERROR - Error processing callback for node 69bfcbcd7544ab5cd4a7ecb5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41224 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:02:01,137 - lava_callback - INFO - Received callback for node 69bfcc157544ab5cd4a7ecd0 INFO: 172.20.0.1:40812 - "POST /node/69bfcc157544ab5cd4a7ecd0 HTTP/1.0" 202 Accepted 2026-03-24 11:02:01,142 - lava_callback - INFO - Starting async job submit for node 69bfcc157544ab5cd4a7ecd0 2026-03-24 11:02:01,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:02:01,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcc157544ab5cd4a7ecd0 HTTP/1.1" 200 None 2026-03-24 11:02:01,184 - lava_callback - DEBUG - Processing node 69bfcc157544ab5cd4a7ecd0: baseline-arm-clabbe 2026-03-24 11:02:01,185 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcc157544ab5cd4a7ecd0 2026-03-24 11:02:01,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:02:01,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcc157544ab5cd4a7ecd0 HTTP/1.1" 200 None 2026-03-24 11:02:01,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:02:01,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 11:02:01,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:02:01,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcc157544ab5cd4a7ecd0 HTTP/1.1" 200 None 2026-03-24 11:02:01,274 - lava_callback - ERROR - Error processing callback for node 69bfcc157544ab5cd4a7ecd0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44816 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:02:21,128 - lava_callback - INFO - Received callback for node 69bfcc3a7544ab5cd4a7ece1 INFO: 172.20.0.1:34644 - "POST /node/69bfcc3a7544ab5cd4a7ece1 HTTP/1.0" 202 Accepted 2026-03-24 11:02:21,133 - lava_callback - INFO - Starting async job submit for node 69bfcc3a7544ab5cd4a7ece1 2026-03-24 11:02:21,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:02:21,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcc3a7544ab5cd4a7ece1 HTTP/1.1" 200 None 2026-03-24 11:02:21,174 - lava_callback - DEBUG - Processing node 69bfcc3a7544ab5cd4a7ece1: baseline-arm-clabbe 2026-03-24 11:02:21,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcc3a7544ab5cd4a7ece1 2026-03-24 11:02:21,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:02:21,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcc3a7544ab5cd4a7ece1 HTTP/1.1" 200 None 2026-03-24 11:02:21,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:02:21,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 11:02:21,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:02:21,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcc3a7544ab5cd4a7ece1 HTTP/1.1" 200 None 2026-03-24 11:02:21,232 - lava_callback - ERROR - Error processing callback for node 69bfcc3a7544ab5cd4a7ece1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58288 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:03:01,168 - lava_callback - INFO - Received callback for node 69bfcc5e7544ab5cd4a7eced INFO: 172.20.0.1:50592 - "POST /node/69bfcc5e7544ab5cd4a7eced HTTP/1.0" 202 Accepted 2026-03-24 11:03:01,174 - lava_callback - INFO - Starting async job submit for node 69bfcc5e7544ab5cd4a7eced 2026-03-24 11:03:01,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:03:01,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcc5e7544ab5cd4a7eced HTTP/1.1" 200 None 2026-03-24 11:03:01,193 - lava_callback - DEBUG - Processing node 69bfcc5e7544ab5cd4a7eced: baseline-arm-clabbe 2026-03-24 11:03:01,193 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcc5e7544ab5cd4a7eced 2026-03-24 11:03:01,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:03:01,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcc5e7544ab5cd4a7eced HTTP/1.1" 200 None 2026-03-24 11:03:01,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:03:01,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 11:03:01,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:03:01,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcc5e7544ab5cd4a7eced HTTP/1.1" 200 None 2026-03-24 11:03:01,251 - lava_callback - ERROR - Error processing callback for node 69bfcc5e7544ab5cd4a7eced: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51548 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:05:01,327 - lava_callback - INFO - Received callback for node 69bfcccc7544ab5cd4a7ed0a INFO: 172.20.0.1:38738 - "POST /node/69bfcccc7544ab5cd4a7ed0a HTTP/1.0" 202 Accepted 2026-03-24 11:05:01,332 - lava_callback - INFO - Starting async job submit for node 69bfcccc7544ab5cd4a7ed0a 2026-03-24 11:05:01,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:05:01,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcccc7544ab5cd4a7ed0a HTTP/1.1" 200 None 2026-03-24 11:05:01,372 - lava_callback - DEBUG - Processing node 69bfcccc7544ab5cd4a7ed0a: baseline-arm-clabbe 2026-03-24 11:05:01,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcccc7544ab5cd4a7ed0a 2026-03-24 11:05:01,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:05:01,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcccc7544ab5cd4a7ed0a HTTP/1.1" 200 None 2026-03-24 11:05:01,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:05:01,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 11:05:01,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:05:01,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcccc7544ab5cd4a7ed0a HTTP/1.1" 200 None 2026-03-24 11:05:01,457 - lava_callback - ERROR - Error processing callback for node 69bfcccc7544ab5cd4a7ed0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53526 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:05:41,252 - lava_callback - INFO - Received callback for node 69bfccf17544ab5cd4a7ed19 INFO: 172.20.0.1:56908 - "POST /node/69bfccf17544ab5cd4a7ed19 HTTP/1.0" 202 Accepted 2026-03-24 11:05:41,258 - lava_callback - INFO - Starting async job submit for node 69bfccf17544ab5cd4a7ed19 2026-03-24 11:05:41,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:05:41,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfccf17544ab5cd4a7ed19 HTTP/1.1" 200 None 2026-03-24 11:05:41,274 - lava_callback - DEBUG - Processing node 69bfccf17544ab5cd4a7ed19: baseline-arm-clabbe 2026-03-24 11:05:41,275 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfccf17544ab5cd4a7ed19 2026-03-24 11:05:41,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:05:41,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfccf17544ab5cd4a7ed19 HTTP/1.1" 200 None 2026-03-24 11:05:41,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:05:41,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd6400b80c8ffdcb829f HTTP/1.1" 200 None 2026-03-24 11:05:41,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:05:41,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfccf17544ab5cd4a7ed19 HTTP/1.1" 200 None 2026-03-24 11:05:41,343 - lava_callback - ERROR - Error processing callback for node 69bfccf17544ab5cd4a7ed19: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48864 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:14:07,153 - lava_callback - INFO - Received callback for node 69bfcef57544ab5cd4a7ed78 INFO: 172.20.0.1:48874 - "POST /node/69bfcef57544ab5cd4a7ed78 HTTP/1.0" 202 Accepted 2026-03-24 11:14:07,159 - lava_callback - INFO - Starting async job submit for node 69bfcef57544ab5cd4a7ed78 2026-03-24 11:14:07,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:14:07,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcef57544ab5cd4a7ed78 HTTP/1.1" 200 None 2026-03-24 11:14:07,176 - lava_callback - DEBUG - Processing node 69bfcef57544ab5cd4a7ed78: baseline-arm64-baylibre 2026-03-24 11:14:07,176 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcef57544ab5cd4a7ed78 2026-03-24 11:14:07,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:14:07,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcef57544ab5cd4a7ed78 HTTP/1.1" 200 None 2026-03-24 11:14:07,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:14:07,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:14:07,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:14:07,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcef57544ab5cd4a7ed78 HTTP/1.1" 200 None 2026-03-24 11:14:07,237 - lava_callback - ERROR - Error processing callback for node 69bfcef57544ab5cd4a7ed78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59010 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:16:47,260 - lava_callback - INFO - Received callback for node 69bfcf8c7544ab5cd4a7ed92 INFO: 172.20.0.1:59018 - "POST /node/69bfcf8c7544ab5cd4a7ed92 HTTP/1.0" 202 Accepted 2026-03-24 11:16:47,266 - lava_callback - INFO - Starting async job submit for node 69bfcf8c7544ab5cd4a7ed92 2026-03-24 11:16:47,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:16:47,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcf8c7544ab5cd4a7ed92 HTTP/1.1" 200 None 2026-03-24 11:16:47,282 - lava_callback - DEBUG - Processing node 69bfcf8c7544ab5cd4a7ed92: baseline-arm64-baylibre 2026-03-24 11:16:47,282 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfcf8c7544ab5cd4a7ed92 2026-03-24 11:16:47,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:16:47,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfcf8c7544ab5cd4a7ed92 HTTP/1.1" 200 None 2026-03-24 11:16:47,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:16:47,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:16:47,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:16:47,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfcf8c7544ab5cd4a7ed92 HTTP/1.1" 200 None 2026-03-24 11:16:47,391 - lava_callback - ERROR - Error processing callback for node 69bfcf8c7544ab5cd4a7ed92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60422 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:19:07,332 - lava_callback - INFO - Received callback for node 69bfd0227544ab5cd4a7edb4 INFO: 172.20.0.1:60436 - "POST /node/69bfd0227544ab5cd4a7edb4 HTTP/1.0" 202 Accepted 2026-03-24 11:19:07,337 - lava_callback - INFO - Starting async job submit for node 69bfd0227544ab5cd4a7edb4 2026-03-24 11:19:07,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:19:07,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd0227544ab5cd4a7edb4 HTTP/1.1" 200 None 2026-03-24 11:19:07,375 - lava_callback - DEBUG - Processing node 69bfd0227544ab5cd4a7edb4: baseline-arm64-baylibre 2026-03-24 11:19:07,375 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd0227544ab5cd4a7edb4 2026-03-24 11:19:07,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:19:07,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd0227544ab5cd4a7edb4 HTTP/1.1" 200 None 2026-03-24 11:19:07,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:19:07,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:19:07,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:19:07,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd0227544ab5cd4a7edb4 HTTP/1.1" 200 None 2026-03-24 11:19:07,450 - lava_callback - ERROR - Error processing callback for node 69bfd0227544ab5cd4a7edb4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34126 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:21:47,797 - lava_callback - INFO - Received callback for node 69bfd0b87544ab5cd4a7edcd INFO: 172.20.0.1:34134 - "POST /node/69bfd0b87544ab5cd4a7edcd HTTP/1.0" 202 Accepted 2026-03-24 11:21:47,802 - lava_callback - INFO - Starting async job submit for node 69bfd0b87544ab5cd4a7edcd 2026-03-24 11:21:47,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:21:47,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd0b87544ab5cd4a7edcd HTTP/1.1" 200 None 2026-03-24 11:21:47,819 - lava_callback - DEBUG - Processing node 69bfd0b87544ab5cd4a7edcd: baseline-arm64-baylibre 2026-03-24 11:21:47,819 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd0b87544ab5cd4a7edcd 2026-03-24 11:21:47,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:21:47,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd0b87544ab5cd4a7edcd HTTP/1.1" 200 None 2026-03-24 11:21:47,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:21:47,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:21:47,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:21:47,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd0b87544ab5cd4a7edcd HTTP/1.1" 200 None 2026-03-24 11:21:47,874 - lava_callback - ERROR - Error processing callback for node 69bfd0b87544ab5cd4a7edcd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60282 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:24:07,579 - lava_callback - INFO - Received callback for node 69bfd14c7544ab5cd4a7edf0 INFO: 172.20.0.1:60294 - "POST /node/69bfd14c7544ab5cd4a7edf0 HTTP/1.0" 202 Accepted 2026-03-24 11:24:07,587 - lava_callback - INFO - Starting async job submit for node 69bfd14c7544ab5cd4a7edf0 2026-03-24 11:24:07,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:24:07,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd14c7544ab5cd4a7edf0 HTTP/1.1" 200 None 2026-03-24 11:24:07,623 - lava_callback - DEBUG - Processing node 69bfd14c7544ab5cd4a7edf0: baseline-arm64-baylibre 2026-03-24 11:24:07,623 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd14c7544ab5cd4a7edf0 2026-03-24 11:24:07,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:24:07,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd14c7544ab5cd4a7edf0 HTTP/1.1" 200 None 2026-03-24 11:24:07,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:24:07,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 11:24:07,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:24:07,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd14c7544ab5cd4a7edf0 HTTP/1.1" 200 None 2026-03-24 11:24:07,745 - lava_callback - ERROR - Error processing callback for node 69bfd14c7544ab5cd4a7edf0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55134 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:26:27,685 - lava_callback - INFO - Received callback for node 69bfd1e17544ab5cd4a7ee0b INFO: 172.20.0.1:55138 - "POST /node/69bfd1e17544ab5cd4a7ee0b HTTP/1.0" 202 Accepted 2026-03-24 11:26:27,692 - lava_callback - INFO - Starting async job submit for node 69bfd1e17544ab5cd4a7ee0b 2026-03-24 11:26:27,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:26:27,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd1e17544ab5cd4a7ee0b HTTP/1.1" 200 None 2026-03-24 11:26:27,721 - lava_callback - DEBUG - Processing node 69bfd1e17544ab5cd4a7ee0b: baseline-arm64-baylibre 2026-03-24 11:26:27,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd1e17544ab5cd4a7ee0b 2026-03-24 11:26:27,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:26:27,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd1e17544ab5cd4a7ee0b HTTP/1.1" 200 None 2026-03-24 11:26:27,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:26:27,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:26:27,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:26:27,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd1e17544ab5cd4a7ee0b HTTP/1.1" 200 None 2026-03-24 11:26:27,795 - lava_callback - ERROR - Error processing callback for node 69bfd1e17544ab5cd4a7ee0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52990 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:27:47,951 - lava_callback - INFO - Received callback for node 69bfd22b7544ab5cd4a7ee16 INFO: 172.20.0.1:52994 - "POST /node/69bfd22b7544ab5cd4a7ee16 HTTP/1.0" 202 Accepted 2026-03-24 11:27:47,957 - lava_callback - INFO - Starting async job submit for node 69bfd22b7544ab5cd4a7ee16 2026-03-24 11:27:47,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:27:47,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd22b7544ab5cd4a7ee16 HTTP/1.1" 200 None 2026-03-24 11:27:47,973 - lava_callback - DEBUG - Processing node 69bfd22b7544ab5cd4a7ee16: baseline-arm64-baylibre 2026-03-24 11:27:47,974 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd22b7544ab5cd4a7ee16 2026-03-24 11:27:47,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:27:47,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd22b7544ab5cd4a7ee16 HTTP/1.1" 200 None 2026-03-24 11:27:47,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:27:48,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 11:27:48,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:27:48,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd22b7544ab5cd4a7ee16 HTTP/1.1" 200 None 2026-03-24 11:27:48,029 - lava_callback - ERROR - Error processing callback for node 69bfd22b7544ab5cd4a7ee16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38130 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:29:22,392 - lava_callback - INFO - Received callback for node 69bfd27d7544ab5cd4a7ee25 INFO: 172.20.0.1:47084 - "POST /node/69bfd27d7544ab5cd4a7ee25 HTTP/1.0" 202 Accepted 2026-03-24 11:29:22,398 - lava_callback - INFO - Starting async job submit for node 69bfd27d7544ab5cd4a7ee25 2026-03-24 11:29:22,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:29:22,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd27d7544ab5cd4a7ee25 HTTP/1.1" 200 None 2026-03-24 11:29:22,413 - lava_callback - DEBUG - Processing node 69bfd27d7544ab5cd4a7ee25: baseline-arm64-clabbe 2026-03-24 11:29:22,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd27d7544ab5cd4a7ee25 2026-03-24 11:29:22,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:29:22,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd27d7544ab5cd4a7ee25 HTTP/1.1" 200 None 2026-03-24 11:29:22,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:29:22,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 11:29:22,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:29:22,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd27d7544ab5cd4a7ee25 HTTP/1.1" 200 None 2026-03-24 11:29:22,487 - lava_callback - ERROR - Error processing callback for node 69bfd27d7544ab5cd4a7ee25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53370 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:30:22,370 - lava_callback - INFO - Received callback for node 69bfd2cb7544ab5cd4a7ee38 INFO: 172.20.0.1:50408 - "POST /node/69bfd2cb7544ab5cd4a7ee38 HTTP/1.0" 202 Accepted 2026-03-24 11:30:22,377 - lava_callback - INFO - Starting async job submit for node 69bfd2cb7544ab5cd4a7ee38 2026-03-24 11:30:22,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:30:22,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd2cb7544ab5cd4a7ee38 HTTP/1.1" 200 None 2026-03-24 11:30:22,420 - lava_callback - DEBUG - Processing node 69bfd2cb7544ab5cd4a7ee38: baseline-arm64-clabbe 2026-03-24 11:30:22,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd2cb7544ab5cd4a7ee38 2026-03-24 11:30:22,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:30:22,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd2cb7544ab5cd4a7ee38 HTTP/1.1" 200 None 2026-03-24 11:30:22,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:30:22,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:30:22,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:30:22,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd2cb7544ab5cd4a7ee38 HTTP/1.1" 200 None 2026-03-24 11:30:22,497 - lava_callback - ERROR - Error processing callback for node 69bfd2cb7544ab5cd4a7ee38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52706 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:31:42,425 - lava_callback - INFO - Received callback for node 69bfd3187544ab5cd4a7ee3e INFO: 172.20.0.1:53426 - "POST /node/69bfd3187544ab5cd4a7ee3e HTTP/1.0" 202 Accepted 2026-03-24 11:31:42,431 - lava_callback - INFO - Starting async job submit for node 69bfd3187544ab5cd4a7ee3e 2026-03-24 11:31:42,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:31:42,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd3187544ab5cd4a7ee3e HTTP/1.1" 200 None 2026-03-24 11:31:42,456 - lava_callback - DEBUG - Processing node 69bfd3187544ab5cd4a7ee3e: baseline-arm64-clabbe 2026-03-24 11:31:42,456 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd3187544ab5cd4a7ee3e 2026-03-24 11:31:42,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:31:42,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd3187544ab5cd4a7ee3e HTTP/1.1" 200 None 2026-03-24 11:31:42,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:31:42,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:31:42,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:31:42,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd3187544ab5cd4a7ee3e HTTP/1.1" 200 None 2026-03-24 11:31:42,536 - lava_callback - ERROR - Error processing callback for node 69bfd3187544ab5cd4a7ee3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53664 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:33:02,492 - lava_callback - INFO - Received callback for node 69bfd3627544ab5cd4a7ee57 INFO: 172.20.0.1:35852 - "POST /node/69bfd3627544ab5cd4a7ee57 HTTP/1.0" 202 Accepted 2026-03-24 11:33:02,497 - lava_callback - INFO - Starting async job submit for node 69bfd3627544ab5cd4a7ee57 2026-03-24 11:33:02,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:33:02,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd3627544ab5cd4a7ee57 HTTP/1.1" 200 None 2026-03-24 11:33:02,536 - lava_callback - DEBUG - Processing node 69bfd3627544ab5cd4a7ee57: baseline-arm64-clabbe 2026-03-24 11:33:02,537 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd3627544ab5cd4a7ee57 2026-03-24 11:33:02,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:33:02,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd3627544ab5cd4a7ee57 HTTP/1.1" 200 None 2026-03-24 11:33:02,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:33:02,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 11:33:02,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:33:02,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd3627544ab5cd4a7ee57 HTTP/1.1" 200 None 2026-03-24 11:33:02,604 - lava_callback - ERROR - Error processing callback for node 69bfd3627544ab5cd4a7ee57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56284 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:34:22,607 - lava_callback - INFO - Received callback for node 69bfd3ac7544ab5cd4a7ee5d INFO: 172.20.0.1:35808 - "POST /node/69bfd3ac7544ab5cd4a7ee5d HTTP/1.0" 202 Accepted 2026-03-24 11:34:22,612 - lava_callback - INFO - Starting async job submit for node 69bfd3ac7544ab5cd4a7ee5d 2026-03-24 11:34:22,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:34:22,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd3ac7544ab5cd4a7ee5d HTTP/1.1" 200 None 2026-03-24 11:34:22,629 - lava_callback - DEBUG - Processing node 69bfd3ac7544ab5cd4a7ee5d: baseline-arm64-clabbe 2026-03-24 11:34:22,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd3ac7544ab5cd4a7ee5d 2026-03-24 11:34:22,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:34:22,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd3ac7544ab5cd4a7ee5d HTTP/1.1" 200 None 2026-03-24 11:34:22,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:34:22,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:34:22,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:34:22,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd3ac7544ab5cd4a7ee5d HTTP/1.1" 200 None 2026-03-24 11:34:22,690 - lava_callback - ERROR - Error processing callback for node 69bfd3ac7544ab5cd4a7ee5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36810 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:35:22,611 - lava_callback - INFO - Received callback for node 69bfd3f77544ab5cd4a7ee71 INFO: 172.20.0.1:53932 - "POST /node/69bfd3f77544ab5cd4a7ee71 HTTP/1.0" 202 Accepted 2026-03-24 11:35:22,616 - lava_callback - INFO - Starting async job submit for node 69bfd3f77544ab5cd4a7ee71 2026-03-24 11:35:22,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:35:22,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd3f77544ab5cd4a7ee71 HTTP/1.1" 200 None 2026-03-24 11:35:22,634 - lava_callback - DEBUG - Processing node 69bfd3f77544ab5cd4a7ee71: baseline-arm64-clabbe 2026-03-24 11:35:22,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd3f77544ab5cd4a7ee71 2026-03-24 11:35:22,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:35:22,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd3f77544ab5cd4a7ee71 HTTP/1.1" 200 None 2026-03-24 11:35:22,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:35:22,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 11:35:22,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:35:22,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd3f77544ab5cd4a7ee71 HTTP/1.1" 200 None 2026-03-24 11:35:22,721 - lava_callback - ERROR - Error processing callback for node 69bfd3f77544ab5cd4a7ee71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32772 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:36:42,728 - lava_callback - INFO - Received callback for node 69bfd4427544ab5cd4a7ee78 INFO: 172.20.0.1:58436 - "POST /node/69bfd4427544ab5cd4a7ee78 HTTP/1.0" 202 Accepted 2026-03-24 11:36:42,734 - lava_callback - INFO - Starting async job submit for node 69bfd4427544ab5cd4a7ee78 2026-03-24 11:36:42,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:36:42,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd4427544ab5cd4a7ee78 HTTP/1.1" 200 None 2026-03-24 11:36:42,780 - lava_callback - DEBUG - Processing node 69bfd4427544ab5cd4a7ee78: baseline-arm64-clabbe 2026-03-24 11:36:42,780 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd4427544ab5cd4a7ee78 2026-03-24 11:36:42,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:36:42,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd4427544ab5cd4a7ee78 HTTP/1.1" 200 None 2026-03-24 11:36:42,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:36:42,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 11:36:42,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:36:42,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd4427544ab5cd4a7ee78 HTTP/1.1" 200 None 2026-03-24 11:36:42,843 - lava_callback - ERROR - Error processing callback for node 69bfd4427544ab5cd4a7ee78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50634 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:38:02,759 - lava_callback - INFO - Received callback for node 69bfd48c7544ab5cd4a7ee8b INFO: 172.20.0.1:46288 - "POST /node/69bfd48c7544ab5cd4a7ee8b HTTP/1.0" 202 Accepted 2026-03-24 11:38:02,764 - lava_callback - INFO - Starting async job submit for node 69bfd48c7544ab5cd4a7ee8b 2026-03-24 11:38:02,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:38:02,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd48c7544ab5cd4a7ee8b HTTP/1.1" 200 None 2026-03-24 11:38:02,799 - lava_callback - DEBUG - Processing node 69bfd48c7544ab5cd4a7ee8b: baseline-arm64-clabbe 2026-03-24 11:38:02,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd48c7544ab5cd4a7ee8b 2026-03-24 11:38:02,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:38:02,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd48c7544ab5cd4a7ee8b HTTP/1.1" 200 None 2026-03-24 11:38:02,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:38:02,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:38:02,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:38:02,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd48c7544ab5cd4a7ee8b HTTP/1.1" 200 None 2026-03-24 11:38:02,909 - lava_callback - ERROR - Error processing callback for node 69bfd48c7544ab5cd4a7ee8b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51188 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:39:22,824 - lava_callback - INFO - Received callback for node 69bfd4d77544ab5cd4a7ee92 INFO: 172.20.0.1:57234 - "POST /node/69bfd4d77544ab5cd4a7ee92 HTTP/1.0" 202 Accepted 2026-03-24 11:39:22,829 - lava_callback - INFO - Starting async job submit for node 69bfd4d77544ab5cd4a7ee92 2026-03-24 11:39:22,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:39:22,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd4d77544ab5cd4a7ee92 HTTP/1.1" 200 None 2026-03-24 11:39:22,846 - lava_callback - DEBUG - Processing node 69bfd4d77544ab5cd4a7ee92: baseline-arm64-clabbe 2026-03-24 11:39:22,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd4d77544ab5cd4a7ee92 2026-03-24 11:39:22,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:39:22,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd4d77544ab5cd4a7ee92 HTTP/1.1" 200 None 2026-03-24 11:39:22,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:39:22,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 11:39:22,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:39:22,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd4d77544ab5cd4a7ee92 HTTP/1.1" 200 None 2026-03-24 11:39:22,908 - lava_callback - ERROR - Error processing callback for node 69bfd4d77544ab5cd4a7ee92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39360 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:40:22,928 - lava_callback - INFO - Received callback for node 69bfd5237544ab5cd4a7eea5 INFO: 172.20.0.1:48754 - "POST /node/69bfd5237544ab5cd4a7eea5 HTTP/1.0" 202 Accepted 2026-03-24 11:40:22,933 - lava_callback - INFO - Starting async job submit for node 69bfd5237544ab5cd4a7eea5 2026-03-24 11:40:22,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:40:22,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd5237544ab5cd4a7eea5 HTTP/1.1" 200 None 2026-03-24 11:40:22,950 - lava_callback - DEBUG - Processing node 69bfd5237544ab5cd4a7eea5: baseline-arm64-clabbe 2026-03-24 11:40:22,950 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd5237544ab5cd4a7eea5 2026-03-24 11:40:22,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:40:22,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd5237544ab5cd4a7eea5 HTTP/1.1" 200 None 2026-03-24 11:40:22,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:40:22,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:40:22,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:40:23,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd5237544ab5cd4a7eea5 HTTP/1.1" 200 None 2026-03-24 11:40:23,001 - lava_callback - ERROR - Error processing callback for node 69bfd5237544ab5cd4a7eea5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44388 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:41:42,958 - lava_callback - INFO - Received callback for node 69bfd5737544ab5cd4a7eeb0 INFO: 172.20.0.1:42750 - "POST /node/69bfd5737544ab5cd4a7eeb0 HTTP/1.0" 202 Accepted 2026-03-24 11:41:42,964 - lava_callback - INFO - Starting async job submit for node 69bfd5737544ab5cd4a7eeb0 2026-03-24 11:41:42,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:41:43,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd5737544ab5cd4a7eeb0 HTTP/1.1" 200 None 2026-03-24 11:41:43,004 - lava_callback - DEBUG - Processing node 69bfd5737544ab5cd4a7eeb0: baseline-arm64-clabbe 2026-03-24 11:41:43,004 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd5737544ab5cd4a7eeb0 2026-03-24 11:41:43,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:41:43,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd5737544ab5cd4a7eeb0 HTTP/1.1" 200 None 2026-03-24 11:41:43,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:41:43,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4e4a85a0f2cbb32a746 HTTP/1.1" 200 None 2026-03-24 11:41:43,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:41:43,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd5737544ab5cd4a7eeb0 HTTP/1.1" 200 None 2026-03-24 11:41:43,090 - lava_callback - ERROR - Error processing callback for node 69bfd5737544ab5cd4a7eeb0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57028 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:43:43,030 - lava_callback - INFO - Received callback for node 69bfd5e27544ab5cd4a7eecc INFO: 172.20.0.1:38524 - "POST /node/69bfd5e27544ab5cd4a7eecc HTTP/1.0" 202 Accepted 2026-03-24 11:43:43,036 - lava_callback - INFO - Starting async job submit for node 69bfd5e27544ab5cd4a7eecc 2026-03-24 11:43:43,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:43:43,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd5e27544ab5cd4a7eecc HTTP/1.1" 200 None 2026-03-24 11:43:43,070 - lava_callback - DEBUG - Processing node 69bfd5e27544ab5cd4a7eecc: baseline-arm64-clabbe 2026-03-24 11:43:43,070 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd5e27544ab5cd4a7eecc 2026-03-24 11:43:43,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:43:43,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd5e27544ab5cd4a7eecc HTTP/1.1" 200 None 2026-03-24 11:43:43,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:43:43,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7cd7000b80c8ffdcb82ae HTTP/1.1" 200 None 2026-03-24 11:43:43,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:43:43,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd5e27544ab5cd4a7eecc HTTP/1.1" 200 None 2026-03-24 11:43:43,134 - lava_callback - ERROR - Error processing callback for node 69bfd5e27544ab5cd4a7eecc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55412 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:48:23,250 - lava_callback - INFO - Received callback for node 69bfd6f37544ab5cd4a7ef13 INFO: 172.20.0.1:43984 - "POST /node/69bfd6f37544ab5cd4a7ef13 HTTP/1.0" 202 Accepted 2026-03-24 11:48:23,256 - lava_callback - INFO - Starting async job submit for node 69bfd6f37544ab5cd4a7ef13 2026-03-24 11:48:23,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:48:23,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd6f37544ab5cd4a7ef13 HTTP/1.1" 200 None 2026-03-24 11:48:23,273 - lava_callback - DEBUG - Processing node 69bfd6f37544ab5cd4a7ef13: baseline-arm-clabbe 2026-03-24 11:48:23,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd6f37544ab5cd4a7ef13 2026-03-24 11:48:23,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:48:23,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd6f37544ab5cd4a7ef13 HTTP/1.1" 200 None 2026-03-24 11:48:23,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:48:23,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 11:48:23,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:48:23,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd6f37544ab5cd4a7ef13 HTTP/1.1" 200 None 2026-03-24 11:48:23,356 - lava_callback - ERROR - Error processing callback for node 69bfd6f37544ab5cd4a7ef13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58280 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:48:43,231 - lava_callback - INFO - Received callback for node 69bfd7187544ab5cd4a7ef1f INFO: 172.20.0.1:37512 - "POST /node/69bfd7187544ab5cd4a7ef1f HTTP/1.0" 202 Accepted 2026-03-24 11:48:43,239 - lava_callback - INFO - Starting async job submit for node 69bfd7187544ab5cd4a7ef1f 2026-03-24 11:48:43,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:48:43,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd7187544ab5cd4a7ef1f HTTP/1.1" 200 None 2026-03-24 11:48:43,260 - lava_callback - DEBUG - Processing node 69bfd7187544ab5cd4a7ef1f: baseline-arm-clabbe 2026-03-24 11:48:43,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd7187544ab5cd4a7ef1f 2026-03-24 11:48:43,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:48:43,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd7187544ab5cd4a7ef1f HTTP/1.1" 200 None 2026-03-24 11:48:43,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:48:43,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 11:48:43,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:48:43,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd7187544ab5cd4a7ef1f HTTP/1.1" 200 None 2026-03-24 11:48:43,329 - lava_callback - ERROR - Error processing callback for node 69bfd7187544ab5cd4a7ef1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35624 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35640 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:49:23,269 - lava_callback - INFO - Received callback for node 69bfd73d7544ab5cd4a7ef32 INFO: 172.20.0.1:42828 - "POST /node/69bfd73d7544ab5cd4a7ef32 HTTP/1.0" 202 Accepted 2026-03-24 11:49:23,274 - lava_callback - INFO - Starting async job submit for node 69bfd73d7544ab5cd4a7ef32 2026-03-24 11:49:23,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:49:23,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd73d7544ab5cd4a7ef32 HTTP/1.1" 200 None 2026-03-24 11:49:23,291 - lava_callback - DEBUG - Processing node 69bfd73d7544ab5cd4a7ef32: baseline-arm-clabbe 2026-03-24 11:49:23,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd73d7544ab5cd4a7ef32 2026-03-24 11:49:23,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:49:23,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd73d7544ab5cd4a7ef32 HTTP/1.1" 200 None 2026-03-24 11:49:23,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:49:23,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 11:49:23,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:49:23,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd73d7544ab5cd4a7ef32 HTTP/1.1" 200 None 2026-03-24 11:49:23,355 - lava_callback - ERROR - Error processing callback for node 69bfd73d7544ab5cd4a7ef32: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40996 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:50:03,324 - lava_callback - INFO - Received callback for node 69bfd7617544ab5cd4a7ef39 INFO: 172.20.0.1:37424 - "POST /node/69bfd7617544ab5cd4a7ef39 HTTP/1.0" 202 Accepted 2026-03-24 11:50:03,330 - lava_callback - INFO - Starting async job submit for node 69bfd7617544ab5cd4a7ef39 2026-03-24 11:50:03,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:50:03,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd7617544ab5cd4a7ef39 HTTP/1.1" 200 None 2026-03-24 11:50:03,346 - lava_callback - DEBUG - Processing node 69bfd7617544ab5cd4a7ef39: baseline-arm-clabbe 2026-03-24 11:50:03,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd7617544ab5cd4a7ef39 2026-03-24 11:50:03,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:50:03,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd7617544ab5cd4a7ef39 HTTP/1.1" 200 None 2026-03-24 11:50:03,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:50:03,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 11:50:03,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:50:03,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd7617544ab5cd4a7ef39 HTTP/1.1" 200 None 2026-03-24 11:50:03,434 - lava_callback - ERROR - Error processing callback for node 69bfd7617544ab5cd4a7ef39: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47558 - "GET / HTTP/1.0" 200 OK 2026-03-24 11:50:43,333 - lava_callback - INFO - Received callback for node 69bfd7857544ab5cd4a7ef43 INFO: 172.20.0.1:33084 - "POST /node/69bfd7857544ab5cd4a7ef43 HTTP/1.0" 202 Accepted 2026-03-24 11:50:43,338 - lava_callback - INFO - Starting async job submit for node 69bfd7857544ab5cd4a7ef43 2026-03-24 11:50:43,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:50:43,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd7857544ab5cd4a7ef43 HTTP/1.1" 200 None 2026-03-24 11:50:43,393 - lava_callback - DEBUG - Processing node 69bfd7857544ab5cd4a7ef43: baseline-arm-clabbe 2026-03-24 11:50:43,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfd7857544ab5cd4a7ef43 2026-03-24 11:50:43,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:50:43,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfd7857544ab5cd4a7ef43 HTTP/1.1" 200 None 2026-03-24 11:50:43,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:50:43,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bec4d5a85a0f2cbb32a731 HTTP/1.1" 200 None 2026-03-24 11:50:43,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 11:50:43,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfd7857544ab5cd4a7ef43 HTTP/1.1" 200 None 2026-03-24 11:50:43,492 - lava_callback - ERROR - Error processing callback for node 69bfd7857544ab5cd4a7ef43: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35454 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58092 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:46:45,865 - lava_callback - INFO - Received callback for node 69bfe4b17544ab5cd4a7efc6 INFO: 172.20.0.1:43364 - "POST /node/69bfe4b17544ab5cd4a7efc6 HTTP/1.0" 202 Accepted 2026-03-24 12:46:45,871 - lava_callback - INFO - Starting async job submit for node 69bfe4b17544ab5cd4a7efc6 2026-03-24 12:46:45,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:46:45,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe4b17544ab5cd4a7efc6 HTTP/1.1" 200 None 2026-03-24 12:46:45,886 - lava_callback - DEBUG - Processing node 69bfe4b17544ab5cd4a7efc6: baseline-arm-clabbe 2026-03-24 12:46:45,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe4b17544ab5cd4a7efc6 2026-03-24 12:46:45,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:46:45,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe4b17544ab5cd4a7efc6 HTTP/1.1" 200 None 2026-03-24 12:46:45,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:46:45,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 12:46:45,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:46:45,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe4b17544ab5cd4a7efc6 HTTP/1.1" 200 None 2026-03-24 12:46:45,958 - lava_callback - ERROR - Error processing callback for node 69bfe4b17544ab5cd4a7efc6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37240 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:47:25,838 - lava_callback - INFO - Received callback for node 69bfe4d57544ab5cd4a7efd9 INFO: 172.20.0.1:45906 - "POST /node/69bfe4d57544ab5cd4a7efd9 HTTP/1.0" 202 Accepted 2026-03-24 12:47:25,844 - lava_callback - INFO - Starting async job submit for node 69bfe4d57544ab5cd4a7efd9 2026-03-24 12:47:25,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:47:25,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe4d57544ab5cd4a7efd9 HTTP/1.1" 200 None 2026-03-24 12:47:25,861 - lava_callback - DEBUG - Processing node 69bfe4d57544ab5cd4a7efd9: baseline-arm-clabbe 2026-03-24 12:47:25,861 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe4d57544ab5cd4a7efd9 2026-03-24 12:47:25,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:47:25,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe4d57544ab5cd4a7efd9 HTTP/1.1" 200 None 2026-03-24 12:47:25,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:47:25,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 12:47:25,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:47:25,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe4d57544ab5cd4a7efd9 HTTP/1.1" 200 None 2026-03-24 12:47:25,913 - lava_callback - ERROR - Error processing callback for node 69bfe4d57544ab5cd4a7efd9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59628 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:48:05,900 - lava_callback - INFO - Received callback for node 69bfe4fa7544ab5cd4a7efe6 INFO: 172.20.0.1:45974 - "POST /node/69bfe4fa7544ab5cd4a7efe6 HTTP/1.0" 202 Accepted 2026-03-24 12:48:05,906 - lava_callback - INFO - Starting async job submit for node 69bfe4fa7544ab5cd4a7efe6 2026-03-24 12:48:05,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:48:05,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe4fa7544ab5cd4a7efe6 HTTP/1.1" 200 None 2026-03-24 12:48:05,922 - lava_callback - DEBUG - Processing node 69bfe4fa7544ab5cd4a7efe6: baseline-arm-clabbe 2026-03-24 12:48:05,922 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe4fa7544ab5cd4a7efe6 2026-03-24 12:48:05,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:48:05,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe4fa7544ab5cd4a7efe6 HTTP/1.1" 200 None 2026-03-24 12:48:05,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:48:05,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 12:48:05,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:48:05,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe4fa7544ab5cd4a7efe6 HTTP/1.1" 200 None 2026-03-24 12:48:05,981 - lava_callback - ERROR - Error processing callback for node 69bfe4fa7544ab5cd4a7efe6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33044 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:48:45,835 - lava_callback - INFO - Received callback for node 69bfe51e7544ab5cd4a7eff0 INFO: 172.20.0.1:33244 - "POST /node/69bfe51e7544ab5cd4a7eff0 HTTP/1.0" 202 Accepted 2026-03-24 12:48:45,841 - lava_callback - INFO - Starting async job submit for node 69bfe51e7544ab5cd4a7eff0 2026-03-24 12:48:45,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:48:45,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe51e7544ab5cd4a7eff0 HTTP/1.1" 200 None 2026-03-24 12:48:45,857 - lava_callback - DEBUG - Processing node 69bfe51e7544ab5cd4a7eff0: baseline-arm-clabbe 2026-03-24 12:48:45,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe51e7544ab5cd4a7eff0 2026-03-24 12:48:45,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:48:45,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe51e7544ab5cd4a7eff0 HTTP/1.1" 200 None 2026-03-24 12:48:45,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:48:45,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 12:48:45,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:48:45,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe51e7544ab5cd4a7eff0 HTTP/1.1" 200 None 2026-03-24 12:48:45,922 - lava_callback - ERROR - Error processing callback for node 69bfe51e7544ab5cd4a7eff0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49254 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:49:25,923 - lava_callback - INFO - Received callback for node 69bfe5427544ab5cd4a7f003 INFO: 172.20.0.1:49266 - "POST /node/69bfe5427544ab5cd4a7f003 HTTP/1.0" 202 Accepted 2026-03-24 12:49:25,929 - lava_callback - INFO - Starting async job submit for node 69bfe5427544ab5cd4a7f003 2026-03-24 12:49:25,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:49:25,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe5427544ab5cd4a7f003 HTTP/1.1" 200 None 2026-03-24 12:49:25,946 - lava_callback - DEBUG - Processing node 69bfe5427544ab5cd4a7f003: baseline-arm-clabbe 2026-03-24 12:49:25,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe5427544ab5cd4a7f003 2026-03-24 12:49:25,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:49:25,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe5427544ab5cd4a7f003 HTTP/1.1" 200 None 2026-03-24 12:49:25,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:49:25,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-24 12:49:25,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:49:25,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe5427544ab5cd4a7f003 HTTP/1.1" 200 None 2026-03-24 12:49:25,996 - lava_callback - ERROR - Error processing callback for node 69bfe5427544ab5cd4a7f003: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:49:45,916 - lava_callback - INFO - Received callback for node 69bfe5667544ab5cd4a7f015 INFO: 172.20.0.1:39354 - "POST /node/69bfe5667544ab5cd4a7f015 HTTP/1.0" 202 Accepted 2026-03-24 12:49:45,921 - lava_callback - INFO - Starting async job submit for node 69bfe5667544ab5cd4a7f015 2026-03-24 12:49:45,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:49:45,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe5667544ab5cd4a7f015 HTTP/1.1" 200 None 2026-03-24 12:49:45,937 - lava_callback - DEBUG - Processing node 69bfe5667544ab5cd4a7f015: baseline-arm-clabbe 2026-03-24 12:49:45,937 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe5667544ab5cd4a7f015 2026-03-24 12:49:45,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:49:45,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe5667544ab5cd4a7f015 HTTP/1.1" 200 None 2026-03-24 12:49:45,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:49:45,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:49:45,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:49:45,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe5667544ab5cd4a7f015 HTTP/1.1" 200 None 2026-03-24 12:49:45,989 - lava_callback - ERROR - Error processing callback for node 69bfe5667544ab5cd4a7f015: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48438 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:50:25,970 - lava_callback - INFO - Received callback for node 69bfe58a7544ab5cd4a7f02a INFO: 172.20.0.1:48442 - "POST /node/69bfe58a7544ab5cd4a7f02a HTTP/1.0" 202 Accepted 2026-03-24 12:50:25,975 - lava_callback - INFO - Starting async job submit for node 69bfe58a7544ab5cd4a7f02a 2026-03-24 12:50:25,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:50:25,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe58a7544ab5cd4a7f02a HTTP/1.1" 200 None 2026-03-24 12:50:25,991 - lava_callback - DEBUG - Processing node 69bfe58a7544ab5cd4a7f02a: baseline-arm-clabbe 2026-03-24 12:50:25,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe58a7544ab5cd4a7f02a 2026-03-24 12:50:25,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:50:26,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe58a7544ab5cd4a7f02a HTTP/1.1" 200 None 2026-03-24 12:50:26,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:50:26,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:50:26,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:50:26,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe58a7544ab5cd4a7f02a HTTP/1.1" 200 None 2026-03-24 12:50:26,065 - lava_callback - ERROR - Error processing callback for node 69bfe58a7544ab5cd4a7f02a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49234 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:51:06,010 - lava_callback - INFO - Received callback for node 69bfe5ae7544ab5cd4a7f039 INFO: 172.20.0.1:60602 - "POST /node/69bfe5ae7544ab5cd4a7f039 HTTP/1.0" 202 Accepted 2026-03-24 12:51:06,016 - lava_callback - INFO - Starting async job submit for node 69bfe5ae7544ab5cd4a7f039 2026-03-24 12:51:06,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:51:06,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe5ae7544ab5cd4a7f039 HTTP/1.1" 200 None 2026-03-24 12:51:06,041 - lava_callback - DEBUG - Processing node 69bfe5ae7544ab5cd4a7f039: baseline-arm-clabbe 2026-03-24 12:51:06,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe5ae7544ab5cd4a7f039 2026-03-24 12:51:06,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:51:06,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe5ae7544ab5cd4a7f039 HTTP/1.1" 200 None 2026-03-24 12:51:06,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:51:06,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:51:06,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:51:06,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe5ae7544ab5cd4a7f039 HTTP/1.1" 200 None 2026-03-24 12:51:06,106 - lava_callback - ERROR - Error processing callback for node 69bfe5ae7544ab5cd4a7f039: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49358 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:51:46,044 - lava_callback - INFO - Received callback for node 69bfe5d27544ab5cd4a7f044 INFO: 172.20.0.1:43348 - "POST /node/69bfe5d27544ab5cd4a7f044 HTTP/1.0" 202 Accepted 2026-03-24 12:51:46,050 - lava_callback - INFO - Starting async job submit for node 69bfe5d27544ab5cd4a7f044 2026-03-24 12:51:46,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:51:46,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe5d27544ab5cd4a7f044 HTTP/1.1" 200 None 2026-03-24 12:51:46,076 - lava_callback - DEBUG - Processing node 69bfe5d27544ab5cd4a7f044: baseline-arm-clabbe 2026-03-24 12:51:46,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe5d27544ab5cd4a7f044 2026-03-24 12:51:46,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:51:46,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe5d27544ab5cd4a7f044 HTTP/1.1" 200 None 2026-03-24 12:51:46,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:51:46,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:51:46,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:51:46,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe5d27544ab5cd4a7f044 HTTP/1.1" 200 None 2026-03-24 12:51:46,132 - lava_callback - ERROR - Error processing callback for node 69bfe5d27544ab5cd4a7f044: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49270 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:52:26,024 - lava_callback - INFO - Received callback for node 69bfe5f67544ab5cd4a7f053 INFO: 172.20.0.1:49276 - "POST /node/69bfe5f67544ab5cd4a7f053 HTTP/1.0" 202 Accepted 2026-03-24 12:52:26,029 - lava_callback - INFO - Starting async job submit for node 69bfe5f67544ab5cd4a7f053 2026-03-24 12:52:26,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:52:26,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe5f67544ab5cd4a7f053 HTTP/1.1" 200 None 2026-03-24 12:52:26,047 - lava_callback - DEBUG - Processing node 69bfe5f67544ab5cd4a7f053: baseline-arm-clabbe 2026-03-24 12:52:26,047 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe5f67544ab5cd4a7f053 2026-03-24 12:52:26,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:52:26,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe5f67544ab5cd4a7f053 HTTP/1.1" 200 None 2026-03-24 12:52:26,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:52:26,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:52:26,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:52:26,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe5f67544ab5cd4a7f053 HTTP/1.1" 200 None 2026-03-24 12:52:26,110 - lava_callback - ERROR - Error processing callback for node 69bfe5f67544ab5cd4a7f053: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51884 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:52:52,598 - lava_callback - INFO - Received callback for node 69bfe6157544ab5cd4a7f063 INFO: 172.20.0.1:44536 - "POST /node/69bfe6157544ab5cd4a7f063 HTTP/1.0" 202 Accepted 2026-03-24 12:52:52,603 - lava_callback - INFO - Starting async job submit for node 69bfe6157544ab5cd4a7f063 2026-03-24 12:52:52,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:52:52,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6157544ab5cd4a7f063 HTTP/1.1" 200 None 2026-03-24 12:52:52,620 - lava_callback - DEBUG - Processing node 69bfe6157544ab5cd4a7f063: baseline-arm-baylibre 2026-03-24 12:52:52,621 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe6157544ab5cd4a7f063 2026-03-24 12:52:52,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:52:52,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6157544ab5cd4a7f063 HTTP/1.1" 200 None 2026-03-24 12:52:52,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:52:52,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 12:52:52,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:52:52,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe6157544ab5cd4a7f063 HTTP/1.1" 200 None 2026-03-24 12:52:52,679 - lava_callback - ERROR - Error processing callback for node 69bfe6157544ab5cd4a7f063: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54570 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:53:26,120 - lava_callback - INFO - Received callback for node 69bfe63d7544ab5cd4a7f071 INFO: 172.20.0.1:54584 - "POST /node/69bfe63d7544ab5cd4a7f071 HTTP/1.0" 202 Accepted 2026-03-24 12:53:26,126 - lava_callback - INFO - Starting async job submit for node 69bfe63d7544ab5cd4a7f071 2026-03-24 12:53:26,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:53:26,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe63d7544ab5cd4a7f071 HTTP/1.1" 200 None 2026-03-24 12:53:26,142 - lava_callback - DEBUG - Processing node 69bfe63d7544ab5cd4a7f071: baseline-arm-clabbe 2026-03-24 12:53:26,142 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe63d7544ab5cd4a7f071 2026-03-24 12:53:26,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:53:26,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe63d7544ab5cd4a7f071 HTTP/1.1" 200 None 2026-03-24 12:53:26,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:53:26,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:53:26,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:53:26,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe63d7544ab5cd4a7f071 HTTP/1.1" 200 None 2026-03-24 12:53:26,198 - lava_callback - ERROR - Error processing callback for node 69bfe63d7544ab5cd4a7f071: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48662 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:54:06,098 - lava_callback - INFO - Received callback for node 69bfe6617544ab5cd4a7f085 INFO: 172.20.0.1:54610 - "POST /node/69bfe6617544ab5cd4a7f085 HTTP/1.0" 202 Accepted 2026-03-24 12:54:06,104 - lava_callback - INFO - Starting async job submit for node 69bfe6617544ab5cd4a7f085 2026-03-24 12:54:06,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:54:06,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6617544ab5cd4a7f085 HTTP/1.1" 200 None 2026-03-24 12:54:06,133 - lava_callback - DEBUG - Processing node 69bfe6617544ab5cd4a7f085: baseline-arm-clabbe 2026-03-24 12:54:06,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe6617544ab5cd4a7f085 2026-03-24 12:54:06,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:54:06,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6617544ab5cd4a7f085 HTTP/1.1" 200 None 2026-03-24 12:54:06,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:54:06,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:54:06,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:54:06,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe6617544ab5cd4a7f085 HTTP/1.1" 200 None 2026-03-24 12:54:06,218 - lava_callback - ERROR - Error processing callback for node 69bfe6617544ab5cd4a7f085: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48672 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:54:32,622 - lava_callback - INFO - Received callback for node 69bfe6817544ab5cd4a7f08f INFO: 172.20.0.1:56668 - "POST /node/69bfe6817544ab5cd4a7f08f HTTP/1.0" 202 Accepted 2026-03-24 12:54:32,628 - lava_callback - INFO - Starting async job submit for node 69bfe6817544ab5cd4a7f08f 2026-03-24 12:54:32,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:54:32,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6817544ab5cd4a7f08f HTTP/1.1" 200 None 2026-03-24 12:54:32,644 - lava_callback - DEBUG - Processing node 69bfe6817544ab5cd4a7f08f: baseline-arm-baylibre 2026-03-24 12:54:32,644 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe6817544ab5cd4a7f08f 2026-03-24 12:54:32,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:54:32,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6817544ab5cd4a7f08f HTTP/1.1" 200 None 2026-03-24 12:54:32,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:54:32,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 12:54:32,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:54:32,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe6817544ab5cd4a7f08f HTTP/1.1" 200 None 2026-03-24 12:54:32,697 - lava_callback - ERROR - Error processing callback for node 69bfe6817544ab5cd4a7f08f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55824 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:55:26,220 - lava_callback - INFO - Received callback for node 69bfe6a97544ab5cd4a7f09e INFO: 172.20.0.1:55836 - "POST /node/69bfe6a97544ab5cd4a7f09e HTTP/1.0" 202 Accepted 2026-03-24 12:55:26,225 - lava_callback - INFO - Starting async job submit for node 69bfe6a97544ab5cd4a7f09e 2026-03-24 12:55:26,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:55:26,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6a97544ab5cd4a7f09e HTTP/1.1" 200 None 2026-03-24 12:55:26,240 - lava_callback - DEBUG - Processing node 69bfe6a97544ab5cd4a7f09e: baseline-arm-clabbe 2026-03-24 12:55:26,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe6a97544ab5cd4a7f09e 2026-03-24 12:55:26,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:55:26,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6a97544ab5cd4a7f09e HTTP/1.1" 200 None 2026-03-24 12:55:26,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:55:26,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:55:26,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:55:26,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe6a97544ab5cd4a7f09e HTTP/1.1" 200 None 2026-03-24 12:55:26,301 - lava_callback - ERROR - Error processing callback for node 69bfe6a97544ab5cd4a7f09e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53784 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:55:46,207 - lava_callback - INFO - Received callback for node 69bfe6cd7544ab5cd4a7f0a9 INFO: 172.20.0.1:46890 - "POST /node/69bfe6cd7544ab5cd4a7f0a9 HTTP/1.0" 202 Accepted 2026-03-24 12:55:46,213 - lava_callback - INFO - Starting async job submit for node 69bfe6cd7544ab5cd4a7f0a9 2026-03-24 12:55:46,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:55:46,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6cd7544ab5cd4a7f0a9 HTTP/1.1" 200 None 2026-03-24 12:55:46,241 - lava_callback - DEBUG - Processing node 69bfe6cd7544ab5cd4a7f0a9: baseline-arm-clabbe 2026-03-24 12:55:46,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe6cd7544ab5cd4a7f0a9 2026-03-24 12:55:46,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:55:46,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6cd7544ab5cd4a7f0a9 HTTP/1.1" 200 None 2026-03-24 12:55:46,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:55:46,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:55:46,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:55:46,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe6cd7544ab5cd4a7f0a9 HTTP/1.1" 200 None 2026-03-24 12:55:46,324 - lava_callback - ERROR - Error processing callback for node 69bfe6cd7544ab5cd4a7f0a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55078 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:56:26,231 - lava_callback - INFO - Received callback for node 69bfe6f17544ab5cd4a7f0b6 INFO: 172.20.0.1:55082 - "POST /node/69bfe6f17544ab5cd4a7f0b6 HTTP/1.0" 202 Accepted 2026-03-24 12:56:26,237 - lava_callback - INFO - Starting async job submit for node 69bfe6f17544ab5cd4a7f0b6 2026-03-24 12:56:26,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:56:26,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6f17544ab5cd4a7f0b6 HTTP/1.1" 200 None 2026-03-24 12:56:26,259 - lava_callback - DEBUG - Processing node 69bfe6f17544ab5cd4a7f0b6: baseline-arm-clabbe 2026-03-24 12:56:26,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe6f17544ab5cd4a7f0b6 2026-03-24 12:56:26,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:56:26,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe6f17544ab5cd4a7f0b6 HTTP/1.1" 200 None 2026-03-24 12:56:26,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:56:26,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:56:26,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:56:26,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe6f17544ab5cd4a7f0b6 HTTP/1.1" 200 None 2026-03-24 12:56:26,334 - lava_callback - ERROR - Error processing callback for node 69bfe6f17544ab5cd4a7f0b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47856 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:57:06,201 - lava_callback - INFO - Received callback for node 69bfe7157544ab5cd4a7f0ca INFO: 172.20.0.1:59258 - "POST /node/69bfe7157544ab5cd4a7f0ca HTTP/1.0" 202 Accepted 2026-03-24 12:57:06,207 - lava_callback - INFO - Starting async job submit for node 69bfe7157544ab5cd4a7f0ca 2026-03-24 12:57:06,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:57:06,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7157544ab5cd4a7f0ca HTTP/1.1" 200 None 2026-03-24 12:57:06,240 - lava_callback - DEBUG - Processing node 69bfe7157544ab5cd4a7f0ca: baseline-arm-clabbe 2026-03-24 12:57:06,240 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe7157544ab5cd4a7f0ca 2026-03-24 12:57:06,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:57:06,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7157544ab5cd4a7f0ca HTTP/1.1" 200 None 2026-03-24 12:57:06,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:57:06,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:57:06,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:57:06,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe7157544ab5cd4a7f0ca HTTP/1.1" 200 None 2026-03-24 12:57:06,311 - lava_callback - ERROR - Error processing callback for node 69bfe7157544ab5cd4a7f0ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45926 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:57:46,319 - lava_callback - INFO - Received callback for node 69bfe7397544ab5cd4a7f0d7 INFO: 172.20.0.1:37132 - "POST /node/69bfe7397544ab5cd4a7f0d7 HTTP/1.0" 202 Accepted 2026-03-24 12:57:46,325 - lava_callback - INFO - Starting async job submit for node 69bfe7397544ab5cd4a7f0d7 2026-03-24 12:57:46,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:57:46,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7397544ab5cd4a7f0d7 HTTP/1.1" 200 None 2026-03-24 12:57:46,341 - lava_callback - DEBUG - Processing node 69bfe7397544ab5cd4a7f0d7: baseline-arm-clabbe 2026-03-24 12:57:46,342 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe7397544ab5cd4a7f0d7 2026-03-24 12:57:46,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:57:46,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7397544ab5cd4a7f0d7 HTTP/1.1" 200 None 2026-03-24 12:57:46,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:57:46,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:57:46,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:57:46,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe7397544ab5cd4a7f0d7 HTTP/1.1" 200 None 2026-03-24 12:57:46,413 - lava_callback - ERROR - Error processing callback for node 69bfe7397544ab5cd4a7f0d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55690 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:58:12,845 - lava_callback - INFO - Received callback for node 69bfe7597544ab5cd4a7f0e4 INFO: 172.20.0.1:51774 - "POST /node/69bfe7597544ab5cd4a7f0e4 HTTP/1.0" 202 Accepted 2026-03-24 12:58:12,851 - lava_callback - INFO - Starting async job submit for node 69bfe7597544ab5cd4a7f0e4 2026-03-24 12:58:12,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:58:12,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7597544ab5cd4a7f0e4 HTTP/1.1" 200 None 2026-03-24 12:58:12,877 - lava_callback - DEBUG - Processing node 69bfe7597544ab5cd4a7f0e4: baseline-arm-baylibre 2026-03-24 12:58:12,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe7597544ab5cd4a7f0e4 2026-03-24 12:58:12,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:58:12,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7597544ab5cd4a7f0e4 HTTP/1.1" 200 None 2026-03-24 12:58:12,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:58:12,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 12:58:12,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:58:12,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe7597544ab5cd4a7f0e4 HTTP/1.1" 200 None 2026-03-24 12:58:12,950 - lava_callback - ERROR - Error processing callback for node 69bfe7597544ab5cd4a7f0e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38668 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:58:46,292 - lava_callback - INFO - Received callback for node 69bfe7817544ab5cd4a7f0f6 INFO: 172.20.0.1:48616 - "POST /node/69bfe7817544ab5cd4a7f0f6 HTTP/1.0" 202 Accepted 2026-03-24 12:58:46,298 - lava_callback - INFO - Starting async job submit for node 69bfe7817544ab5cd4a7f0f6 2026-03-24 12:58:46,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:58:46,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7817544ab5cd4a7f0f6 HTTP/1.1" 200 None 2026-03-24 12:58:46,314 - lava_callback - DEBUG - Processing node 69bfe7817544ab5cd4a7f0f6: baseline-arm-clabbe 2026-03-24 12:58:46,315 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe7817544ab5cd4a7f0f6 2026-03-24 12:58:46,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:58:46,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7817544ab5cd4a7f0f6 HTTP/1.1" 200 None 2026-03-24 12:58:46,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:58:46,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:58:46,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:58:46,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe7817544ab5cd4a7f0f6 HTTP/1.1" 200 None 2026-03-24 12:58:46,413 - lava_callback - ERROR - Error processing callback for node 69bfe7817544ab5cd4a7f0f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46982 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:59:26,291 - lava_callback - INFO - Received callback for node 69bfe7a67544ab5cd4a7f106 INFO: 172.20.0.1:46984 - "POST /node/69bfe7a67544ab5cd4a7f106 HTTP/1.0" 202 Accepted 2026-03-24 12:59:26,297 - lava_callback - INFO - Starting async job submit for node 69bfe7a67544ab5cd4a7f106 2026-03-24 12:59:26,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:59:26,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7a67544ab5cd4a7f106 HTTP/1.1" 200 None 2026-03-24 12:59:26,328 - lava_callback - DEBUG - Processing node 69bfe7a67544ab5cd4a7f106: baseline-arm-clabbe 2026-03-24 12:59:26,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe7a67544ab5cd4a7f106 2026-03-24 12:59:26,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:59:26,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7a67544ab5cd4a7f106 HTTP/1.1" 200 None 2026-03-24 12:59:26,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:59:26,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 12:59:26,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:59:26,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe7a67544ab5cd4a7f106 HTTP/1.1" 200 None 2026-03-24 12:59:26,381 - lava_callback - ERROR - Error processing callback for node 69bfe7a67544ab5cd4a7f106: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37498 - "GET / HTTP/1.0" 200 OK 2026-03-24 12:59:52,813 - lava_callback - INFO - Received callback for node 69bfe7c67544ab5cd4a7f114 INFO: 172.20.0.1:40722 - "POST /node/69bfe7c67544ab5cd4a7f114 HTTP/1.0" 202 Accepted 2026-03-24 12:59:52,819 - lava_callback - INFO - Starting async job submit for node 69bfe7c67544ab5cd4a7f114 2026-03-24 12:59:52,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:59:52,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7c67544ab5cd4a7f114 HTTP/1.1" 200 None 2026-03-24 12:59:52,835 - lava_callback - DEBUG - Processing node 69bfe7c67544ab5cd4a7f114: baseline-arm-baylibre 2026-03-24 12:59:52,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe7c67544ab5cd4a7f114 2026-03-24 12:59:52,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:59:52,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7c67544ab5cd4a7f114 HTTP/1.1" 200 None 2026-03-24 12:59:52,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:59:52,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 12:59:52,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 12:59:52,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe7c67544ab5cd4a7f114 HTTP/1.1" 200 None 2026-03-24 12:59:52,915 - lava_callback - ERROR - Error processing callback for node 69bfe7c67544ab5cd4a7f114: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39792 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:00:32,803 - lava_callback - INFO - Received callback for node 69bfe7eb7544ab5cd4a7f11d INFO: 172.20.0.1:39620 - "POST /node/69bfe7eb7544ab5cd4a7f11d HTTP/1.0" 202 Accepted 2026-03-24 13:00:32,809 - lava_callback - INFO - Starting async job submit for node 69bfe7eb7544ab5cd4a7f11d 2026-03-24 13:00:32,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:00:32,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7eb7544ab5cd4a7f11d HTTP/1.1" 200 None 2026-03-24 13:00:32,836 - lava_callback - DEBUG - Processing node 69bfe7eb7544ab5cd4a7f11d: baseline-arm-baylibre 2026-03-24 13:00:32,836 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe7eb7544ab5cd4a7f11d 2026-03-24 13:00:32,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:00:32,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe7eb7544ab5cd4a7f11d HTTP/1.1" 200 None 2026-03-24 13:00:32,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:00:32,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 13:00:32,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:00:32,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe7eb7544ab5cd4a7f11d HTTP/1.1" 200 None 2026-03-24 13:00:32,901 - lava_callback - ERROR - Error processing callback for node 69bfe7eb7544ab5cd4a7f11d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60420 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:01:26,447 - lava_callback - INFO - Received callback for node 69bfe8107544ab5cd4a7f128 INFO: 172.20.0.1:60430 - "POST /node/69bfe8107544ab5cd4a7f128 HTTP/1.0" 202 Accepted 2026-03-24 13:01:26,452 - lava_callback - INFO - Starting async job submit for node 69bfe8107544ab5cd4a7f128 2026-03-24 13:01:26,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:01:26,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe8107544ab5cd4a7f128 HTTP/1.1" 200 None 2026-03-24 13:01:26,473 - lava_callback - DEBUG - Processing node 69bfe8107544ab5cd4a7f128: baseline-arm-clabbe 2026-03-24 13:01:26,474 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe8107544ab5cd4a7f128 2026-03-24 13:01:26,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:60434 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:01:26,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe8107544ab5cd4a7f128 HTTP/1.1" 200 None 2026-03-24 13:01:26,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:01:26,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-24 13:01:26,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:01:26,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe8107544ab5cd4a7f128 HTTP/1.1" 200 None 2026-03-24 13:01:26,526 - lava_callback - ERROR - Error processing callback for node 69bfe8107544ab5cd4a7f128: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49316 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:01:52,920 - lava_callback - INFO - Received callback for node 69bfe8327544ab5cd4a7f134 INFO: 172.20.0.1:33236 - "POST /node/69bfe8327544ab5cd4a7f134 HTTP/1.0" 202 Accepted 2026-03-24 13:01:52,926 - lava_callback - INFO - Starting async job submit for node 69bfe8327544ab5cd4a7f134 2026-03-24 13:01:52,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:01:52,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe8327544ab5cd4a7f134 HTTP/1.1" 200 None 2026-03-24 13:01:52,988 - lava_callback - DEBUG - Processing node 69bfe8327544ab5cd4a7f134: baseline-arm-baylibre 2026-03-24 13:01:52,988 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe8327544ab5cd4a7f134 2026-03-24 13:01:52,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:01:53,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe8327544ab5cd4a7f134 HTTP/1.1" 200 None 2026-03-24 13:01:53,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:01:53,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 13:01:53,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:01:53,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe8327544ab5cd4a7f134 HTTP/1.1" 200 None 2026-03-24 13:01:53,072 - lava_callback - ERROR - Error processing callback for node 69bfe8327544ab5cd4a7f134: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49632 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:02:26,444 - lava_callback - INFO - Received callback for node 69bfe8577544ab5cd4a7f13f INFO: 172.20.0.1:49642 - "POST /node/69bfe8577544ab5cd4a7f13f HTTP/1.0" 202 Accepted 2026-03-24 13:02:26,449 - lava_callback - INFO - Starting async job submit for node 69bfe8577544ab5cd4a7f13f 2026-03-24 13:02:26,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:02:26,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe8577544ab5cd4a7f13f HTTP/1.1" 200 None 2026-03-24 13:02:26,480 - lava_callback - DEBUG - Processing node 69bfe8577544ab5cd4a7f13f: baseline-arm-clabbe 2026-03-24 13:02:26,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe8577544ab5cd4a7f13f 2026-03-24 13:02:26,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:49652 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:02:26,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe8577544ab5cd4a7f13f HTTP/1.1" 200 None 2026-03-24 13:02:26,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:02:26,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 13:02:26,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:02:26,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe8577544ab5cd4a7f13f HTTP/1.1" 200 None 2026-03-24 13:02:26,552 - lava_callback - ERROR - Error processing callback for node 69bfe8577544ab5cd4a7f13f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54902 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:03:06,437 - lava_callback - INFO - Received callback for node 69bfe87b7544ab5cd4a7f14f INFO: 172.20.0.1:34368 - "POST /node/69bfe87b7544ab5cd4a7f14f HTTP/1.0" 202 Accepted 2026-03-24 13:03:06,442 - lava_callback - INFO - Starting async job submit for node 69bfe87b7544ab5cd4a7f14f 2026-03-24 13:03:06,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:03:06,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe87b7544ab5cd4a7f14f HTTP/1.1" 200 None 2026-03-24 13:03:06,478 - lava_callback - DEBUG - Processing node 69bfe87b7544ab5cd4a7f14f: baseline-arm-clabbe 2026-03-24 13:03:06,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe87b7544ab5cd4a7f14f INFO: 172.20.0.1:34372 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:03:06,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:03:06,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe87b7544ab5cd4a7f14f HTTP/1.1" 200 None 2026-03-24 13:03:06,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:03:06,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 13:03:06,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:03:06,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe87b7544ab5cd4a7f14f HTTP/1.1" 200 None 2026-03-24 13:03:06,535 - lava_callback - ERROR - Error processing callback for node 69bfe87b7544ab5cd4a7f14f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47368 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:03:46,505 - lava_callback - INFO - Received callback for node 69bfe89e7544ab5cd4a7f157 INFO: 172.20.0.1:47370 - "POST /node/69bfe89e7544ab5cd4a7f157 HTTP/1.0" 202 Accepted 2026-03-24 13:03:46,510 - lava_callback - INFO - Starting async job submit for node 69bfe89e7544ab5cd4a7f157 2026-03-24 13:03:46,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:03:46,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe89e7544ab5cd4a7f157 HTTP/1.1" 200 None 2026-03-24 13:03:46,561 - lava_callback - DEBUG - Processing node 69bfe89e7544ab5cd4a7f157: baseline-arm-clabbe 2026-03-24 13:03:46,561 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe89e7544ab5cd4a7f157 2026-03-24 13:03:46,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:03:46,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe89e7544ab5cd4a7f157 HTTP/1.1" 200 None 2026-03-24 13:03:46,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:03:46,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 13:03:46,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:03:46,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe89e7544ab5cd4a7f157 HTTP/1.1" 200 None 2026-03-24 13:03:46,640 - lava_callback - ERROR - Error processing callback for node 69bfe89e7544ab5cd4a7f157: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37386 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:04:06,452 - lava_callback - INFO - Received callback for node 69bfe8c37544ab5cd4a7f163 INFO: 172.20.0.1:47448 - "POST /node/69bfe8c37544ab5cd4a7f163 HTTP/1.0" 202 Accepted 2026-03-24 13:04:06,457 - lava_callback - INFO - Starting async job submit for node 69bfe8c37544ab5cd4a7f163 2026-03-24 13:04:06,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:04:06,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe8c37544ab5cd4a7f163 HTTP/1.1" 200 None 2026-03-24 13:04:06,495 - lava_callback - DEBUG - Processing node 69bfe8c37544ab5cd4a7f163: baseline-arm-clabbe 2026-03-24 13:04:06,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe8c37544ab5cd4a7f163 2026-03-24 13:04:06,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:47454 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:04:06,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe8c37544ab5cd4a7f163 HTTP/1.1" 200 None 2026-03-24 13:04:06,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:04:06,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 13:04:06,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:04:06,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe8c37544ab5cd4a7f163 HTTP/1.1" 200 None 2026-03-24 13:04:06,574 - lava_callback - ERROR - Error processing callback for node 69bfe8c37544ab5cd4a7f163: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55924 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:04:46,476 - lava_callback - INFO - Received callback for node 69bfe8e77544ab5cd4a7f171 INFO: 172.20.0.1:55926 - "POST /node/69bfe8e77544ab5cd4a7f171 HTTP/1.0" 202 Accepted 2026-03-24 13:04:46,482 - lava_callback - INFO - Starting async job submit for node 69bfe8e77544ab5cd4a7f171 2026-03-24 13:04:46,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:04:46,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe8e77544ab5cd4a7f171 HTTP/1.1" 200 None 2026-03-24 13:04:46,499 - lava_callback - DEBUG - Processing node 69bfe8e77544ab5cd4a7f171: baseline-arm-clabbe 2026-03-24 13:04:46,499 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfe8e77544ab5cd4a7f171 2026-03-24 13:04:46,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:04:46,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfe8e77544ab5cd4a7f171 HTTP/1.1" 200 None 2026-03-24 13:04:46,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:04:46,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e96f113612b7716b9dee HTTP/1.1" 200 None 2026-03-24 13:04:46,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:04:46,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfe8e77544ab5cd4a7f171 HTTP/1.1" 200 None 2026-03-24 13:04:46,560 - lava_callback - ERROR - Error processing callback for node 69bfe8e77544ab5cd4a7f171: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40536 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50186 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:09:17,334 - lava_callback - INFO - Received callback for node 69c17ec02b2b51e9ad97a953 INFO: 172.20.0.1:50192 - "POST /node/69c17ec02b2b51e9ad97a953 HTTP/1.0" 202 Accepted 2026-03-24 13:09:17,340 - lava_callback - INFO - Starting async job submit for node 69c17ec02b2b51e9ad97a953 2026-03-24 13:09:17,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:09:17,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c17ec02b2b51e9ad97a953 HTTP/1.1" 200 None 2026-03-24 13:09:17,385 - lava_callback - DEBUG - Processing node 69c17ec02b2b51e9ad97a953: baseline-arm64-broonie 2026-03-24 13:09:17,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 13:09:17,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 13:09:17,799 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c17ec02b2b51e9ad97a953/log.txt.gz 2026-03-24 13:09:17,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 13:09:18,150 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 13:09:18,152 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c17ec02b2b51e9ad97a953/lava-logs.yaml 2026-03-24 13:09:18,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 13:09:18,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 13:09:18,309 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c17ec02b2b51e9ad97a953/lava_callback.json.gz 2026-03-24 13:09:18,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:09:18,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c17ec02b2b51e9ad97a953 HTTP/1.1" 200 None 2026-03-24 13:09:18,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:09:18,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-24 13:09:18,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:09:18,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c17ec02b2b51e9ad97a953 HTTP/1.1" 200 None 2026-03-24 13:09:18,400 - lava_callback - INFO - Completed processing callback for node 69c17ec02b2b51e9ad97a953 INFO: 172.20.0.1:57686 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:09:25,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:09:25,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48384 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:09:37,749 - lava_callback - INFO - Received callback for node 69c251834a2c5046eb8802e0 INFO: 172.20.0.1:48398 - "POST /node/69c251834a2c5046eb8802e0 HTTP/1.0" 202 Accepted 2026-03-24 13:09:37,755 - lava_callback - INFO - Starting async job submit for node 69c251834a2c5046eb8802e0 2026-03-24 13:09:37,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:09:37,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251834a2c5046eb8802e0 HTTP/1.1" 200 None 2026-03-24 13:09:37,792 - lava_callback - DEBUG - Processing node 69c251834a2c5046eb8802e0: baseline-arm64-broonie 2026-03-24 13:09:38,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 13:09:38,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 13:09:38,150 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251834a2c5046eb8802e0/log.txt.gz 2026-03-24 13:09:38,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 13:09:38,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 13:09:38,492 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251834a2c5046eb8802e0/lava-logs.yaml 2026-03-24 13:09:38,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-24 13:09:38,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-24 13:09:38,666 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c251834a2c5046eb8802e0/lava_callback.json.gz 2026-03-24 13:09:38,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:09:38,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c251834a2c5046eb8802e0 HTTP/1.1" 200 None 2026-03-24 13:09:38,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:09:38,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c248e54a2c5046eb87f89f HTTP/1.1" 200 None 2026-03-24 13:09:38,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:09:38,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c251834a2c5046eb8802e0 HTTP/1.1" 200 None 2026-03-24 13:09:38,749 - lava_callback - INFO - Completed processing callback for node 69c251834a2c5046eb8802e0 INFO: 172.20.0.1:55376 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:09:55,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:09:55,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54038 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:24:14,066 - lava_callback - INFO - Received callback for node 69bfed6a7544ab5cd4a7f20c INFO: 172.20.0.1:44744 - "POST /node/69bfed6a7544ab5cd4a7f20c HTTP/1.0" 202 Accepted 2026-03-24 13:24:14,071 - lava_callback - INFO - Starting async job submit for node 69bfed6a7544ab5cd4a7f20c 2026-03-24 13:24:14,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:24:14,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfed6a7544ab5cd4a7f20c HTTP/1.1" 200 None 2026-03-24 13:24:14,088 - lava_callback - DEBUG - Processing node 69bfed6a7544ab5cd4a7f20c: baseline-arm64-baylibre 2026-03-24 13:24:14,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfed6a7544ab5cd4a7f20c 2026-03-24 13:24:14,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:24:14,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfed6a7544ab5cd4a7f20c HTTP/1.1" 200 None 2026-03-24 13:24:14,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:24:14,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 13:24:14,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:24:14,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfed6a7544ab5cd4a7f20c HTTP/1.1" 200 None 2026-03-24 13:24:14,232 - lava_callback - ERROR - Error processing callback for node 69bfed6a7544ab5cd4a7f20c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45182 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:25:27,417 - lava_callback - INFO - Received callback for node 69bfedba7544ab5cd4a7f219 INFO: 172.20.0.1:45186 - "POST /node/69bfedba7544ab5cd4a7f219 HTTP/1.0" 202 Accepted 2026-03-24 13:25:27,423 - lava_callback - INFO - Starting async job submit for node 69bfedba7544ab5cd4a7f219 2026-03-24 13:25:27,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:25:27,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfedba7544ab5cd4a7f219 HTTP/1.1" 200 None 2026-03-24 13:25:27,459 - lava_callback - DEBUG - Processing node 69bfedba7544ab5cd4a7f219: baseline-arm64-clabbe 2026-03-24 13:25:27,460 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfedba7544ab5cd4a7f219 2026-03-24 13:25:27,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:25:27,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfedba7544ab5cd4a7f219 HTTP/1.1" 200 None 2026-03-24 13:25:27,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:25:27,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 13:25:27,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:25:27,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfedba7544ab5cd4a7f219 HTTP/1.1" 200 None 2026-03-24 13:25:27,529 - lava_callback - ERROR - Error processing callback for node 69bfedba7544ab5cd4a7f219: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40496 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:26:47,412 - lava_callback - INFO - Received callback for node 69bfee077544ab5cd4a7f22c INFO: 172.20.0.1:40498 - "POST /node/69bfee077544ab5cd4a7f22c HTTP/1.0" 202 Accepted 2026-03-24 13:26:47,417 - lava_callback - INFO - Starting async job submit for node 69bfee077544ab5cd4a7f22c 2026-03-24 13:26:47,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:26:47,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfee077544ab5cd4a7f22c HTTP/1.1" 200 None 2026-03-24 13:26:47,434 - lava_callback - DEBUG - Processing node 69bfee077544ab5cd4a7f22c: baseline-arm64-clabbe 2026-03-24 13:26:47,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfee077544ab5cd4a7f22c 2026-03-24 13:26:47,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:26:47,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfee077544ab5cd4a7f22c HTTP/1.1" 200 None 2026-03-24 13:26:47,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:26:47,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 13:26:47,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:26:47,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfee077544ab5cd4a7f22c HTTP/1.1" 200 None 2026-03-24 13:26:47,501 - lava_callback - ERROR - Error processing callback for node 69bfee077544ab5cd4a7f22c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52332 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:28:07,511 - lava_callback - INFO - Received callback for node 69bfee5f7544ab5cd4a7f23d INFO: 172.20.0.1:52342 - "POST /node/69bfee5f7544ab5cd4a7f23d HTTP/1.0" 202 Accepted 2026-03-24 13:28:07,519 - lava_callback - INFO - Starting async job submit for node 69bfee5f7544ab5cd4a7f23d 2026-03-24 13:28:07,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:28:07,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfee5f7544ab5cd4a7f23d HTTP/1.1" 200 None 2026-03-24 13:28:07,536 - lava_callback - DEBUG - Processing node 69bfee5f7544ab5cd4a7f23d: baseline-arm64-clabbe 2026-03-24 13:28:07,536 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfee5f7544ab5cd4a7f23d 2026-03-24 13:28:07,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:28:07,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfee5f7544ab5cd4a7f23d HTTP/1.1" 200 None 2026-03-24 13:28:07,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:28:07,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 13:28:07,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:28:07,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfee5f7544ab5cd4a7f23d HTTP/1.1" 200 None 2026-03-24 13:28:07,596 - lava_callback - ERROR - Error processing callback for node 69bfee5f7544ab5cd4a7f23d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52246 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:29:27,574 - lava_callback - INFO - Received callback for node 69bfeeab7544ab5cd4a7f24a INFO: 172.20.0.1:52258 - "POST /node/69bfeeab7544ab5cd4a7f24a HTTP/1.0" 202 Accepted 2026-03-24 13:29:27,580 - lava_callback - INFO - Starting async job submit for node 69bfeeab7544ab5cd4a7f24a 2026-03-24 13:29:27,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:29:27,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfeeab7544ab5cd4a7f24a HTTP/1.1" 200 None 2026-03-24 13:29:27,612 - lava_callback - DEBUG - Processing node 69bfeeab7544ab5cd4a7f24a: baseline-arm64-clabbe 2026-03-24 13:29:27,612 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfeeab7544ab5cd4a7f24a 2026-03-24 13:29:27,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:29:27,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfeeab7544ab5cd4a7f24a HTTP/1.1" 200 None 2026-03-24 13:29:27,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:29:27,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 13:29:27,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:29:27,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfeeab7544ab5cd4a7f24a HTTP/1.1" 200 None 2026-03-24 13:29:27,674 - lava_callback - ERROR - Error processing callback for node 69bfeeab7544ab5cd4a7f24a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55090 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:30:47,563 - lava_callback - INFO - Received callback for node 69bfeef77544ab5cd4a7f25d INFO: 172.20.0.1:55102 - "POST /node/69bfeef77544ab5cd4a7f25d HTTP/1.0" 202 Accepted 2026-03-24 13:30:47,569 - lava_callback - INFO - Starting async job submit for node 69bfeef77544ab5cd4a7f25d 2026-03-24 13:30:47,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:30:47,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfeef77544ab5cd4a7f25d HTTP/1.1" 200 None 2026-03-24 13:30:47,585 - lava_callback - DEBUG - Processing node 69bfeef77544ab5cd4a7f25d: baseline-arm64-clabbe 2026-03-24 13:30:47,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfeef77544ab5cd4a7f25d 2026-03-24 13:30:47,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:30:47,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfeef77544ab5cd4a7f25d HTTP/1.1" 200 None 2026-03-24 13:30:47,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:30:47,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 13:30:47,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:30:47,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfeef77544ab5cd4a7f25d HTTP/1.1" 200 None 2026-03-24 13:30:47,641 - lava_callback - ERROR - Error processing callback for node 69bfeef77544ab5cd4a7f25d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39936 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:32:07,645 - lava_callback - INFO - Received callback for node 69bfef417544ab5cd4a7f26c INFO: 172.20.0.1:39938 - "POST /node/69bfef417544ab5cd4a7f26c HTTP/1.0" 202 Accepted 2026-03-24 13:32:07,651 - lava_callback - INFO - Starting async job submit for node 69bfef417544ab5cd4a7f26c 2026-03-24 13:32:07,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:32:07,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfef417544ab5cd4a7f26c HTTP/1.1" 200 None 2026-03-24 13:32:07,673 - lava_callback - DEBUG - Processing node 69bfef417544ab5cd4a7f26c: baseline-arm64-clabbe 2026-03-24 13:32:07,673 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfef417544ab5cd4a7f26c 2026-03-24 13:32:07,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:32:07,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfef417544ab5cd4a7f26c HTTP/1.1" 200 None 2026-03-24 13:32:07,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:32:07,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-24 13:32:07,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:32:07,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfef417544ab5cd4a7f26c HTTP/1.1" 200 None 2026-03-24 13:32:07,751 - lava_callback - ERROR - Error processing callback for node 69bfef417544ab5cd4a7f26c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38042 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:41:35,252 - lava_callback - INFO - Received callback for node 69bff17b7544ab5cd4a7f281 INFO: 172.20.0.1:50750 - "POST /node/69bff17b7544ab5cd4a7f281 HTTP/1.0" 202 Accepted 2026-03-24 13:41:35,259 - lava_callback - INFO - Starting async job submit for node 69bff17b7544ab5cd4a7f281 2026-03-24 13:41:35,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:41:35,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff17b7544ab5cd4a7f281 HTTP/1.1" 200 None 2026-03-24 13:41:35,279 - lava_callback - DEBUG - Processing node 69bff17b7544ab5cd4a7f281: baseline-arm64-baylibre 2026-03-24 13:41:35,279 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff17b7544ab5cd4a7f281 2026-03-24 13:41:35,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:41:35,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff17b7544ab5cd4a7f281 HTTP/1.1" 200 None 2026-03-24 13:41:35,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:41:35,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:41:35,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:41:35,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff17b7544ab5cd4a7f281 HTTP/1.1" 200 None 2026-03-24 13:41:35,336 - lava_callback - ERROR - Error processing callback for node 69bff17b7544ab5cd4a7f281: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32836 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:42:35,300 - lava_callback - INFO - Received callback for node 69bff1c67544ab5cd4a7f295 INFO: 172.20.0.1:54286 - "POST /node/69bff1c67544ab5cd4a7f295 HTTP/1.0" 202 Accepted 2026-03-24 13:42:35,306 - lava_callback - INFO - Starting async job submit for node 69bff1c67544ab5cd4a7f295 2026-03-24 13:42:35,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:42:35,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff1c67544ab5cd4a7f295 HTTP/1.1" 200 None 2026-03-24 13:42:35,332 - lava_callback - DEBUG - Processing node 69bff1c67544ab5cd4a7f295: baseline-arm64-baylibre 2026-03-24 13:42:35,332 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff1c67544ab5cd4a7f295 2026-03-24 13:42:35,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:42:35,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff1c67544ab5cd4a7f295 HTTP/1.1" 200 None 2026-03-24 13:42:35,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:42:35,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:42:35,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:42:35,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff1c67544ab5cd4a7f295 HTTP/1.1" 200 None 2026-03-24 13:42:35,420 - lava_callback - ERROR - Error processing callback for node 69bff1c67544ab5cd4a7f295: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56630 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:44:08,189 - lava_callback - INFO - Received callback for node 69bff2167544ab5cd4a7f2b3 INFO: 172.20.0.1:56638 - "POST /node/69bff2167544ab5cd4a7f2b3 HTTP/1.0" 202 Accepted 2026-03-24 13:44:08,195 - lava_callback - INFO - Starting async job submit for node 69bff2167544ab5cd4a7f2b3 2026-03-24 13:44:08,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:44:08,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff2167544ab5cd4a7f2b3 HTTP/1.1" 200 None 2026-03-24 13:44:08,210 - lava_callback - DEBUG - Processing node 69bff2167544ab5cd4a7f2b3: baseline-arm64-clabbe 2026-03-24 13:44:08,210 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff2167544ab5cd4a7f2b3 2026-03-24 13:44:08,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:44:08,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff2167544ab5cd4a7f2b3 HTTP/1.1" 200 None 2026-03-24 13:44:08,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:44:08,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:44:08,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:44:08,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff2167544ab5cd4a7f2b3 HTTP/1.1" 200 None 2026-03-24 13:44:08,277 - lava_callback - ERROR - Error processing callback for node 69bff2167544ab5cd4a7f2b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60156 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:45:08,290 - lava_callback - INFO - Received callback for node 69bff2617544ab5cd4a7f2be INFO: 172.20.0.1:60168 - "POST /node/69bff2617544ab5cd4a7f2be HTTP/1.0" 202 Accepted 2026-03-24 13:45:08,296 - lava_callback - INFO - Starting async job submit for node 69bff2617544ab5cd4a7f2be 2026-03-24 13:45:08,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:45:08,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff2617544ab5cd4a7f2be HTTP/1.1" 200 None 2026-03-24 13:45:08,313 - lava_callback - DEBUG - Processing node 69bff2617544ab5cd4a7f2be: baseline-arm64-clabbe 2026-03-24 13:45:08,313 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff2617544ab5cd4a7f2be 2026-03-24 13:45:08,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:45:08,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff2617544ab5cd4a7f2be HTTP/1.1" 200 None 2026-03-24 13:45:08,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:45:08,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:45:08,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:45:08,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff2617544ab5cd4a7f2be HTTP/1.1" 200 None 2026-03-24 13:45:08,378 - lava_callback - ERROR - Error processing callback for node 69bff2617544ab5cd4a7f2be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44082 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:46:28,286 - lava_callback - INFO - Received callback for node 69bff2ad7544ab5cd4a7f2dc INFO: 172.20.0.1:44092 - "POST /node/69bff2ad7544ab5cd4a7f2dc HTTP/1.0" 202 Accepted 2026-03-24 13:46:28,292 - lava_callback - INFO - Starting async job submit for node 69bff2ad7544ab5cd4a7f2dc 2026-03-24 13:46:28,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:46:28,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff2ad7544ab5cd4a7f2dc HTTP/1.1" 200 None 2026-03-24 13:46:28,432 - lava_callback - DEBUG - Processing node 69bff2ad7544ab5cd4a7f2dc: baseline-arm64-clabbe 2026-03-24 13:46:28,432 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff2ad7544ab5cd4a7f2dc 2026-03-24 13:46:28,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:46:28,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff2ad7544ab5cd4a7f2dc HTTP/1.1" 200 None 2026-03-24 13:46:28,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:46:28,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:46:28,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:46:28,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff2ad7544ab5cd4a7f2dc HTTP/1.1" 200 None 2026-03-24 13:46:28,483 - lava_callback - ERROR - Error processing callback for node 69bff2ad7544ab5cd4a7f2dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38428 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:47:48,383 - lava_callback - INFO - Received callback for node 69bff2f87544ab5cd4a7f2e7 INFO: 172.20.0.1:38434 - "POST /node/69bff2f87544ab5cd4a7f2e7 HTTP/1.0" 202 Accepted 2026-03-24 13:47:48,389 - lava_callback - INFO - Starting async job submit for node 69bff2f87544ab5cd4a7f2e7 2026-03-24 13:47:48,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:47:48,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff2f87544ab5cd4a7f2e7 HTTP/1.1" 200 None 2026-03-24 13:47:48,404 - lava_callback - DEBUG - Processing node 69bff2f87544ab5cd4a7f2e7: baseline-arm64-clabbe 2026-03-24 13:47:48,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff2f87544ab5cd4a7f2e7 2026-03-24 13:47:48,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:47:48,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff2f87544ab5cd4a7f2e7 HTTP/1.1" 200 None 2026-03-24 13:47:48,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:47:48,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:47:48,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:47:48,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff2f87544ab5cd4a7f2e7 HTTP/1.1" 200 None 2026-03-24 13:47:48,455 - lava_callback - ERROR - Error processing callback for node 69bff2f87544ab5cd4a7f2e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54276 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:49:08,388 - lava_callback - INFO - Received callback for node 69bff3447544ab5cd4a7f301 INFO: 172.20.0.1:54286 - "POST /node/69bff3447544ab5cd4a7f301 HTTP/1.0" 202 Accepted 2026-03-24 13:49:08,395 - lava_callback - INFO - Starting async job submit for node 69bff3447544ab5cd4a7f301 2026-03-24 13:49:08,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:49:08,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff3447544ab5cd4a7f301 HTTP/1.1" 200 None 2026-03-24 13:49:08,415 - lava_callback - DEBUG - Processing node 69bff3447544ab5cd4a7f301: baseline-arm64-clabbe 2026-03-24 13:49:08,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff3447544ab5cd4a7f301 2026-03-24 13:49:08,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:49:08,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff3447544ab5cd4a7f301 HTTP/1.1" 200 None 2026-03-24 13:49:08,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:49:08,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:49:08,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:49:08,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff3447544ab5cd4a7f301 HTTP/1.1" 200 None 2026-03-24 13:49:08,481 - lava_callback - ERROR - Error processing callback for node 69bff3447544ab5cd4a7f301: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50854 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:50:28,472 - lava_callback - INFO - Received callback for node 69bff3917544ab5cd4a7f308 INFO: 172.20.0.1:50858 - "POST /node/69bff3917544ab5cd4a7f308 HTTP/1.0" 202 Accepted 2026-03-24 13:50:28,478 - lava_callback - INFO - Starting async job submit for node 69bff3917544ab5cd4a7f308 2026-03-24 13:50:28,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:50:28,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff3917544ab5cd4a7f308 HTTP/1.1" 200 None 2026-03-24 13:50:28,494 - lava_callback - DEBUG - Processing node 69bff3917544ab5cd4a7f308: baseline-arm64-clabbe 2026-03-24 13:50:28,494 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff3917544ab5cd4a7f308 2026-03-24 13:50:28,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:50:28,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff3917544ab5cd4a7f308 HTTP/1.1" 200 None 2026-03-24 13:50:28,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:50:28,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:50:28,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:50:28,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff3917544ab5cd4a7f308 HTTP/1.1" 200 None 2026-03-24 13:50:28,570 - lava_callback - ERROR - Error processing callback for node 69bff3917544ab5cd4a7f308: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60064 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:51:28,480 - lava_callback - INFO - Received callback for node 69bff3dd7544ab5cd4a7f31f INFO: 172.20.0.1:60074 - "POST /node/69bff3dd7544ab5cd4a7f31f HTTP/1.0" 202 Accepted 2026-03-24 13:51:28,486 - lava_callback - INFO - Starting async job submit for node 69bff3dd7544ab5cd4a7f31f 2026-03-24 13:51:28,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:51:28,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff3dd7544ab5cd4a7f31f HTTP/1.1" 200 None 2026-03-24 13:51:28,501 - lava_callback - DEBUG - Processing node 69bff3dd7544ab5cd4a7f31f: baseline-arm64-clabbe 2026-03-24 13:51:28,502 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff3dd7544ab5cd4a7f31f 2026-03-24 13:51:28,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:51:28,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff3dd7544ab5cd4a7f31f HTTP/1.1" 200 None 2026-03-24 13:51:28,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:51:28,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:51:28,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:51:28,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff3dd7544ab5cd4a7f31f HTTP/1.1" 200 None 2026-03-24 13:51:28,571 - lava_callback - ERROR - Error processing callback for node 69bff3dd7544ab5cd4a7f31f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60896 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:52:48,531 - lava_callback - INFO - Received callback for node 69bff4287544ab5cd4a7f328 INFO: 172.20.0.1:60910 - "POST /node/69bff4287544ab5cd4a7f328 HTTP/1.0" 202 Accepted 2026-03-24 13:52:48,537 - lava_callback - INFO - Starting async job submit for node 69bff4287544ab5cd4a7f328 2026-03-24 13:52:48,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:52:48,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff4287544ab5cd4a7f328 HTTP/1.1" 200 None 2026-03-24 13:52:48,562 - lava_callback - DEBUG - Processing node 69bff4287544ab5cd4a7f328: baseline-arm64-clabbe 2026-03-24 13:52:48,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff4287544ab5cd4a7f328 2026-03-24 13:52:48,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:52:48,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff4287544ab5cd4a7f328 HTTP/1.1" 200 None 2026-03-24 13:52:48,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:52:48,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:52:48,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:52:48,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff4287544ab5cd4a7f328 HTTP/1.1" 200 None 2026-03-24 13:52:48,618 - lava_callback - ERROR - Error processing callback for node 69bff4287544ab5cd4a7f328: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35244 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:54:08,462 - lava_callback - INFO - Received callback for node 69bff4757544ab5cd4a7f33e INFO: 172.20.0.1:35260 - "POST /node/69bff4757544ab5cd4a7f33e HTTP/1.0" 202 Accepted 2026-03-24 13:54:08,467 - lava_callback - INFO - Starting async job submit for node 69bff4757544ab5cd4a7f33e 2026-03-24 13:54:08,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:54:08,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff4757544ab5cd4a7f33e HTTP/1.1" 200 None 2026-03-24 13:54:08,484 - lava_callback - DEBUG - Processing node 69bff4757544ab5cd4a7f33e: baseline-arm64-clabbe 2026-03-24 13:54:08,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff4757544ab5cd4a7f33e 2026-03-24 13:54:08,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:54:08,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff4757544ab5cd4a7f33e HTTP/1.1" 200 None 2026-03-24 13:54:08,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:54:08,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:54:08,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:54:08,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff4757544ab5cd4a7f33e HTTP/1.1" 200 None 2026-03-24 13:54:08,549 - lava_callback - ERROR - Error processing callback for node 69bff4757544ab5cd4a7f33e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56764 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:55:28,710 - lava_callback - INFO - Received callback for node 69bff4c07544ab5cd4a7f349 INFO: 172.20.0.1:56768 - "POST /node/69bff4c07544ab5cd4a7f349 HTTP/1.0" 202 Accepted 2026-03-24 13:55:28,716 - lava_callback - INFO - Starting async job submit for node 69bff4c07544ab5cd4a7f349 2026-03-24 13:55:28,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:55:28,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff4c07544ab5cd4a7f349 HTTP/1.1" 200 None 2026-03-24 13:55:28,732 - lava_callback - DEBUG - Processing node 69bff4c07544ab5cd4a7f349: baseline-arm64-clabbe 2026-03-24 13:55:28,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff4c07544ab5cd4a7f349 2026-03-24 13:55:28,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:55:28,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff4c07544ab5cd4a7f349 HTTP/1.1" 200 None 2026-03-24 13:55:28,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:55:28,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:55:28,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:55:28,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff4c07544ab5cd4a7f349 HTTP/1.1" 200 None 2026-03-24 13:55:28,786 - lava_callback - ERROR - Error processing callback for node 69bff4c07544ab5cd4a7f349: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56942 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:56:48,694 - lava_callback - INFO - Received callback for node 69bff50c7544ab5cd4a7f35a INFO: 172.20.0.1:56952 - "POST /node/69bff50c7544ab5cd4a7f35a HTTP/1.0" 202 Accepted 2026-03-24 13:56:48,700 - lava_callback - INFO - Starting async job submit for node 69bff50c7544ab5cd4a7f35a 2026-03-24 13:56:48,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:56:48,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff50c7544ab5cd4a7f35a HTTP/1.1" 200 None 2026-03-24 13:56:48,716 - lava_callback - DEBUG - Processing node 69bff50c7544ab5cd4a7f35a: baseline-arm64-clabbe 2026-03-24 13:56:48,716 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff50c7544ab5cd4a7f35a 2026-03-24 13:56:48,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:56:48,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff50c7544ab5cd4a7f35a HTTP/1.1" 200 None 2026-03-24 13:56:48,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:56:48,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:56:48,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:56:48,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff50c7544ab5cd4a7f35a HTTP/1.1" 200 None 2026-03-24 13:56:48,772 - lava_callback - ERROR - Error processing callback for node 69bff50c7544ab5cd4a7f35a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38678 - "GET / HTTP/1.0" 200 OK 2026-03-24 13:57:48,778 - lava_callback - INFO - Received callback for node 69bff5577544ab5cd4a7f368 INFO: 172.20.0.1:38694 - "POST /node/69bff5577544ab5cd4a7f368 HTTP/1.0" 202 Accepted 2026-03-24 13:57:48,783 - lava_callback - INFO - Starting async job submit for node 69bff5577544ab5cd4a7f368 2026-03-24 13:57:48,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:57:48,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff5577544ab5cd4a7f368 HTTP/1.1" 200 None 2026-03-24 13:57:48,799 - lava_callback - DEBUG - Processing node 69bff5577544ab5cd4a7f368: baseline-arm64-clabbe 2026-03-24 13:57:48,800 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff5577544ab5cd4a7f368 2026-03-24 13:57:48,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:57:48,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff5577544ab5cd4a7f368 HTTP/1.1" 200 None 2026-03-24 13:57:48,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:57:48,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 13:57:48,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 13:57:48,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff5577544ab5cd4a7f368 HTTP/1.1" 200 None 2026-03-24 13:57:48,878 - lava_callback - ERROR - Error processing callback for node 69bff5577544ab5cd4a7f368: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39388 - "GET /robots.txt HTTP/1.0" 404 Not Found INFO: 172.20.0.1:41506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60828 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:09:16,796 - lava_callback - INFO - Received callback for node 69bff7fa7544ab5cd4a7f386 INFO: 172.20.0.1:60844 - "POST /node/69bff7fa7544ab5cd4a7f386 HTTP/1.0" 202 Accepted 2026-03-24 14:09:16,801 - lava_callback - INFO - Starting async job submit for node 69bff7fa7544ab5cd4a7f386 2026-03-24 14:09:16,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:09:16,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff7fa7544ab5cd4a7f386 HTTP/1.1" 200 None 2026-03-24 14:09:16,855 - lava_callback - DEBUG - Processing node 69bff7fa7544ab5cd4a7f386: baseline-arm64-baylibre 2026-03-24 14:09:16,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff7fa7544ab5cd4a7f386 2026-03-24 14:09:16,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:09:16,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff7fa7544ab5cd4a7f386 HTTP/1.1" 200 None 2026-03-24 14:09:16,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:09:16,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 14:09:16,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:09:16,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff7fa7544ab5cd4a7f386 HTTP/1.1" 200 None 2026-03-24 14:09:16,924 - lava_callback - ERROR - Error processing callback for node 69bff7fa7544ab5cd4a7f386: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60618 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:12:09,435 - lava_callback - INFO - Received callback for node 69bff8b37544ab5cd4a7f396 INFO: 172.20.0.1:60628 - "POST /node/69bff8b37544ab5cd4a7f396 HTTP/1.0" 202 Accepted 2026-03-24 14:12:09,441 - lava_callback - INFO - Starting async job submit for node 69bff8b37544ab5cd4a7f396 2026-03-24 14:12:09,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:12:09,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff8b37544ab5cd4a7f396 HTTP/1.1" 200 None 2026-03-24 14:12:09,482 - lava_callback - DEBUG - Processing node 69bff8b37544ab5cd4a7f396: baseline-arm64-clabbe 2026-03-24 14:12:09,482 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff8b37544ab5cd4a7f396 2026-03-24 14:12:09,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:12:09,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff8b37544ab5cd4a7f396 HTTP/1.1" 200 None 2026-03-24 14:12:09,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:12:09,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 14:12:09,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:12:09,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff8b37544ab5cd4a7f396 HTTP/1.1" 200 None 2026-03-24 14:12:09,540 - lava_callback - ERROR - Error processing callback for node 69bff8b37544ab5cd4a7f396: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54528 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:13:29,481 - lava_callback - INFO - Received callback for node 69bff9007544ab5cd4a7f3ad INFO: 172.20.0.1:54530 - "POST /node/69bff9007544ab5cd4a7f3ad HTTP/1.0" 202 Accepted 2026-03-24 14:13:29,486 - lava_callback - INFO - Starting async job submit for node 69bff9007544ab5cd4a7f3ad 2026-03-24 14:13:29,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:13:29,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff9007544ab5cd4a7f3ad HTTP/1.1" 200 None 2026-03-24 14:13:29,520 - lava_callback - DEBUG - Processing node 69bff9007544ab5cd4a7f3ad: baseline-arm64-clabbe 2026-03-24 14:13:29,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bff9007544ab5cd4a7f3ad 2026-03-24 14:13:29,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:13:29,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bff9007544ab5cd4a7f3ad HTTP/1.1" 200 None 2026-03-24 14:13:29,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:13:29,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-24 14:13:29,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:13:29,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bff9007544ab5cd4a7f3ad HTTP/1.1" 200 None 2026-03-24 14:13:29,571 - lava_callback - ERROR - Error processing callback for node 69bff9007544ab5cd4a7f3ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33758 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57620 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:38:58,206 - lava_callback - INFO - Received callback for node 69bffefa7544ab5cd4a7f3d0 INFO: 172.20.0.1:57622 - "POST /node/69bffefa7544ab5cd4a7f3d0 HTTP/1.0" 202 Accepted 2026-03-24 14:38:58,212 - lava_callback - INFO - Starting async job submit for node 69bffefa7544ab5cd4a7f3d0 2026-03-24 14:38:58,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:38:58,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bffefa7544ab5cd4a7f3d0 HTTP/1.1" 200 None 2026-03-24 14:38:58,232 - lava_callback - DEBUG - Processing node 69bffefa7544ab5cd4a7f3d0: baseline-arm-baylibre 2026-03-24 14:38:58,232 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bffefa7544ab5cd4a7f3d0 2026-03-24 14:38:58,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:38:58,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bffefa7544ab5cd4a7f3d0 HTTP/1.1" 200 None 2026-03-24 14:38:58,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:38:58,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:38:58,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:38:58,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bffefa7544ab5cd4a7f3d0 HTTP/1.1" 200 None 2026-03-24 14:38:58,294 - lava_callback - ERROR - Error processing callback for node 69bffefa7544ab5cd4a7f3d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59682 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:40:58,310 - lava_callback - INFO - Received callback for node 69bfff707544ab5cd4a7f3db INFO: 172.20.0.1:59686 - "POST /node/69bfff707544ab5cd4a7f3db HTTP/1.0" 202 Accepted 2026-03-24 14:40:58,316 - lava_callback - INFO - Starting async job submit for node 69bfff707544ab5cd4a7f3db 2026-03-24 14:40:58,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:40:58,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfff707544ab5cd4a7f3db HTTP/1.1" 200 None 2026-03-24 14:40:58,332 - lava_callback - DEBUG - Processing node 69bfff707544ab5cd4a7f3db: baseline-arm-baylibre 2026-03-24 14:40:58,332 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfff707544ab5cd4a7f3db 2026-03-24 14:40:58,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:40:58,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfff707544ab5cd4a7f3db HTTP/1.1" 200 None 2026-03-24 14:40:58,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:40:58,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:40:58,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:40:58,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfff707544ab5cd4a7f3db HTTP/1.1" 200 None 2026-03-24 14:40:58,433 - lava_callback - ERROR - Error processing callback for node 69bfff707544ab5cd4a7f3db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48278 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:41:38,285 - lava_callback - INFO - Received callback for node 69bfff947544ab5cd4a7f3e7 INFO: 172.20.0.1:48292 - "POST /node/69bfff947544ab5cd4a7f3e7 HTTP/1.0" 202 Accepted 2026-03-24 14:41:38,291 - lava_callback - INFO - Starting async job submit for node 69bfff947544ab5cd4a7f3e7 2026-03-24 14:41:38,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:41:38,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfff947544ab5cd4a7f3e7 HTTP/1.1" 200 None 2026-03-24 14:41:38,329 - lava_callback - DEBUG - Processing node 69bfff947544ab5cd4a7f3e7: baseline-arm-baylibre 2026-03-24 14:41:38,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfff947544ab5cd4a7f3e7 2026-03-24 14:41:38,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:41:38,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfff947544ab5cd4a7f3e7 HTTP/1.1" 200 None 2026-03-24 14:41:38,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:41:38,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:41:38,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:41:38,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfff947544ab5cd4a7f3e7 HTTP/1.1" 200 None 2026-03-24 14:41:38,408 - lava_callback - ERROR - Error processing callback for node 69bfff947544ab5cd4a7f3e7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39202 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:42:18,361 - lava_callback - INFO - Received callback for node 69bfffb97544ab5cd4a7f3f4 INFO: 172.20.0.1:39206 - "POST /node/69bfffb97544ab5cd4a7f3f4 HTTP/1.0" 202 Accepted 2026-03-24 14:42:18,367 - lava_callback - INFO - Starting async job submit for node 69bfffb97544ab5cd4a7f3f4 2026-03-24 14:42:18,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:42:18,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfffb97544ab5cd4a7f3f4 HTTP/1.1" 200 None 2026-03-24 14:42:18,406 - lava_callback - DEBUG - Processing node 69bfffb97544ab5cd4a7f3f4: baseline-arm-baylibre 2026-03-24 14:42:18,406 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfffb97544ab5cd4a7f3f4 2026-03-24 14:42:18,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:42:18,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfffb97544ab5cd4a7f3f4 HTTP/1.1" 200 None 2026-03-24 14:42:18,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:42:18,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:42:18,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:42:18,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfffb97544ab5cd4a7f3f4 HTTP/1.1" 200 None 2026-03-24 14:42:18,502 - lava_callback - ERROR - Error processing callback for node 69bfffb97544ab5cd4a7f3f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58564 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:42:58,369 - lava_callback - INFO - Received callback for node 69bfffdd7544ab5cd4a7f401 INFO: 172.20.0.1:58578 - "POST /node/69bfffdd7544ab5cd4a7f401 HTTP/1.0" 202 Accepted 2026-03-24 14:42:58,374 - lava_callback - INFO - Starting async job submit for node 69bfffdd7544ab5cd4a7f401 2026-03-24 14:42:58,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:42:58,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfffdd7544ab5cd4a7f401 HTTP/1.1" 200 None 2026-03-24 14:42:58,391 - lava_callback - DEBUG - Processing node 69bfffdd7544ab5cd4a7f401: baseline-arm-baylibre 2026-03-24 14:42:58,391 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69bfffdd7544ab5cd4a7f401 2026-03-24 14:42:58,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:42:58,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfffdd7544ab5cd4a7f401 HTTP/1.1" 200 None 2026-03-24 14:42:58,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:42:58,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:42:58,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:42:58,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69bfffdd7544ab5cd4a7f401 HTTP/1.1" 200 None 2026-03-24 14:42:58,473 - lava_callback - ERROR - Error processing callback for node 69bfffdd7544ab5cd4a7f401: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58918 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:43:18,365 - lava_callback - INFO - Received callback for node 69c000007544ab5cd4a7f410 INFO: 172.20.0.1:58922 - "POST /node/69c000007544ab5cd4a7f410 HTTP/1.0" 202 Accepted 2026-03-24 14:43:18,371 - lava_callback - INFO - Starting async job submit for node 69c000007544ab5cd4a7f410 2026-03-24 14:43:18,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:43:18,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000007544ab5cd4a7f410 HTTP/1.1" 200 None 2026-03-24 14:43:18,397 - lava_callback - DEBUG - Processing node 69c000007544ab5cd4a7f410: baseline-arm-baylibre 2026-03-24 14:43:18,397 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c000007544ab5cd4a7f410 2026-03-24 14:43:18,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:43:18,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000007544ab5cd4a7f410 HTTP/1.1" 200 None 2026-03-24 14:43:18,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:43:18,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:43:18,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:43:18,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c000007544ab5cd4a7f410 HTTP/1.1" 200 None 2026-03-24 14:43:18,449 - lava_callback - ERROR - Error processing callback for node 69c000007544ab5cd4a7f410: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37350 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:43:58,442 - lava_callback - INFO - Received callback for node 69c000267544ab5cd4a7f41d INFO: 172.20.0.1:37358 - "POST /node/69c000267544ab5cd4a7f41d HTTP/1.0" 202 Accepted 2026-03-24 14:43:58,448 - lava_callback - INFO - Starting async job submit for node 69c000267544ab5cd4a7f41d 2026-03-24 14:43:58,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:43:58,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000267544ab5cd4a7f41d HTTP/1.1" 200 None 2026-03-24 14:43:58,471 - lava_callback - DEBUG - Processing node 69c000267544ab5cd4a7f41d: baseline-arm-baylibre 2026-03-24 14:43:58,472 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c000267544ab5cd4a7f41d 2026-03-24 14:43:58,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:43:58,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000267544ab5cd4a7f41d HTTP/1.1" 200 None 2026-03-24 14:43:58,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:43:58,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:43:58,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:43:58,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c000267544ab5cd4a7f41d HTTP/1.1" 200 None 2026-03-24 14:43:58,531 - lava_callback - ERROR - Error processing callback for node 69c000267544ab5cd4a7f41d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56358 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:44:38,527 - lava_callback - INFO - Received callback for node 69c000497544ab5cd4a7f425 INFO: 172.20.0.1:56360 - "POST /node/69c000497544ab5cd4a7f425 HTTP/1.0" 202 Accepted 2026-03-24 14:44:38,533 - lava_callback - INFO - Starting async job submit for node 69c000497544ab5cd4a7f425 2026-03-24 14:44:38,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:44:38,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000497544ab5cd4a7f425 HTTP/1.1" 200 None 2026-03-24 14:44:38,554 - lava_callback - DEBUG - Processing node 69c000497544ab5cd4a7f425: baseline-arm-baylibre 2026-03-24 14:44:38,555 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c000497544ab5cd4a7f425 2026-03-24 14:44:38,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:44:38,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000497544ab5cd4a7f425 HTTP/1.1" 200 None 2026-03-24 14:44:38,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:44:38,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 14:44:38,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:44:38,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c000497544ab5cd4a7f425 HTTP/1.1" 200 None 2026-03-24 14:44:38,611 - lava_callback - ERROR - Error processing callback for node 69c000497544ab5cd4a7f425: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52564 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:45:10,973 - lava_callback - INFO - Received callback for node 69c0006f7544ab5cd4a7f434 INFO: 172.20.0.1:34924 - "POST /node/69c0006f7544ab5cd4a7f434 HTTP/1.0" 202 Accepted 2026-03-24 14:45:10,979 - lava_callback - INFO - Starting async job submit for node 69c0006f7544ab5cd4a7f434 2026-03-24 14:45:10,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:45:10,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0006f7544ab5cd4a7f434 HTTP/1.1" 200 None 2026-03-24 14:45:10,996 - lava_callback - DEBUG - Processing node 69c0006f7544ab5cd4a7f434: baseline-arm-clabbe 2026-03-24 14:45:10,996 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0006f7544ab5cd4a7f434 2026-03-24 14:45:10,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:45:11,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0006f7544ab5cd4a7f434 HTTP/1.1" 200 None 2026-03-24 14:45:11,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:45:11,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 14:45:11,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:45:11,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0006f7544ab5cd4a7f434 HTTP/1.1" 200 None 2026-03-24 14:45:11,057 - lava_callback - ERROR - Error processing callback for node 69c0006f7544ab5cd4a7f434: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35726 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:45:50,989 - lava_callback - INFO - Received callback for node 69c000947544ab5cd4a7f43f INFO: 172.20.0.1:33050 - "POST /node/69c000947544ab5cd4a7f43f HTTP/1.0" 202 Accepted 2026-03-24 14:45:50,994 - lava_callback - INFO - Starting async job submit for node 69c000947544ab5cd4a7f43f 2026-03-24 14:45:50,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:45:51,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000947544ab5cd4a7f43f HTTP/1.1" 200 None 2026-03-24 14:45:51,010 - lava_callback - DEBUG - Processing node 69c000947544ab5cd4a7f43f: baseline-arm-clabbe 2026-03-24 14:45:51,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c000947544ab5cd4a7f43f 2026-03-24 14:45:51,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:45:51,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000947544ab5cd4a7f43f HTTP/1.1" 200 None 2026-03-24 14:45:51,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:45:51,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 14:45:51,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:45:51,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c000947544ab5cd4a7f43f HTTP/1.1" 200 None 2026-03-24 14:45:51,070 - lava_callback - ERROR - Error processing callback for node 69c000947544ab5cd4a7f43f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40564 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:46:31,047 - lava_callback - INFO - Received callback for node 69c000b87544ab5cd4a7f450 INFO: 172.20.0.1:56234 - "POST /node/69c000b87544ab5cd4a7f450 HTTP/1.0" 202 Accepted 2026-03-24 14:46:31,053 - lava_callback - INFO - Starting async job submit for node 69c000b87544ab5cd4a7f450 2026-03-24 14:46:31,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:46:31,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000b87544ab5cd4a7f450 HTTP/1.1" 200 None 2026-03-24 14:46:31,069 - lava_callback - DEBUG - Processing node 69c000b87544ab5cd4a7f450: baseline-arm-clabbe 2026-03-24 14:46:31,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c000b87544ab5cd4a7f450 2026-03-24 14:46:31,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:46:31,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000b87544ab5cd4a7f450 HTTP/1.1" 200 None 2026-03-24 14:46:31,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:46:31,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 14:46:31,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:46:31,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c000b87544ab5cd4a7f450 HTTP/1.1" 200 None 2026-03-24 14:46:31,127 - lava_callback - ERROR - Error processing callback for node 69c000b87544ab5cd4a7f450: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36142 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:46:58,659 - lava_callback - INFO - Received callback for node 69c000d87544ab5cd4a7f45c INFO: 172.20.0.1:36148 - "POST /node/69c000d87544ab5cd4a7f45c HTTP/1.0" 202 Accepted 2026-03-24 14:46:58,665 - lava_callback - INFO - Starting async job submit for node 69c000d87544ab5cd4a7f45c 2026-03-24 14:46:58,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:46:58,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000d87544ab5cd4a7f45c HTTP/1.1" 200 None 2026-03-24 14:46:58,681 - lava_callback - DEBUG - Processing node 69c000d87544ab5cd4a7f45c: baseline-arm-baylibre 2026-03-24 14:46:58,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c000d87544ab5cd4a7f45c 2026-03-24 14:46:58,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:46:58,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000d87544ab5cd4a7f45c HTTP/1.1" 200 None 2026-03-24 14:46:58,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:46:58,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:46:58,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:46:58,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c000d87544ab5cd4a7f45c HTTP/1.1" 200 None 2026-03-24 14:46:58,742 - lava_callback - ERROR - Error processing callback for node 69c000d87544ab5cd4a7f45c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55732 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:47:38,624 - lava_callback - INFO - Received callback for node 69c000fc7544ab5cd4a7f467 INFO: 172.20.0.1:55746 - "POST /node/69c000fc7544ab5cd4a7f467 HTTP/1.0" 202 Accepted 2026-03-24 14:47:38,629 - lava_callback - INFO - Starting async job submit for node 69c000fc7544ab5cd4a7f467 2026-03-24 14:47:38,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:47:38,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000fc7544ab5cd4a7f467 HTTP/1.1" 200 None 2026-03-24 14:47:38,645 - lava_callback - DEBUG - Processing node 69c000fc7544ab5cd4a7f467: baseline-arm-baylibre 2026-03-24 14:47:38,645 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c000fc7544ab5cd4a7f467 2026-03-24 14:47:38,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:47:38,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c000fc7544ab5cd4a7f467 HTTP/1.1" 200 None 2026-03-24 14:47:38,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:47:38,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:47:38,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:47:38,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c000fc7544ab5cd4a7f467 HTTP/1.1" 200 None 2026-03-24 14:47:38,719 - lava_callback - ERROR - Error processing callback for node 69c000fc7544ab5cd4a7f467: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42518 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:48:11,140 - lava_callback - INFO - Received callback for node 69c001247544ab5cd4a7f474 INFO: 172.20.0.1:53732 - "POST /node/69c001247544ab5cd4a7f474 HTTP/1.0" 202 Accepted 2026-03-24 14:48:11,146 - lava_callback - INFO - Starting async job submit for node 69c001247544ab5cd4a7f474 2026-03-24 14:48:11,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:48:11,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001247544ab5cd4a7f474 HTTP/1.1" 200 None 2026-03-24 14:48:11,167 - lava_callback - DEBUG - Processing node 69c001247544ab5cd4a7f474: baseline-arm-clabbe 2026-03-24 14:48:11,168 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c001247544ab5cd4a7f474 2026-03-24 14:48:11,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:48:11,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001247544ab5cd4a7f474 HTTP/1.1" 200 None 2026-03-24 14:48:11,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:48:11,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 14:48:11,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:48:11,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c001247544ab5cd4a7f474 HTTP/1.1" 200 None 2026-03-24 14:48:11,220 - lava_callback - ERROR - Error processing callback for node 69c001247544ab5cd4a7f474: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43904 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:48:51,144 - lava_callback - INFO - Received callback for node 69c001497544ab5cd4a7f481 INFO: 172.20.0.1:35230 - "POST /node/69c001497544ab5cd4a7f481 HTTP/1.0" 202 Accepted 2026-03-24 14:48:51,150 - lava_callback - INFO - Starting async job submit for node 69c001497544ab5cd4a7f481 2026-03-24 14:48:51,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:48:51,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001497544ab5cd4a7f481 HTTP/1.1" 200 None 2026-03-24 14:48:51,166 - lava_callback - DEBUG - Processing node 69c001497544ab5cd4a7f481: baseline-arm-clabbe 2026-03-24 14:48:51,167 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c001497544ab5cd4a7f481 2026-03-24 14:48:51,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:48:51,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001497544ab5cd4a7f481 HTTP/1.1" 200 None 2026-03-24 14:48:51,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:48:51,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 14:48:51,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:48:51,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c001497544ab5cd4a7f481 HTTP/1.1" 200 None 2026-03-24 14:48:51,233 - lava_callback - ERROR - Error processing callback for node 69c001497544ab5cd4a7f481: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57530 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:49:18,723 - lava_callback - INFO - Received callback for node 69c0016b7544ab5cd4a7f49e INFO: 172.20.0.1:57546 - "POST /node/69c0016b7544ab5cd4a7f49e HTTP/1.0" 202 Accepted 2026-03-24 14:49:18,728 - lava_callback - INFO - Starting async job submit for node 69c0016b7544ab5cd4a7f49e 2026-03-24 14:49:18,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:49:18,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0016b7544ab5cd4a7f49e HTTP/1.1" 200 None 2026-03-24 14:49:18,746 - lava_callback - DEBUG - Processing node 69c0016b7544ab5cd4a7f49e: baseline-arm-baylibre 2026-03-24 14:49:18,746 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0016b7544ab5cd4a7f49e 2026-03-24 14:49:18,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:49:18,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0016b7544ab5cd4a7f49e HTTP/1.1" 200 None 2026-03-24 14:49:18,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:49:18,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 14:49:18,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:49:18,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0016b7544ab5cd4a7f49e HTTP/1.1" 200 None 2026-03-24 14:49:18,804 - lava_callback - ERROR - Error processing callback for node 69c0016b7544ab5cd4a7f49e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39176 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:50:11,189 - lava_callback - INFO - Received callback for node 69c001917544ab5cd4a7f4be INFO: 172.20.0.1:43966 - "POST /node/69c001917544ab5cd4a7f4be HTTP/1.0" 202 Accepted 2026-03-24 14:50:11,194 - lava_callback - INFO - Starting async job submit for node 69c001917544ab5cd4a7f4be 2026-03-24 14:50:11,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:50:11,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001917544ab5cd4a7f4be HTTP/1.1" 200 None 2026-03-24 14:50:11,211 - lava_callback - DEBUG - Processing node 69c001917544ab5cd4a7f4be: baseline-arm-clabbe 2026-03-24 14:50:11,211 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c001917544ab5cd4a7f4be 2026-03-24 14:50:11,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:50:11,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001917544ab5cd4a7f4be HTTP/1.1" 200 None 2026-03-24 14:50:11,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:50:11,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:50:11,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:50:11,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c001917544ab5cd4a7f4be HTTP/1.1" 200 None 2026-03-24 14:50:11,278 - lava_callback - ERROR - Error processing callback for node 69c001917544ab5cd4a7f4be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59248 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:50:51,216 - lava_callback - INFO - Received callback for node 69c001b67544ab5cd4a7f4ce INFO: 172.20.0.1:60576 - "POST /node/69c001b67544ab5cd4a7f4ce HTTP/1.0" 202 Accepted 2026-03-24 14:50:51,222 - lava_callback - INFO - Starting async job submit for node 69c001b67544ab5cd4a7f4ce 2026-03-24 14:50:51,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:50:51,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001b67544ab5cd4a7f4ce HTTP/1.1" 200 None 2026-03-24 14:50:51,236 - lava_callback - DEBUG - Processing node 69c001b67544ab5cd4a7f4ce: baseline-arm-clabbe 2026-03-24 14:50:51,237 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c001b67544ab5cd4a7f4ce 2026-03-24 14:50:51,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:50:51,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001b67544ab5cd4a7f4ce HTTP/1.1" 200 None 2026-03-24 14:50:51,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:50:51,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:50:51,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:50:51,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c001b67544ab5cd4a7f4ce HTTP/1.1" 200 None 2026-03-24 14:50:51,289 - lava_callback - ERROR - Error processing callback for node 69c001b67544ab5cd4a7f4ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42956 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:51:11,271 - lava_callback - INFO - Received callback for node 69c001da7544ab5cd4a7f4d9 INFO: 172.20.0.1:33294 - "POST /node/69c001da7544ab5cd4a7f4d9 HTTP/1.0" 202 Accepted 2026-03-24 14:51:11,277 - lava_callback - INFO - Starting async job submit for node 69c001da7544ab5cd4a7f4d9 2026-03-24 14:51:11,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:51:11,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001da7544ab5cd4a7f4d9 HTTP/1.1" 200 None 2026-03-24 14:51:11,298 - lava_callback - DEBUG - Processing node 69c001da7544ab5cd4a7f4d9: baseline-arm-clabbe 2026-03-24 14:51:11,298 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c001da7544ab5cd4a7f4d9 2026-03-24 14:51:11,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:51:11,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001da7544ab5cd4a7f4d9 HTTP/1.1" 200 None 2026-03-24 14:51:11,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:51:11,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:51:11,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:51:11,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c001da7544ab5cd4a7f4d9 HTTP/1.1" 200 None 2026-03-24 14:51:11,359 - lava_callback - ERROR - Error processing callback for node 69c001da7544ab5cd4a7f4d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49500 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:51:51,290 - lava_callback - INFO - Received callback for node 69c001ff7544ab5cd4a7f4e4 INFO: 172.20.0.1:57052 - "POST /node/69c001ff7544ab5cd4a7f4e4 HTTP/1.0" 202 Accepted 2026-03-24 14:51:51,296 - lava_callback - INFO - Starting async job submit for node 69c001ff7544ab5cd4a7f4e4 2026-03-24 14:51:51,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:51:51,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001ff7544ab5cd4a7f4e4 HTTP/1.1" 200 None 2026-03-24 14:51:51,335 - lava_callback - DEBUG - Processing node 69c001ff7544ab5cd4a7f4e4: baseline-arm-clabbe 2026-03-24 14:51:51,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c001ff7544ab5cd4a7f4e4 2026-03-24 14:51:51,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:51:51,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c001ff7544ab5cd4a7f4e4 HTTP/1.1" 200 None 2026-03-24 14:51:51,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:51:51,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 14:51:51,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:51:51,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c001ff7544ab5cd4a7f4e4 HTTP/1.1" 200 None 2026-03-24 14:51:51,388 - lava_callback - ERROR - Error processing callback for node 69c001ff7544ab5cd4a7f4e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56852 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:58:31,559 - lava_callback - INFO - Received callback for node 69c003947544ab5cd4a7f576 INFO: 172.20.0.1:43152 - "POST /node/69c003947544ab5cd4a7f576 HTTP/1.0" 202 Accepted 2026-03-24 14:58:31,565 - lava_callback - INFO - Starting async job submit for node 69c003947544ab5cd4a7f576 2026-03-24 14:58:31,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:58:31,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c003947544ab5cd4a7f576 HTTP/1.1" 200 None 2026-03-24 14:58:31,581 - lava_callback - DEBUG - Processing node 69c003947544ab5cd4a7f576: baseline-arm-clabbe 2026-03-24 14:58:31,581 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c003947544ab5cd4a7f576 2026-03-24 14:58:31,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:58:31,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c003947544ab5cd4a7f576 HTTP/1.1" 200 None 2026-03-24 14:58:31,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:58:31,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 14:58:31,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:58:31,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c003947544ab5cd4a7f576 HTTP/1.1" 200 None 2026-03-24 14:58:31,642 - lava_callback - ERROR - Error processing callback for node 69c003947544ab5cd4a7f576: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48730 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:59:11,569 - lava_callback - INFO - Received callback for node 69c003b87544ab5cd4a7f57f INFO: 172.20.0.1:42230 - "POST /node/69c003b87544ab5cd4a7f57f HTTP/1.0" 202 Accepted 2026-03-24 14:59:11,574 - lava_callback - INFO - Starting async job submit for node 69c003b87544ab5cd4a7f57f 2026-03-24 14:59:11,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:59:11,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c003b87544ab5cd4a7f57f HTTP/1.1" 200 None 2026-03-24 14:59:11,590 - lava_callback - DEBUG - Processing node 69c003b87544ab5cd4a7f57f: baseline-arm-clabbe 2026-03-24 14:59:11,591 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c003b87544ab5cd4a7f57f 2026-03-24 14:59:11,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:59:11,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c003b87544ab5cd4a7f57f HTTP/1.1" 200 None 2026-03-24 14:59:11,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:59:11,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 14:59:11,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:59:11,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c003b87544ab5cd4a7f57f HTTP/1.1" 200 None 2026-03-24 14:59:11,643 - lava_callback - ERROR - Error processing callback for node 69c003b87544ab5cd4a7f57f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44722 - "GET / HTTP/1.0" 200 OK 2026-03-24 14:59:59,626 - lava_callback - INFO - Received callback for node 69c003e47544ab5cd4a7f588 INFO: 172.20.0.1:44736 - "POST /node/69c003e47544ab5cd4a7f588 HTTP/1.0" 202 Accepted 2026-03-24 14:59:59,632 - lava_callback - INFO - Starting async job submit for node 69c003e47544ab5cd4a7f588 2026-03-24 14:59:59,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:59:59,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c003e47544ab5cd4a7f588 HTTP/1.1" 200 None 2026-03-24 14:59:59,648 - lava_callback - DEBUG - Processing node 69c003e47544ab5cd4a7f588: baseline-arm64-baylibre 2026-03-24 14:59:59,648 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c003e47544ab5cd4a7f588 2026-03-24 14:59:59,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:59:59,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c003e47544ab5cd4a7f588 HTTP/1.1" 200 None 2026-03-24 14:59:59,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:59:59,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 14:59:59,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 14:59:59,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c003e47544ab5cd4a7f588 HTTP/1.1" 200 None 2026-03-24 14:59:59,711 - lava_callback - ERROR - Error processing callback for node 69c003e47544ab5cd4a7f588: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50946 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:01:11,698 - lava_callback - INFO - Received callback for node 69c004277544ab5cd4a7f597 INFO: 172.20.0.1:59340 - "POST /node/69c004277544ab5cd4a7f597 HTTP/1.0" 202 Accepted 2026-03-24 15:01:11,704 - lava_callback - INFO - Starting async job submit for node 69c004277544ab5cd4a7f597 2026-03-24 15:01:11,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:01:11,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c004277544ab5cd4a7f597 HTTP/1.1" 200 None 2026-03-24 15:01:11,720 - lava_callback - DEBUG - Processing node 69c004277544ab5cd4a7f597: baseline-arm-clabbe 2026-03-24 15:01:11,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c004277544ab5cd4a7f597 2026-03-24 15:01:11,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:01:11,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c004277544ab5cd4a7f597 HTTP/1.1" 200 None 2026-03-24 15:01:11,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:01:11,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:01:11,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:01:11,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c004277544ab5cd4a7f597 HTTP/1.1" 200 None 2026-03-24 15:01:11,780 - lava_callback - ERROR - Error processing callback for node 69c004277544ab5cd4a7f597: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58524 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:01:51,661 - lava_callback - INFO - Received callback for node 69c004597544ab5cd4a7f59e INFO: 172.20.0.1:38448 - "POST /node/69c004597544ab5cd4a7f59e HTTP/1.0" 202 Accepted 2026-03-24 15:01:51,667 - lava_callback - INFO - Starting async job submit for node 69c004597544ab5cd4a7f59e 2026-03-24 15:01:51,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:01:51,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c004597544ab5cd4a7f59e HTTP/1.1" 200 None 2026-03-24 15:01:51,684 - lava_callback - DEBUG - Processing node 69c004597544ab5cd4a7f59e: baseline-arm64-clabbe 2026-03-24 15:01:51,684 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c004597544ab5cd4a7f59e 2026-03-24 15:01:51,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:01:51,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c004597544ab5cd4a7f59e HTTP/1.1" 200 None 2026-03-24 15:01:51,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:01:51,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 15:01:51,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:01:51,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c004597544ab5cd4a7f59e HTTP/1.1" 200 None 2026-03-24 15:01:51,737 - lava_callback - ERROR - Error processing callback for node 69c004597544ab5cd4a7f59e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47788 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:03:11,754 - lava_callback - INFO - Received callback for node 69c004a47544ab5cd4a7f5ae INFO: 172.20.0.1:38998 - "POST /node/69c004a47544ab5cd4a7f5ae HTTP/1.0" 202 Accepted 2026-03-24 15:03:11,760 - lava_callback - INFO - Starting async job submit for node 69c004a47544ab5cd4a7f5ae 2026-03-24 15:03:11,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:03:11,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c004a47544ab5cd4a7f5ae HTTP/1.1" 200 None 2026-03-24 15:03:11,776 - lava_callback - DEBUG - Processing node 69c004a47544ab5cd4a7f5ae: baseline-arm64-clabbe 2026-03-24 15:03:11,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c004a47544ab5cd4a7f5ae 2026-03-24 15:03:11,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:03:11,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c004a47544ab5cd4a7f5ae HTTP/1.1" 200 None 2026-03-24 15:03:11,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:03:11,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 15:03:11,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:03:11,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c004a47544ab5cd4a7f5ae HTTP/1.1" 200 None 2026-03-24 15:03:11,858 - lava_callback - ERROR - Error processing callback for node 69c004a47544ab5cd4a7f5ae: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52104 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:04:31,821 - lava_callback - INFO - Received callback for node 69c004f07544ab5cd4a7f5bc INFO: 172.20.0.1:51092 - "POST /node/69c004f07544ab5cd4a7f5bc HTTP/1.0" 202 Accepted 2026-03-24 15:04:31,827 - lava_callback - INFO - Starting async job submit for node 69c004f07544ab5cd4a7f5bc 2026-03-24 15:04:31,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:04:31,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c004f07544ab5cd4a7f5bc HTTP/1.1" 200 None 2026-03-24 15:04:31,842 - lava_callback - DEBUG - Processing node 69c004f07544ab5cd4a7f5bc: baseline-arm64-clabbe 2026-03-24 15:04:31,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c004f07544ab5cd4a7f5bc 2026-03-24 15:04:31,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:04:31,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c004f07544ab5cd4a7f5bc HTTP/1.1" 200 None 2026-03-24 15:04:31,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:04:31,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 15:04:31,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:04:31,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c004f07544ab5cd4a7f5bc HTTP/1.1" 200 None 2026-03-24 15:04:31,898 - lava_callback - ERROR - Error processing callback for node 69c004f07544ab5cd4a7f5bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51136 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:05:51,874 - lava_callback - INFO - Received callback for node 69c0053a7544ab5cd4a7f5ce INFO: 172.20.0.1:44144 - "POST /node/69c0053a7544ab5cd4a7f5ce HTTP/1.0" 202 Accepted 2026-03-24 15:05:51,880 - lava_callback - INFO - Starting async job submit for node 69c0053a7544ab5cd4a7f5ce 2026-03-24 15:05:51,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:05:51,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0053a7544ab5cd4a7f5ce HTTP/1.1" 200 None 2026-03-24 15:05:51,895 - lava_callback - DEBUG - Processing node 69c0053a7544ab5cd4a7f5ce: baseline-arm64-clabbe 2026-03-24 15:05:51,895 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0053a7544ab5cd4a7f5ce 2026-03-24 15:05:51,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:05:51,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0053a7544ab5cd4a7f5ce HTTP/1.1" 200 None 2026-03-24 15:05:51,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:05:51,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 15:05:51,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:05:51,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0053a7544ab5cd4a7f5ce HTTP/1.1" 200 None 2026-03-24 15:05:51,955 - lava_callback - ERROR - Error processing callback for node 69c0053a7544ab5cd4a7f5ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34750 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:06:51,914 - lava_callback - INFO - Received callback for node 69c005847544ab5cd4a7f5da INFO: 172.20.0.1:35292 - "POST /node/69c005847544ab5cd4a7f5da HTTP/1.0" 202 Accepted 2026-03-24 15:06:51,919 - lava_callback - INFO - Starting async job submit for node 69c005847544ab5cd4a7f5da 2026-03-24 15:06:51,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:06:51,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c005847544ab5cd4a7f5da HTTP/1.1" 200 None 2026-03-24 15:06:51,936 - lava_callback - DEBUG - Processing node 69c005847544ab5cd4a7f5da: baseline-arm64-clabbe 2026-03-24 15:06:51,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c005847544ab5cd4a7f5da 2026-03-24 15:06:51,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:06:51,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c005847544ab5cd4a7f5da HTTP/1.1" 200 None 2026-03-24 15:06:51,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:06:51,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 15:06:51,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:06:51,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c005847544ab5cd4a7f5da HTTP/1.1" 200 None 2026-03-24 15:06:51,988 - lava_callback - ERROR - Error processing callback for node 69c005847544ab5cd4a7f5da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40512 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:08:11,957 - lava_callback - INFO - Received callback for node 69c005d07544ab5cd4a7f5ec INFO: 172.20.0.1:44136 - "POST /node/69c005d07544ab5cd4a7f5ec HTTP/1.0" 202 Accepted 2026-03-24 15:08:11,962 - lava_callback - INFO - Starting async job submit for node 69c005d07544ab5cd4a7f5ec 2026-03-24 15:08:11,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:08:11,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c005d07544ab5cd4a7f5ec HTTP/1.1" 200 None 2026-03-24 15:08:11,979 - lava_callback - DEBUG - Processing node 69c005d07544ab5cd4a7f5ec: baseline-arm64-clabbe 2026-03-24 15:08:11,979 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c005d07544ab5cd4a7f5ec 2026-03-24 15:08:11,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:08:11,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c005d07544ab5cd4a7f5ec HTTP/1.1" 200 None 2026-03-24 15:08:11,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:08:12,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5827544ab5cd4a7d9dd HTTP/1.1" 200 None 2026-03-24 15:08:12,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:08:12,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c005d07544ab5cd4a7f5ec HTTP/1.1" 200 None 2026-03-24 15:08:12,043 - lava_callback - ERROR - Error processing callback for node 69c005d07544ab5cd4a7f5ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44212 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:09:19,897 - lava_callback - INFO - Received callback for node 69c006157544ab5cd4a7f5f8 INFO: 172.20.0.1:38124 - "POST /node/69c006157544ab5cd4a7f5f8 HTTP/1.0" 202 Accepted 2026-03-24 15:09:19,903 - lava_callback - INFO - Starting async job submit for node 69c006157544ab5cd4a7f5f8 2026-03-24 15:09:19,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:09:19,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c006157544ab5cd4a7f5f8 HTTP/1.1" 200 None 2026-03-24 15:09:19,919 - lava_callback - DEBUG - Processing node 69c006157544ab5cd4a7f5f8: baseline-arm64-baylibre 2026-03-24 15:09:19,919 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c006157544ab5cd4a7f5f8 2026-03-24 15:09:19,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:09:19,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c006157544ab5cd4a7f5f8 HTTP/1.1" 200 None 2026-03-24 15:09:19,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:09:19,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:09:19,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:09:19,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c006157544ab5cd4a7f5f8 HTTP/1.1" 200 None 2026-03-24 15:09:19,991 - lava_callback - ERROR - Error processing callback for node 69c006157544ab5cd4a7f5f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59540 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:10:32,112 - lava_callback - INFO - Received callback for node 69c006577544ab5cd4a7f606 INFO: 172.20.0.1:53084 - "POST /node/69c006577544ab5cd4a7f606 HTTP/1.0" 202 Accepted 2026-03-24 15:10:32,117 - lava_callback - INFO - Starting async job submit for node 69c006577544ab5cd4a7f606 2026-03-24 15:10:32,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:10:32,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c006577544ab5cd4a7f606 HTTP/1.1" 200 None 2026-03-24 15:10:32,136 - lava_callback - DEBUG - Processing node 69c006577544ab5cd4a7f606: baseline-arm-clabbe 2026-03-24 15:10:32,136 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c006577544ab5cd4a7f606 2026-03-24 15:10:32,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:10:32,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c006577544ab5cd4a7f606 HTTP/1.1" 200 None 2026-03-24 15:10:32,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:10:32,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 15:10:32,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:10:32,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c006577544ab5cd4a7f606 HTTP/1.1" 200 None 2026-03-24 15:10:32,218 - lava_callback - ERROR - Error processing callback for node 69c006577544ab5cd4a7f606: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39720 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:11:12,053 - lava_callback - INFO - Received callback for node 69c0067b7544ab5cd4a7f612 INFO: 172.20.0.1:36554 - "POST /node/69c0067b7544ab5cd4a7f612 HTTP/1.0" 202 Accepted 2026-03-24 15:11:12,059 - lava_callback - INFO - Starting async job submit for node 69c0067b7544ab5cd4a7f612 2026-03-24 15:11:12,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:11:12,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0067b7544ab5cd4a7f612 HTTP/1.1" 200 None 2026-03-24 15:11:12,090 - lava_callback - DEBUG - Processing node 69c0067b7544ab5cd4a7f612: baseline-arm-clabbe 2026-03-24 15:11:12,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0067b7544ab5cd4a7f612 2026-03-24 15:11:12,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:11:12,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0067b7544ab5cd4a7f612 HTTP/1.1" 200 None 2026-03-24 15:11:12,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:11:12,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 15:11:12,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:11:12,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0067b7544ab5cd4a7f612 HTTP/1.1" 200 None 2026-03-24 15:11:12,145 - lava_callback - ERROR - Error processing callback for node 69c0067b7544ab5cd4a7f612: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53210 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:11:32,143 - lava_callback - INFO - Received callback for node 69c0069f7544ab5cd4a7f61a INFO: 172.20.0.1:41858 - "POST /node/69c0069f7544ab5cd4a7f61a HTTP/1.0" 202 Accepted 2026-03-24 15:11:32,148 - lava_callback - INFO - Starting async job submit for node 69c0069f7544ab5cd4a7f61a 2026-03-24 15:11:32,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:11:32,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0069f7544ab5cd4a7f61a HTTP/1.1" 200 None 2026-03-24 15:11:32,165 - lava_callback - DEBUG - Processing node 69c0069f7544ab5cd4a7f61a: baseline-arm-clabbe 2026-03-24 15:11:32,165 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0069f7544ab5cd4a7f61a 2026-03-24 15:11:32,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:11:32,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0069f7544ab5cd4a7f61a HTTP/1.1" 200 None 2026-03-24 15:11:32,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:11:32,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd026d05e4bbd6ba634c57 HTTP/1.1" 200 None 2026-03-24 15:11:32,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:11:32,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0069f7544ab5cd4a7f61a HTTP/1.1" 200 None 2026-03-24 15:11:32,242 - lava_callback - ERROR - Error processing callback for node 69c0069f7544ab5cd4a7f61a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56430 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:12:32,146 - lava_callback - INFO - Received callback for node 69c006d17544ab5cd4a7f624 INFO: 172.20.0.1:58772 - "POST /node/69c006d17544ab5cd4a7f624 HTTP/1.0" 202 Accepted 2026-03-24 15:12:32,152 - lava_callback - INFO - Starting async job submit for node 69c006d17544ab5cd4a7f624 2026-03-24 15:12:32,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:12:32,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c006d17544ab5cd4a7f624 HTTP/1.1" 200 None 2026-03-24 15:12:32,190 - lava_callback - DEBUG - Processing node 69c006d17544ab5cd4a7f624: baseline-arm64-clabbe 2026-03-24 15:12:32,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c006d17544ab5cd4a7f624 2026-03-24 15:12:32,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:12:32,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c006d17544ab5cd4a7f624 HTTP/1.1" 200 None 2026-03-24 15:12:32,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:12:32,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:12:32,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:12:32,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c006d17544ab5cd4a7f624 HTTP/1.1" 200 None 2026-03-24 15:12:32,265 - lava_callback - ERROR - Error processing callback for node 69c006d17544ab5cd4a7f624: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60660 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:13:32,145 - lava_callback - INFO - Received callback for node 69c0070e7544ab5cd4a7f632 INFO: 172.20.0.1:58930 - "POST /node/69c0070e7544ab5cd4a7f632 HTTP/1.0" 202 Accepted 2026-03-24 15:13:32,150 - lava_callback - INFO - Starting async job submit for node 69c0070e7544ab5cd4a7f632 2026-03-24 15:13:32,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:13:32,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0070e7544ab5cd4a7f632 HTTP/1.1" 200 None 2026-03-24 15:13:32,167 - lava_callback - DEBUG - Processing node 69c0070e7544ab5cd4a7f632: baseline-arm-clabbe 2026-03-24 15:13:32,167 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0070e7544ab5cd4a7f632 2026-03-24 15:13:32,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:13:32,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0070e7544ab5cd4a7f632 HTTP/1.1" 200 None 2026-03-24 15:13:32,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:13:32,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:13:32,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:13:32,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0070e7544ab5cd4a7f632 HTTP/1.1" 200 None 2026-03-24 15:13:32,219 - lava_callback - ERROR - Error processing callback for node 69c0070e7544ab5cd4a7f632: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57828 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:14:00,116 - lava_callback - INFO - Received callback for node 69c0072f7544ab5cd4a7f640 INFO: 172.20.0.1:42926 - "POST /node/69c0072f7544ab5cd4a7f640 HTTP/1.0" 202 Accepted 2026-03-24 15:14:00,121 - lava_callback - INFO - Starting async job submit for node 69c0072f7544ab5cd4a7f640 2026-03-24 15:14:00,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:14:00,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0072f7544ab5cd4a7f640 HTTP/1.1" 200 None 2026-03-24 15:14:00,138 - lava_callback - DEBUG - Processing node 69c0072f7544ab5cd4a7f640: baseline-arm-baylibre 2026-03-24 15:14:00,138 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0072f7544ab5cd4a7f640 2026-03-24 15:14:00,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:14:00,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0072f7544ab5cd4a7f640 HTTP/1.1" 200 None 2026-03-24 15:14:00,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:14:00,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:14:00,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:14:00,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0072f7544ab5cd4a7f640 HTTP/1.1" 200 None 2026-03-24 15:14:00,198 - lava_callback - ERROR - Error processing callback for node 69c0072f7544ab5cd4a7f640: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33486 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:14:52,163 - lava_callback - INFO - Received callback for node 69c007637544ab5cd4a7f649 INFO: 172.20.0.1:34958 - "POST /node/69c007637544ab5cd4a7f649 HTTP/1.0" 202 Accepted 2026-03-24 15:14:52,169 - lava_callback - INFO - Starting async job submit for node 69c007637544ab5cd4a7f649 2026-03-24 15:14:52,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:14:52,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c007637544ab5cd4a7f649 HTTP/1.1" 200 None 2026-03-24 15:14:52,184 - lava_callback - DEBUG - Processing node 69c007637544ab5cd4a7f649: baseline-arm64-clabbe 2026-03-24 15:14:52,185 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c007637544ab5cd4a7f649 2026-03-24 15:14:52,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:14:52,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c007637544ab5cd4a7f649 HTTP/1.1" 200 None 2026-03-24 15:14:52,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:14:52,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:14:52,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:14:52,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c007637544ab5cd4a7f649 HTTP/1.1" 200 None 2026-03-24 15:14:52,242 - lava_callback - ERROR - Error processing callback for node 69c007637544ab5cd4a7f649: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36602 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:15:52,222 - lava_callback - INFO - Received callback for node 69c007a17544ab5cd4a7f657 INFO: 172.20.0.1:38654 - "POST /node/69c007a17544ab5cd4a7f657 HTTP/1.0" 202 Accepted 2026-03-24 15:15:52,228 - lava_callback - INFO - Starting async job submit for node 69c007a17544ab5cd4a7f657 2026-03-24 15:15:52,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:15:52,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c007a17544ab5cd4a7f657 HTTP/1.1" 200 None 2026-03-24 15:15:52,244 - lava_callback - DEBUG - Processing node 69c007a17544ab5cd4a7f657: baseline-arm-clabbe 2026-03-24 15:15:52,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c007a17544ab5cd4a7f657 2026-03-24 15:15:52,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:15:52,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c007a17544ab5cd4a7f657 HTTP/1.1" 200 None 2026-03-24 15:15:52,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:15:52,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:15:52,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:15:52,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c007a17544ab5cd4a7f657 HTTP/1.1" 200 None 2026-03-24 15:15:52,305 - lava_callback - ERROR - Error processing callback for node 69c007a17544ab5cd4a7f657: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51866 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:16:52,189 - lava_callback - INFO - Received callback for node 69c007d37544ab5cd4a7f665 INFO: 172.20.0.1:53564 - "POST /node/69c007d37544ab5cd4a7f665 HTTP/1.0" 202 Accepted 2026-03-24 15:16:52,195 - lava_callback - INFO - Starting async job submit for node 69c007d37544ab5cd4a7f665 2026-03-24 15:16:52,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:16:52,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c007d37544ab5cd4a7f665 HTTP/1.1" 200 None 2026-03-24 15:16:52,219 - lava_callback - DEBUG - Processing node 69c007d37544ab5cd4a7f665: baseline-arm64-clabbe 2026-03-24 15:16:52,220 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c007d37544ab5cd4a7f665 2026-03-24 15:16:52,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:16:52,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c007d37544ab5cd4a7f665 HTTP/1.1" 200 None 2026-03-24 15:16:52,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:16:52,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:16:52,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:16:52,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c007d37544ab5cd4a7f665 HTTP/1.1" 200 None 2026-03-24 15:16:52,290 - lava_callback - ERROR - Error processing callback for node 69c007d37544ab5cd4a7f665: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56840 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:17:40,602 - lava_callback - INFO - Received callback for node 69c0080e7544ab5cd4a7f670 INFO: 172.20.0.1:40648 - "POST /node/69c0080e7544ab5cd4a7f670 HTTP/1.0" 202 Accepted 2026-03-24 15:17:40,607 - lava_callback - INFO - Starting async job submit for node 69c0080e7544ab5cd4a7f670 2026-03-24 15:17:40,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:17:40,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0080e7544ab5cd4a7f670 HTTP/1.1" 200 None 2026-03-24 15:17:40,646 - lava_callback - DEBUG - Processing node 69c0080e7544ab5cd4a7f670: baseline-arm-baylibre 2026-03-24 15:17:40,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0080e7544ab5cd4a7f670 2026-03-24 15:17:40,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:17:40,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0080e7544ab5cd4a7f670 HTTP/1.1" 200 None 2026-03-24 15:17:40,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:17:40,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:17:40,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:17:40,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0080e7544ab5cd4a7f670 HTTP/1.1" 200 None 2026-03-24 15:17:40,728 - lava_callback - ERROR - Error processing callback for node 69c0080e7544ab5cd4a7f670: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58508 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:19:20,273 - lava_callback - INFO - Received callback for node 69c008757544ab5cd4a7f68d INFO: 172.20.0.1:46248 - "POST /node/69c008757544ab5cd4a7f68d HTTP/1.0" 202 Accepted 2026-03-24 15:19:20,279 - lava_callback - INFO - Starting async job submit for node 69c008757544ab5cd4a7f68d 2026-03-24 15:19:20,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:19:20,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c008757544ab5cd4a7f68d HTTP/1.1" 200 None 2026-03-24 15:19:20,294 - lava_callback - DEBUG - Processing node 69c008757544ab5cd4a7f68d: baseline-arm-baylibre 2026-03-24 15:19:20,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c008757544ab5cd4a7f68d 2026-03-24 15:19:20,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:19:20,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c008757544ab5cd4a7f68d HTTP/1.1" 200 None 2026-03-24 15:19:20,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:19:20,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 15:19:20,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:19:20,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c008757544ab5cd4a7f68d HTTP/1.1" 200 None 2026-03-24 15:19:20,346 - lava_callback - ERROR - Error processing callback for node 69c008757544ab5cd4a7f68d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48142 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:20:12,306 - lava_callback - INFO - Received callback for node 69c0089d7544ab5cd4a7f699 INFO: 172.20.0.1:59412 - "POST /node/69c0089d7544ab5cd4a7f699 HTTP/1.0" 202 Accepted 2026-03-24 15:20:12,312 - lava_callback - INFO - Starting async job submit for node 69c0089d7544ab5cd4a7f699 2026-03-24 15:20:12,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:20:12,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0089d7544ab5cd4a7f699 HTTP/1.1" 200 None 2026-03-24 15:20:12,331 - lava_callback - DEBUG - Processing node 69c0089d7544ab5cd4a7f699: baseline-arm-clabbe 2026-03-24 15:20:12,331 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0089d7544ab5cd4a7f699 2026-03-24 15:20:12,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:20:12,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0089d7544ab5cd4a7f699 HTTP/1.1" 200 None 2026-03-24 15:20:12,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:20:12,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:20:12,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:20:12,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0089d7544ab5cd4a7f699 HTTP/1.1" 200 None 2026-03-24 15:20:12,382 - lava_callback - ERROR - Error processing callback for node 69c0089d7544ab5cd4a7f699: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36270 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:21:12,335 - lava_callback - INFO - Received callback for node 69c008db7544ab5cd4a7f6a4 INFO: 172.20.0.1:33790 - "POST /node/69c008db7544ab5cd4a7f6a4 HTTP/1.0" 202 Accepted 2026-03-24 15:21:12,340 - lava_callback - INFO - Starting async job submit for node 69c008db7544ab5cd4a7f6a4 2026-03-24 15:21:12,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:21:12,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c008db7544ab5cd4a7f6a4 HTTP/1.1" 200 None 2026-03-24 15:21:12,356 - lava_callback - DEBUG - Processing node 69c008db7544ab5cd4a7f6a4: baseline-arm-clabbe 2026-03-24 15:21:12,356 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c008db7544ab5cd4a7f6a4 2026-03-24 15:21:12,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:21:12,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c008db7544ab5cd4a7f6a4 HTTP/1.1" 200 None 2026-03-24 15:21:12,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:21:12,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:21:12,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:21:12,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c008db7544ab5cd4a7f6a4 HTTP/1.1" 200 None 2026-03-24 15:21:12,416 - lava_callback - ERROR - Error processing callback for node 69c008db7544ab5cd4a7f6a4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51160 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:22:00,527 - lava_callback - INFO - Received callback for node 69c009057544ab5cd4a7f6ad INFO: 172.20.0.1:42612 - "POST /node/69c009057544ab5cd4a7f6ad HTTP/1.0" 202 Accepted 2026-03-24 15:22:00,532 - lava_callback - INFO - Starting async job submit for node 69c009057544ab5cd4a7f6ad 2026-03-24 15:22:00,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:22:00,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c009057544ab5cd4a7f6ad HTTP/1.1" 200 None 2026-03-24 15:22:00,572 - lava_callback - DEBUG - Processing node 69c009057544ab5cd4a7f6ad: baseline-arm-baylibre 2026-03-24 15:22:00,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c009057544ab5cd4a7f6ad 2026-03-24 15:22:00,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:22:00,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c009057544ab5cd4a7f6ad HTTP/1.1" 200 None 2026-03-24 15:22:00,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:22:00,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 15:22:00,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:22:00,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c009057544ab5cd4a7f6ad HTTP/1.1" 200 None 2026-03-24 15:22:00,647 - lava_callback - ERROR - Error processing callback for node 69c009057544ab5cd4a7f6ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37742 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:22:20,441 - lava_callback - INFO - Received callback for node 69c009297544ab5cd4a7f6b6 INFO: 172.20.0.1:41670 - "POST /node/69c009297544ab5cd4a7f6b6 HTTP/1.0" 202 Accepted 2026-03-24 15:22:20,446 - lava_callback - INFO - Starting async job submit for node 69c009297544ab5cd4a7f6b6 2026-03-24 15:22:20,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:22:20,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c009297544ab5cd4a7f6b6 HTTP/1.1" 200 None 2026-03-24 15:22:20,463 - lava_callback - DEBUG - Processing node 69c009297544ab5cd4a7f6b6: baseline-arm-baylibre 2026-03-24 15:22:20,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c009297544ab5cd4a7f6b6 2026-03-24 15:22:20,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:22:20,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c009297544ab5cd4a7f6b6 HTTP/1.1" 200 None 2026-03-24 15:22:20,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:22:20,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 15:22:20,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:22:20,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c009297544ab5cd4a7f6b6 HTTP/1.1" 200 None 2026-03-24 15:22:20,532 - lava_callback - ERROR - Error processing callback for node 69c009297544ab5cd4a7f6b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43520 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:23:12,439 - lava_callback - INFO - Received callback for node 69c0095b7544ab5cd4a7f6c3 INFO: 172.20.0.1:57964 - "POST /node/69c0095b7544ab5cd4a7f6c3 HTTP/1.0" 202 Accepted 2026-03-24 15:23:12,445 - lava_callback - INFO - Starting async job submit for node 69c0095b7544ab5cd4a7f6c3 2026-03-24 15:23:12,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:23:12,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0095b7544ab5cd4a7f6c3 HTTP/1.1" 200 None 2026-03-24 15:23:12,464 - lava_callback - DEBUG - Processing node 69c0095b7544ab5cd4a7f6c3: baseline-arm64-clabbe 2026-03-24 15:23:12,464 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0095b7544ab5cd4a7f6c3 2026-03-24 15:23:12,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:23:12,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0095b7544ab5cd4a7f6c3 HTTP/1.1" 200 None 2026-03-24 15:23:12,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:23:12,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:23:12,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:23:12,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0095b7544ab5cd4a7f6c3 HTTP/1.1" 200 None 2026-03-24 15:23:12,537 - lava_callback - ERROR - Error processing callback for node 69c0095b7544ab5cd4a7f6c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41610 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:25:32,554 - lava_callback - INFO - Received callback for node 69c009de7544ab5cd4a7f6de INFO: 172.20.0.1:50236 - "POST /node/69c009de7544ab5cd4a7f6de HTTP/1.0" 202 Accepted 2026-03-24 15:25:32,560 - lava_callback - INFO - Starting async job submit for node 69c009de7544ab5cd4a7f6de 2026-03-24 15:25:32,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:25:32,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c009de7544ab5cd4a7f6de HTTP/1.1" 200 None 2026-03-24 15:25:32,580 - lava_callback - DEBUG - Processing node 69c009de7544ab5cd4a7f6de: baseline-arm-clabbe 2026-03-24 15:25:32,580 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c009de7544ab5cd4a7f6de 2026-03-24 15:25:32,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:25:32,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c009de7544ab5cd4a7f6de HTTP/1.1" 200 None 2026-03-24 15:25:32,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:25:32,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 15:25:32,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:25:32,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c009de7544ab5cd4a7f6de HTTP/1.1" 200 None 2026-03-24 15:25:32,670 - lava_callback - ERROR - Error processing callback for node 69c009de7544ab5cd4a7f6de: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59890 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:26:12,549 - lava_callback - INFO - Received callback for node 69c00a027544ab5cd4a7f6eb INFO: 172.20.0.1:40626 - "POST /node/69c00a027544ab5cd4a7f6eb HTTP/1.0" 202 Accepted 2026-03-24 15:26:12,558 - lava_callback - INFO - Starting async job submit for node 69c00a027544ab5cd4a7f6eb 2026-03-24 15:26:12,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:26:12,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00a027544ab5cd4a7f6eb HTTP/1.1" 200 None 2026-03-24 15:26:12,574 - lava_callback - DEBUG - Processing node 69c00a027544ab5cd4a7f6eb: baseline-arm-clabbe 2026-03-24 15:26:12,574 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00a027544ab5cd4a7f6eb 2026-03-24 15:26:12,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:26:12,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00a027544ab5cd4a7f6eb HTTP/1.1" 200 None 2026-03-24 15:26:12,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:26:12,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 15:26:12,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:26:12,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00a027544ab5cd4a7f6eb HTTP/1.1" 200 None 2026-03-24 15:26:12,626 - lava_callback - ERROR - Error processing callback for node 69c00a027544ab5cd4a7f6eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36928 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:26:52,617 - lava_callback - INFO - Received callback for node 69c00a277544ab5cd4a7f6f2 INFO: 172.20.0.1:57574 - "POST /node/69c00a277544ab5cd4a7f6f2 HTTP/1.0" 202 Accepted 2026-03-24 15:26:52,623 - lava_callback - INFO - Starting async job submit for node 69c00a277544ab5cd4a7f6f2 2026-03-24 15:26:52,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:26:52,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00a277544ab5cd4a7f6f2 HTTP/1.1" 200 None 2026-03-24 15:26:52,639 - lava_callback - DEBUG - Processing node 69c00a277544ab5cd4a7f6f2: baseline-arm-clabbe 2026-03-24 15:26:52,639 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00a277544ab5cd4a7f6f2 2026-03-24 15:26:52,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:26:52,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00a277544ab5cd4a7f6f2 HTTP/1.1" 200 None 2026-03-24 15:26:52,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:26:52,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 15:26:52,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:26:52,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00a277544ab5cd4a7f6f2 HTTP/1.1" 200 None 2026-03-24 15:26:52,695 - lava_callback - ERROR - Error processing callback for node 69c00a277544ab5cd4a7f6f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56022 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:27:12,593 - lava_callback - INFO - Received callback for node 69c00a4b7544ab5cd4a7f6fd INFO: 172.20.0.1:40166 - "POST /node/69c00a4b7544ab5cd4a7f6fd HTTP/1.0" 202 Accepted 2026-03-24 15:27:12,599 - lava_callback - INFO - Starting async job submit for node 69c00a4b7544ab5cd4a7f6fd 2026-03-24 15:27:12,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:27:12,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00a4b7544ab5cd4a7f6fd HTTP/1.1" 200 None 2026-03-24 15:27:12,615 - lava_callback - DEBUG - Processing node 69c00a4b7544ab5cd4a7f6fd: baseline-arm-clabbe 2026-03-24 15:27:12,616 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00a4b7544ab5cd4a7f6fd 2026-03-24 15:27:12,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:27:12,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00a4b7544ab5cd4a7f6fd HTTP/1.1" 200 None 2026-03-24 15:27:12,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:27:12,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 15:27:12,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:27:12,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00a4b7544ab5cd4a7f6fd HTTP/1.1" 200 None 2026-03-24 15:27:12,678 - lava_callback - ERROR - Error processing callback for node 69c00a4b7544ab5cd4a7f6fd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51668 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:27:52,669 - lava_callback - INFO - Received callback for node 69c00a707544ab5cd4a7f706 INFO: 172.20.0.1:44298 - "POST /node/69c00a707544ab5cd4a7f706 HTTP/1.0" 202 Accepted 2026-03-24 15:27:52,675 - lava_callback - INFO - Starting async job submit for node 69c00a707544ab5cd4a7f706 2026-03-24 15:27:52,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:27:52,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00a707544ab5cd4a7f706 HTTP/1.1" 200 None 2026-03-24 15:27:52,693 - lava_callback - DEBUG - Processing node 69c00a707544ab5cd4a7f706: baseline-arm-clabbe 2026-03-24 15:27:52,694 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00a707544ab5cd4a7f706 2026-03-24 15:27:52,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:27:52,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00a707544ab5cd4a7f706 HTTP/1.1" 200 None 2026-03-24 15:27:52,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:27:52,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bfa5737544ab5cd4a7d9ce HTTP/1.1" 200 None 2026-03-24 15:27:52,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:27:52,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00a707544ab5cd4a7f706 HTTP/1.1" 200 None 2026-03-24 15:27:52,747 - lava_callback - ERROR - Error processing callback for node 69c00a707544ab5cd4a7f706: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35782 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:28:52,662 - lava_callback - INFO - Received callback for node 69c00aa47544ab5cd4a7f71b INFO: 172.20.0.1:45904 - "POST /node/69c00aa47544ab5cd4a7f71b HTTP/1.0" 202 Accepted 2026-03-24 15:28:52,667 - lava_callback - INFO - Starting async job submit for node 69c00aa47544ab5cd4a7f71b 2026-03-24 15:28:52,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:28:52,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00aa47544ab5cd4a7f71b HTTP/1.1" 200 None 2026-03-24 15:28:52,684 - lava_callback - DEBUG - Processing node 69c00aa47544ab5cd4a7f71b: baseline-arm64-clabbe 2026-03-24 15:28:52,684 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00aa47544ab5cd4a7f71b 2026-03-24 15:28:52,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:28:52,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00aa47544ab5cd4a7f71b HTTP/1.1" 200 None 2026-03-24 15:28:52,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:28:52,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:28:52,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:28:52,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00aa47544ab5cd4a7f71b HTTP/1.1" 200 None 2026-03-24 15:28:52,744 - lava_callback - ERROR - Error processing callback for node 69c00aa47544ab5cd4a7f71b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40050 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:30:12,692 - lava_callback - INFO - Received callback for node 69c00af07544ab5cd4a7f726 INFO: 172.20.0.1:49514 - "POST /node/69c00af07544ab5cd4a7f726 HTTP/1.0" 202 Accepted 2026-03-24 15:30:12,697 - lava_callback - INFO - Starting async job submit for node 69c00af07544ab5cd4a7f726 2026-03-24 15:30:12,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:30:12,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00af07544ab5cd4a7f726 HTTP/1.1" 200 None 2026-03-24 15:30:12,712 - lava_callback - DEBUG - Processing node 69c00af07544ab5cd4a7f726: baseline-arm64-clabbe 2026-03-24 15:30:12,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00af07544ab5cd4a7f726 2026-03-24 15:30:12,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:30:12,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00af07544ab5cd4a7f726 HTTP/1.1" 200 None 2026-03-24 15:30:12,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:30:12,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:30:12,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:30:12,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00af07544ab5cd4a7f726 HTTP/1.1" 200 None 2026-03-24 15:30:12,770 - lava_callback - ERROR - Error processing callback for node 69c00af07544ab5cd4a7f726: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50140 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:31:12,796 - lava_callback - INFO - Received callback for node 69c00b327544ab5cd4a7f72f INFO: 172.20.0.1:42238 - "POST /node/69c00b327544ab5cd4a7f72f HTTP/1.0" 202 Accepted 2026-03-24 15:31:12,802 - lava_callback - INFO - Starting async job submit for node 69c00b327544ab5cd4a7f72f 2026-03-24 15:31:12,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:31:12,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00b327544ab5cd4a7f72f HTTP/1.1" 200 None 2026-03-24 15:31:12,828 - lava_callback - DEBUG - Processing node 69c00b327544ab5cd4a7f72f: baseline-arm-clabbe 2026-03-24 15:31:12,828 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00b327544ab5cd4a7f72f 2026-03-24 15:31:12,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:31:12,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00b327544ab5cd4a7f72f HTTP/1.1" 200 None 2026-03-24 15:31:12,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:31:12,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:31:12,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:31:12,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00b327544ab5cd4a7f72f HTTP/1.1" 200 None 2026-03-24 15:31:12,884 - lava_callback - ERROR - Error processing callback for node 69c00b327544ab5cd4a7f72f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49224 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:32:01,017 - lava_callback - INFO - Received callback for node 69c00b5e7544ab5cd4a7f736 INFO: 172.20.0.1:60092 - "POST /node/69c00b5e7544ab5cd4a7f736 HTTP/1.0" 202 Accepted 2026-03-24 15:32:01,023 - lava_callback - INFO - Starting async job submit for node 69c00b5e7544ab5cd4a7f736 2026-03-24 15:32:01,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:32:01,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00b5e7544ab5cd4a7f736 HTTP/1.1" 200 None 2026-03-24 15:32:01,040 - lava_callback - DEBUG - Processing node 69c00b5e7544ab5cd4a7f736: baseline-arm64-baylibre 2026-03-24 15:32:01,040 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00b5e7544ab5cd4a7f736 2026-03-24 15:32:01,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:32:01,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00b5e7544ab5cd4a7f736 HTTP/1.1" 200 None 2026-03-24 15:32:01,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:32:01,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:32:01,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:32:01,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00b5e7544ab5cd4a7f736 HTTP/1.1" 200 None 2026-03-24 15:32:01,123 - lava_callback - ERROR - Error processing callback for node 69c00b5e7544ab5cd4a7f736: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51946 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:32:52,879 - lava_callback - INFO - Received callback for node 69c00ba07544ab5cd4a7f73f INFO: 172.20.0.1:54342 - "POST /node/69c00ba07544ab5cd4a7f73f HTTP/1.0" 202 Accepted 2026-03-24 15:32:52,885 - lava_callback - INFO - Starting async job submit for node 69c00ba07544ab5cd4a7f73f 2026-03-24 15:32:52,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:32:52,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00ba07544ab5cd4a7f73f HTTP/1.1" 200 None 2026-03-24 15:32:52,906 - lava_callback - DEBUG - Processing node 69c00ba07544ab5cd4a7f73f: baseline-arm-clabbe 2026-03-24 15:32:52,907 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00ba07544ab5cd4a7f73f 2026-03-24 15:32:52,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:32:52,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00ba07544ab5cd4a7f73f HTTP/1.1" 200 None 2026-03-24 15:32:52,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:32:52,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:32:52,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:32:52,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00ba07544ab5cd4a7f73f HTTP/1.1" 200 None 2026-03-24 15:32:52,968 - lava_callback - ERROR - Error processing callback for node 69c00ba07544ab5cd4a7f73f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38800 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:34894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51788 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:33:52,919 - lava_callback - INFO - Received callback for node 69c00bd27544ab5cd4a7f74d INFO: 172.20.0.1:50682 - "POST /node/69c00bd27544ab5cd4a7f74d HTTP/1.0" 202 Accepted 2026-03-24 15:33:52,924 - lava_callback - INFO - Starting async job submit for node 69c00bd27544ab5cd4a7f74d 2026-03-24 15:33:52,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:33:52,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00bd27544ab5cd4a7f74d HTTP/1.1" 200 None 2026-03-24 15:33:52,940 - lava_callback - DEBUG - Processing node 69c00bd27544ab5cd4a7f74d: baseline-arm64-clabbe 2026-03-24 15:33:52,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00bd27544ab5cd4a7f74d 2026-03-24 15:33:52,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:33:52,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00bd27544ab5cd4a7f74d HTTP/1.1" 200 None 2026-03-24 15:33:52,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:33:52,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:33:52,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:33:53,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00bd27544ab5cd4a7f74d HTTP/1.1" 200 None 2026-03-24 15:33:53,005 - lava_callback - ERROR - Error processing callback for node 69c00bd27544ab5cd4a7f74d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50700 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:35:12,934 - lava_callback - INFO - Received callback for node 69c00c1d7544ab5cd4a7f756 INFO: 172.20.0.1:48478 - "POST /node/69c00c1d7544ab5cd4a7f756 HTTP/1.0" 202 Accepted 2026-03-24 15:35:12,939 - lava_callback - INFO - Starting async job submit for node 69c00c1d7544ab5cd4a7f756 2026-03-24 15:35:12,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:35:12,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00c1d7544ab5cd4a7f756 HTTP/1.1" 200 None 2026-03-24 15:35:12,958 - lava_callback - DEBUG - Processing node 69c00c1d7544ab5cd4a7f756: baseline-arm64-clabbe 2026-03-24 15:35:12,958 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00c1d7544ab5cd4a7f756 2026-03-24 15:35:12,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:35:12,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00c1d7544ab5cd4a7f756 HTTP/1.1" 200 None 2026-03-24 15:35:12,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:35:12,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:35:12,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:35:13,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00c1d7544ab5cd4a7f756 HTTP/1.1" 200 None 2026-03-24 15:35:13,027 - lava_callback - ERROR - Error processing callback for node 69c00c1d7544ab5cd4a7f756: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36184 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:36:13,011 - lava_callback - INFO - Received callback for node 69c00c687544ab5cd4a7f75f INFO: 172.20.0.1:49498 - "POST /node/69c00c687544ab5cd4a7f75f HTTP/1.0" 202 Accepted 2026-03-24 15:36:13,017 - lava_callback - INFO - Starting async job submit for node 69c00c687544ab5cd4a7f75f 2026-03-24 15:36:13,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:36:13,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00c687544ab5cd4a7f75f HTTP/1.1" 200 None 2026-03-24 15:36:13,057 - lava_callback - DEBUG - Processing node 69c00c687544ab5cd4a7f75f: baseline-arm64-clabbe 2026-03-24 15:36:13,057 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00c687544ab5cd4a7f75f 2026-03-24 15:36:13,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:36:13,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00c687544ab5cd4a7f75f HTTP/1.1" 200 None 2026-03-24 15:36:13,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:36:13,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:36:13,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:36:13,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00c687544ab5cd4a7f75f HTTP/1.1" 200 None 2026-03-24 15:36:13,113 - lava_callback - ERROR - Error processing callback for node 69c00c687544ab5cd4a7f75f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50384 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59366 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:37:33,087 - lava_callback - INFO - Received callback for node 69c00cb57544ab5cd4a7f76a INFO: 172.20.0.1:36024 - "POST /node/69c00cb57544ab5cd4a7f76a HTTP/1.0" 202 Accepted 2026-03-24 15:37:33,092 - lava_callback - INFO - Starting async job submit for node 69c00cb57544ab5cd4a7f76a 2026-03-24 15:37:33,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:37:33,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00cb57544ab5cd4a7f76a HTTP/1.1" 200 None 2026-03-24 15:37:33,109 - lava_callback - DEBUG - Processing node 69c00cb57544ab5cd4a7f76a: baseline-arm64-clabbe 2026-03-24 15:37:33,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00cb57544ab5cd4a7f76a 2026-03-24 15:37:33,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:37:33,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00cb57544ab5cd4a7f76a HTTP/1.1" 200 None 2026-03-24 15:37:33,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:37:33,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:37:33,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:37:33,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00cb57544ab5cd4a7f76a HTTP/1.1" 200 None 2026-03-24 15:37:33,168 - lava_callback - ERROR - Error processing callback for node 69c00cb57544ab5cd4a7f76a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56422 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:38:53,112 - lava_callback - INFO - Received callback for node 69c00d017544ab5cd4a7f773 INFO: 172.20.0.1:49446 - "POST /node/69c00d017544ab5cd4a7f773 HTTP/1.0" 202 Accepted 2026-03-24 15:38:53,117 - lava_callback - INFO - Starting async job submit for node 69c00d017544ab5cd4a7f773 2026-03-24 15:38:53,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:38:53,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00d017544ab5cd4a7f773 HTTP/1.1" 200 None 2026-03-24 15:38:53,133 - lava_callback - DEBUG - Processing node 69c00d017544ab5cd4a7f773: baseline-arm64-clabbe 2026-03-24 15:38:53,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00d017544ab5cd4a7f773 2026-03-24 15:38:53,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:38:53,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00d017544ab5cd4a7f773 HTTP/1.1" 200 None 2026-03-24 15:38:53,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:38:53,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:38:53,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:38:53,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00d017544ab5cd4a7f773 HTTP/1.1" 200 None 2026-03-24 15:38:53,221 - lava_callback - ERROR - Error processing callback for node 69c00d017544ab5cd4a7f773: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49156 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:40:13,171 - lava_callback - INFO - Received callback for node 69c00d4d7544ab5cd4a7f77d INFO: 172.20.0.1:43218 - "POST /node/69c00d4d7544ab5cd4a7f77d HTTP/1.0" 202 Accepted 2026-03-24 15:40:13,179 - lava_callback - INFO - Starting async job submit for node 69c00d4d7544ab5cd4a7f77d 2026-03-24 15:40:13,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:40:13,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00d4d7544ab5cd4a7f77d HTTP/1.1" 200 None 2026-03-24 15:40:13,205 - lava_callback - DEBUG - Processing node 69c00d4d7544ab5cd4a7f77d: baseline-arm64-clabbe 2026-03-24 15:40:13,205 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00d4d7544ab5cd4a7f77d 2026-03-24 15:40:13,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:40:13,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00d4d7544ab5cd4a7f77d HTTP/1.1" 200 None 2026-03-24 15:40:13,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:40:13,246 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bd027905e4bbd6ba634c66 HTTP/1.1" 200 None 2026-03-24 15:40:13,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:40:13,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00d4d7544ab5cd4a7f77d HTTP/1.1" 200 None 2026-03-24 15:40:13,274 - lava_callback - ERROR - Error processing callback for node 69c00d4d7544ab5cd4a7f77d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41230 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:41:13,221 - lava_callback - INFO - Received callback for node 69c00d8a7544ab5cd4a7f783 INFO: 172.20.0.1:55340 - "POST /node/69c00d8a7544ab5cd4a7f783 HTTP/1.0" 202 Accepted 2026-03-24 15:41:13,227 - lava_callback - INFO - Starting async job submit for node 69c00d8a7544ab5cd4a7f783 2026-03-24 15:41:13,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:41:13,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00d8a7544ab5cd4a7f783 HTTP/1.1" 200 None 2026-03-24 15:41:13,244 - lava_callback - DEBUG - Processing node 69c00d8a7544ab5cd4a7f783: baseline-arm-clabbe 2026-03-24 15:41:13,244 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00d8a7544ab5cd4a7f783 2026-03-24 15:41:13,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:41:13,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00d8a7544ab5cd4a7f783 HTTP/1.1" 200 None 2026-03-24 15:41:13,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:41:13,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:41:13,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:41:13,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00d8a7544ab5cd4a7f783 HTTP/1.1" 200 None 2026-03-24 15:41:13,371 - lava_callback - ERROR - Error processing callback for node 69c00d8a7544ab5cd4a7f783: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33670 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:41:53,190 - lava_callback - INFO - Received callback for node 69c00dae7544ab5cd4a7f78c INFO: 172.20.0.1:43458 - "POST /node/69c00dae7544ab5cd4a7f78c HTTP/1.0" 202 Accepted 2026-03-24 15:41:53,195 - lava_callback - INFO - Starting async job submit for node 69c00dae7544ab5cd4a7f78c 2026-03-24 15:41:53,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:41:53,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00dae7544ab5cd4a7f78c HTTP/1.1" 200 None 2026-03-24 15:41:53,213 - lava_callback - DEBUG - Processing node 69c00dae7544ab5cd4a7f78c: baseline-arm-clabbe 2026-03-24 15:41:53,214 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00dae7544ab5cd4a7f78c 2026-03-24 15:41:53,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:41:53,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00dae7544ab5cd4a7f78c HTTP/1.1" 200 None 2026-03-24 15:41:53,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:41:53,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:41:53,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:41:53,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00dae7544ab5cd4a7f78c HTTP/1.1" 200 None 2026-03-24 15:41:53,270 - lava_callback - ERROR - Error processing callback for node 69c00dae7544ab5cd4a7f78c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55788 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:42:13,289 - lava_callback - INFO - Received callback for node 69c00dd27544ab5cd4a7f797 INFO: 172.20.0.1:35266 - "POST /node/69c00dd27544ab5cd4a7f797 HTTP/1.0" 202 Accepted 2026-03-24 15:42:13,294 - lava_callback - INFO - Starting async job submit for node 69c00dd27544ab5cd4a7f797 2026-03-24 15:42:13,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:42:13,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00dd27544ab5cd4a7f797 HTTP/1.1" 200 None 2026-03-24 15:42:13,310 - lava_callback - DEBUG - Processing node 69c00dd27544ab5cd4a7f797: baseline-arm-clabbe 2026-03-24 15:42:13,310 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00dd27544ab5cd4a7f797 2026-03-24 15:42:13,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:42:13,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00dd27544ab5cd4a7f797 HTTP/1.1" 200 None 2026-03-24 15:42:13,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:42:13,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:42:13,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:42:13,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00dd27544ab5cd4a7f797 HTTP/1.1" 200 None 2026-03-24 15:42:13,371 - lava_callback - ERROR - Error processing callback for node 69c00dd27544ab5cd4a7f797: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:42:53,248 - lava_callback - INFO - Received callback for node 69c00df67544ab5cd4a7f7a1 INFO: 172.20.0.1:46618 - "POST /node/69c00df67544ab5cd4a7f7a1 HTTP/1.0" 202 Accepted 2026-03-24 15:42:53,254 - lava_callback - INFO - Starting async job submit for node 69c00df67544ab5cd4a7f7a1 2026-03-24 15:42:53,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:42:53,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00df67544ab5cd4a7f7a1 HTTP/1.1" 200 None 2026-03-24 15:42:53,272 - lava_callback - DEBUG - Processing node 69c00df67544ab5cd4a7f7a1: baseline-arm-clabbe 2026-03-24 15:42:53,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00df67544ab5cd4a7f7a1 2026-03-24 15:42:53,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:42:53,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00df67544ab5cd4a7f7a1 HTTP/1.1" 200 None 2026-03-24 15:42:53,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:42:53,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f6d537ae4d3c8236e46 HTTP/1.1" 200 None 2026-03-24 15:42:53,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:42:53,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00df67544ab5cd4a7f7a1 HTTP/1.1" 200 None 2026-03-24 15:42:53,325 - lava_callback - ERROR - Error processing callback for node 69c00df67544ab5cd4a7f7a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36364 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:43:41,856 - lava_callback - INFO - Received callback for node 69c00e1f7544ab5cd4a7f7ad INFO: 172.20.0.1:40378 - "POST /node/69c00e1f7544ab5cd4a7f7ad HTTP/1.0" 202 Accepted 2026-03-24 15:43:41,862 - lava_callback - INFO - Starting async job submit for node 69c00e1f7544ab5cd4a7f7ad 2026-03-24 15:43:41,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:43:41,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00e1f7544ab5cd4a7f7ad HTTP/1.1" 200 None 2026-03-24 15:43:41,878 - lava_callback - DEBUG - Processing node 69c00e1f7544ab5cd4a7f7ad: baseline-arm64-baylibre 2026-03-24 15:43:41,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00e1f7544ab5cd4a7f7ad 2026-03-24 15:43:41,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:43:41,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00e1f7544ab5cd4a7f7ad HTTP/1.1" 200 None 2026-03-24 15:43:41,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:43:41,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:43:41,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:43:41,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00e1f7544ab5cd4a7f7ad HTTP/1.1" 200 None 2026-03-24 15:43:41,943 - lava_callback - ERROR - Error processing callback for node 69c00e1f7544ab5cd4a7f7ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45148 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:45:01,729 - lava_callback - INFO - Received callback for node 69c00e6d7544ab5cd4a7f7b4 INFO: 172.20.0.1:56672 - "POST /node/69c00e6d7544ab5cd4a7f7b4 HTTP/1.0" 202 Accepted 2026-03-24 15:45:01,735 - lava_callback - INFO - Starting async job submit for node 69c00e6d7544ab5cd4a7f7b4 2026-03-24 15:45:01,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:45:01,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00e6d7544ab5cd4a7f7b4 HTTP/1.1" 200 None 2026-03-24 15:45:01,752 - lava_callback - DEBUG - Processing node 69c00e6d7544ab5cd4a7f7b4: baseline-arm64-baylibre 2026-03-24 15:45:01,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00e6d7544ab5cd4a7f7b4 2026-03-24 15:45:01,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:45:01,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00e6d7544ab5cd4a7f7b4 HTTP/1.1" 200 None 2026-03-24 15:45:01,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:45:01,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:45:01,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:45:01,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00e6d7544ab5cd4a7f7b4 HTTP/1.1" 200 None 2026-03-24 15:45:01,826 - lava_callback - ERROR - Error processing callback for node 69c00e6d7544ab5cd4a7f7b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41270 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:46:22,487 - lava_callback - INFO - Received callback for node 69c00eb87544ab5cd4a7f7c0 INFO: 172.20.0.1:41286 - "POST /node/69c00eb87544ab5cd4a7f7c0 HTTP/1.0" 202 Accepted 2026-03-24 15:46:22,492 - lava_callback - INFO - Starting async job submit for node 69c00eb87544ab5cd4a7f7c0 2026-03-24 15:46:22,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:46:22,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00eb87544ab5cd4a7f7c0 HTTP/1.1" 200 None 2026-03-24 15:46:22,509 - lava_callback - DEBUG - Processing node 69c00eb87544ab5cd4a7f7c0: baseline-arm64-baylibre 2026-03-24 15:46:22,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00eb87544ab5cd4a7f7c0 2026-03-24 15:46:22,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:46:22,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00eb87544ab5cd4a7f7c0 HTTP/1.1" 200 None 2026-03-24 15:46:22,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:46:22,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:46:22,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:46:22,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00eb87544ab5cd4a7f7c0 HTTP/1.1" 200 None 2026-03-24 15:46:22,574 - lava_callback - ERROR - Error processing callback for node 69c00eb87544ab5cd4a7f7c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44020 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:47:21,707 - lava_callback - INFO - Received callback for node 69c00f047544ab5cd4a7f7c6 INFO: 172.20.0.1:49312 - "POST /node/69c00f047544ab5cd4a7f7c6 HTTP/1.0" 202 Accepted 2026-03-24 15:47:21,713 - lava_callback - INFO - Starting async job submit for node 69c00f047544ab5cd4a7f7c6 2026-03-24 15:47:21,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:47:21,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00f047544ab5cd4a7f7c6 HTTP/1.1" 200 None 2026-03-24 15:47:21,732 - lava_callback - DEBUG - Processing node 69c00f047544ab5cd4a7f7c6: baseline-arm64-baylibre 2026-03-24 15:47:21,732 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00f047544ab5cd4a7f7c6 2026-03-24 15:47:21,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:47:21,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00f047544ab5cd4a7f7c6 HTTP/1.1" 200 None 2026-03-24 15:47:21,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:47:21,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:47:21,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:47:21,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00f047544ab5cd4a7f7c6 HTTP/1.1" 200 None 2026-03-24 15:47:21,792 - lava_callback - ERROR - Error processing callback for node 69c00f047544ab5cd4a7f7c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53904 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:48:41,856 - lava_callback - INFO - Received callback for node 69c00f4d7544ab5cd4a7f7d4 INFO: 172.20.0.1:49166 - "POST /node/69c00f4d7544ab5cd4a7f7d4 HTTP/1.0" 202 Accepted 2026-03-24 15:48:41,862 - lava_callback - INFO - Starting async job submit for node 69c00f4d7544ab5cd4a7f7d4 2026-03-24 15:48:41,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:48:41,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00f4d7544ab5cd4a7f7d4 HTTP/1.1" 200 None 2026-03-24 15:48:41,878 - lava_callback - DEBUG - Processing node 69c00f4d7544ab5cd4a7f7d4: baseline-arm64-baylibre 2026-03-24 15:48:41,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00f4d7544ab5cd4a7f7d4 2026-03-24 15:48:41,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:48:41,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00f4d7544ab5cd4a7f7d4 HTTP/1.1" 200 None 2026-03-24 15:48:41,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:48:41,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:48:41,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:48:41,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00f4d7544ab5cd4a7f7d4 HTTP/1.1" 200 None 2026-03-24 15:48:41,941 - lava_callback - ERROR - Error processing callback for node 69c00f4d7544ab5cd4a7f7d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43698 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:50:02,002 - lava_callback - INFO - Received callback for node 69c00f987544ab5cd4a7f7dd INFO: 172.20.0.1:48096 - "POST /node/69c00f987544ab5cd4a7f7dd HTTP/1.0" 202 Accepted 2026-03-24 15:50:02,007 - lava_callback - INFO - Starting async job submit for node 69c00f987544ab5cd4a7f7dd 2026-03-24 15:50:02,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:50:02,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00f987544ab5cd4a7f7dd HTTP/1.1" 200 None 2026-03-24 15:50:02,023 - lava_callback - DEBUG - Processing node 69c00f987544ab5cd4a7f7dd: baseline-arm64-baylibre 2026-03-24 15:50:02,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00f987544ab5cd4a7f7dd 2026-03-24 15:50:02,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:50:02,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00f987544ab5cd4a7f7dd HTTP/1.1" 200 None 2026-03-24 15:50:02,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:50:02,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:50:02,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:50:02,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00f987544ab5cd4a7f7dd HTTP/1.1" 200 None 2026-03-24 15:50:02,081 - lava_callback - ERROR - Error processing callback for node 69c00f987544ab5cd4a7f7dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41048 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:51:13,713 - lava_callback - INFO - Received callback for node 69c00fe87544ab5cd4a7f7ea INFO: 172.20.0.1:48478 - "POST /node/69c00fe87544ab5cd4a7f7ea HTTP/1.0" 202 Accepted 2026-03-24 15:51:13,718 - lava_callback - INFO - Starting async job submit for node 69c00fe87544ab5cd4a7f7ea 2026-03-24 15:51:13,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:51:13,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00fe87544ab5cd4a7f7ea HTTP/1.1" 200 None 2026-03-24 15:51:13,735 - lava_callback - DEBUG - Processing node 69c00fe87544ab5cd4a7f7ea: baseline-arm64-clabbe 2026-03-24 15:51:13,735 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c00fe87544ab5cd4a7f7ea 2026-03-24 15:51:13,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:51:13,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c00fe87544ab5cd4a7f7ea HTTP/1.1" 200 None 2026-03-24 15:51:13,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:51:13,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:51:13,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:51:13,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c00fe87544ab5cd4a7f7ea HTTP/1.1" 200 None 2026-03-24 15:51:13,791 - lava_callback - ERROR - Error processing callback for node 69c00fe87544ab5cd4a7f7ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48490 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44248 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:52:33,721 - lava_callback - INFO - Received callback for node 69c010337544ab5cd4a7f7f2 INFO: 172.20.0.1:37704 - "POST /node/69c010337544ab5cd4a7f7f2 HTTP/1.0" 202 Accepted 2026-03-24 15:52:33,728 - lava_callback - INFO - Starting async job submit for node 69c010337544ab5cd4a7f7f2 2026-03-24 15:52:33,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:52:33,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c010337544ab5cd4a7f7f2 HTTP/1.1" 200 None 2026-03-24 15:52:33,746 - lava_callback - DEBUG - Processing node 69c010337544ab5cd4a7f7f2: baseline-arm64-clabbe 2026-03-24 15:52:33,746 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c010337544ab5cd4a7f7f2 2026-03-24 15:52:33,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:52:33,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c010337544ab5cd4a7f7f2 HTTP/1.1" 200 None 2026-03-24 15:52:33,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:52:33,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:52:33,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:52:33,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c010337544ab5cd4a7f7f2 HTTP/1.1" 200 None 2026-03-24 15:52:33,800 - lava_callback - ERROR - Error processing callback for node 69c010337544ab5cd4a7f7f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48358 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:53:53,786 - lava_callback - INFO - Received callback for node 69c0107e7544ab5cd4a7f7fe INFO: 172.20.0.1:41522 - "POST /node/69c0107e7544ab5cd4a7f7fe HTTP/1.0" 202 Accepted 2026-03-24 15:53:53,791 - lava_callback - INFO - Starting async job submit for node 69c0107e7544ab5cd4a7f7fe 2026-03-24 15:53:53,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:53:53,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0107e7544ab5cd4a7f7fe HTTP/1.1" 200 None 2026-03-24 15:53:53,810 - lava_callback - DEBUG - Processing node 69c0107e7544ab5cd4a7f7fe: baseline-arm64-clabbe 2026-03-24 15:53:53,810 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0107e7544ab5cd4a7f7fe 2026-03-24 15:53:53,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:53:53,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0107e7544ab5cd4a7f7fe HTTP/1.1" 200 None 2026-03-24 15:53:53,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:53:53,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:53:53,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:53:53,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0107e7544ab5cd4a7f7fe HTTP/1.1" 200 None 2026-03-24 15:53:53,872 - lava_callback - ERROR - Error processing callback for node 69c0107e7544ab5cd4a7f7fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44220 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:54:53,829 - lava_callback - INFO - Received callback for node 69c010c87544ab5cd4a7f806 INFO: 172.20.0.1:60588 - "POST /node/69c010c87544ab5cd4a7f806 HTTP/1.0" 202 Accepted 2026-03-24 15:54:53,835 - lava_callback - INFO - Starting async job submit for node 69c010c87544ab5cd4a7f806 2026-03-24 15:54:53,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:54:53,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c010c87544ab5cd4a7f806 HTTP/1.1" 200 None 2026-03-24 15:54:53,855 - lava_callback - DEBUG - Processing node 69c010c87544ab5cd4a7f806: baseline-arm64-clabbe 2026-03-24 15:54:53,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c010c87544ab5cd4a7f806 2026-03-24 15:54:53,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:54:53,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c010c87544ab5cd4a7f806 HTTP/1.1" 200 None 2026-03-24 15:54:53,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:54:53,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:54:53,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:54:53,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c010c87544ab5cd4a7f806 HTTP/1.1" 200 None 2026-03-24 15:54:53,918 - lava_callback - ERROR - Error processing callback for node 69c010c87544ab5cd4a7f806: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34810 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:56:13,901 - lava_callback - INFO - Received callback for node 69c011147544ab5cd4a7f814 INFO: 172.20.0.1:59058 - "POST /node/69c011147544ab5cd4a7f814 HTTP/1.0" 202 Accepted 2026-03-24 15:56:13,906 - lava_callback - INFO - Starting async job submit for node 69c011147544ab5cd4a7f814 2026-03-24 15:56:13,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:56:13,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c011147544ab5cd4a7f814 HTTP/1.1" 200 None 2026-03-24 15:56:13,936 - lava_callback - DEBUG - Processing node 69c011147544ab5cd4a7f814: baseline-arm64-clabbe 2026-03-24 15:56:13,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c011147544ab5cd4a7f814 2026-03-24 15:56:13,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:56:13,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c011147544ab5cd4a7f814 HTTP/1.1" 200 None 2026-03-24 15:56:13,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:56:13,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:56:13,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:56:14,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c011147544ab5cd4a7f814 HTTP/1.1" 200 None 2026-03-24 15:56:14,034 - lava_callback - ERROR - Error processing callback for node 69c011147544ab5cd4a7f814: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54298 - "GET / HTTP/1.0" 200 OK 2026-03-24 15:57:33,949 - lava_callback - INFO - Received callback for node 69c0115f7544ab5cd4a7f81a INFO: 172.20.0.1:36166 - "POST /node/69c0115f7544ab5cd4a7f81a HTTP/1.0" 202 Accepted 2026-03-24 15:57:33,955 - lava_callback - INFO - Starting async job submit for node 69c0115f7544ab5cd4a7f81a 2026-03-24 15:57:33,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:57:33,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0115f7544ab5cd4a7f81a HTTP/1.1" 200 None 2026-03-24 15:57:33,972 - lava_callback - DEBUG - Processing node 69c0115f7544ab5cd4a7f81a: baseline-arm64-clabbe 2026-03-24 15:57:33,972 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c0115f7544ab5cd4a7f81a 2026-03-24 15:57:33,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:57:33,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0115f7544ab5cd4a7f81a HTTP/1.1" 200 None 2026-03-24 15:57:33,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:57:34,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69b7e980113612b7716b9e01 HTTP/1.1" 200 None 2026-03-24 15:57:34,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 15:57:34,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c0115f7544ab5cd4a7f81a HTTP/1.1" 200 None 2026-03-24 15:57:34,033 - lava_callback - ERROR - Error processing callback for node 69c0115f7544ab5cd4a7f81a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38352 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50728 - "GET / HTTP/1.0" 200 OK 2026-03-24 16:03:22,554 - lava_callback - INFO - Received callback for node 69c012ba7544ab5cd4a7f82f INFO: 172.20.0.1:50736 - "POST /node/69c012ba7544ab5cd4a7f82f HTTP/1.0" 202 Accepted 2026-03-24 16:03:22,560 - lava_callback - INFO - Starting async job submit for node 69c012ba7544ab5cd4a7f82f 2026-03-24 16:03:22,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:03:22,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c012ba7544ab5cd4a7f82f HTTP/1.1" 200 None 2026-03-24 16:03:22,577 - lava_callback - DEBUG - Processing node 69c012ba7544ab5cd4a7f82f: baseline-arm64-baylibre 2026-03-24 16:03:22,577 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c012ba7544ab5cd4a7f82f 2026-03-24 16:03:22,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:03:22,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c012ba7544ab5cd4a7f82f HTTP/1.1" 200 None 2026-03-24 16:03:22,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:03:22,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ba5f7e537ae4d3c8236e55 HTTP/1.1" 200 None 2026-03-24 16:03:22,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-24 16:03:22,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c012ba7544ab5cd4a7f82f HTTP/1.1" 200 None 2026-03-24 16:03:22,652 - lava_callback - ERROR - Error processing callback for node 69c012ba7544ab5cd4a7f82f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50986 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]