INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54888 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:13:31,913 - lava_callback - INFO - Received callback for node 69c3991abbc022e792643e6b 2026-03-27 08:13:31,920 - lava_callback - INFO - Starting async job submit for node 69c3991abbc022e792643e6b INFO: 172.20.0.1:33518 - "POST /node/69c3991abbc022e792643e6b HTTP/1.0" 202 Accepted 2026-03-27 08:13:31,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:13:31,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3991abbc022e792643e6b HTTP/1.1" 200 None 2026-03-27 08:13:31,942 - lava_callback - DEBUG - Processing node 69c3991abbc022e792643e6b: baseline-arm64-baylibre 2026-03-27 08:13:31,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3991abbc022e792643e6b 2026-03-27 08:13:31,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:13:31,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3991abbc022e792643e6b HTTP/1.1" 200 None 2026-03-27 08:13:31,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:13:31,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 08:13:31,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:13:32,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3991abbc022e792643e6b HTTP/1.1" 200 None 2026-03-27 08:13:32,011 - lava_callback - ERROR - Error processing callback for node 69c3991abbc022e792643e6b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53500 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:14:51,988 - lava_callback - INFO - Received callback for node 69c3996abbc022e792643e79 INFO: 172.20.0.1:55228 - "POST /node/69c3996abbc022e792643e79 HTTP/1.0" 202 Accepted 2026-03-27 08:14:51,994 - lava_callback - INFO - Starting async job submit for node 69c3996abbc022e792643e79 2026-03-27 08:14:51,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:14:52,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3996abbc022e792643e79 HTTP/1.1" 200 None 2026-03-27 08:14:52,015 - lava_callback - DEBUG - Processing node 69c3996abbc022e792643e79: baseline-arm64-baylibre 2026-03-27 08:14:52,015 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3996abbc022e792643e79 2026-03-27 08:14:52,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:14:52,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3996abbc022e792643e79 HTTP/1.1" 200 None 2026-03-27 08:14:52,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:14:52,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 08:14:52,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:14:52,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3996abbc022e792643e79 HTTP/1.1" 200 None 2026-03-27 08:14:52,073 - lava_callback - ERROR - Error processing callback for node 69c3996abbc022e792643e79: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46410 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:16:32,071 - lava_callback - INFO - Received callback for node 69c399d0bbc022e792643ea2 INFO: 172.20.0.1:37150 - "POST /node/69c399d0bbc022e792643ea2 HTTP/1.0" 202 Accepted 2026-03-27 08:16:32,080 - lava_callback - INFO - Starting async job submit for node 69c399d0bbc022e792643ea2 2026-03-27 08:16:32,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:16:32,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399d0bbc022e792643ea2 HTTP/1.1" 200 None 2026-03-27 08:16:32,107 - lava_callback - DEBUG - Processing node 69c399d0bbc022e792643ea2: baseline-arm64-baylibre 2026-03-27 08:16:32,107 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c399d0bbc022e792643ea2 2026-03-27 08:16:32,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:16:32,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399d0bbc022e792643ea2 HTTP/1.1" 200 None 2026-03-27 08:16:32,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:16:32,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 08:16:32,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:16:32,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c399d0bbc022e792643ea2 HTTP/1.1" 200 None 2026-03-27 08:16:32,208 - lava_callback - ERROR - Error processing callback for node 69c399d0bbc022e792643ea2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52022 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:17:52,163 - lava_callback - INFO - Received callback for node 69c39a1cbbc022e792643edd INFO: 172.20.0.1:33116 - "POST /node/69c39a1cbbc022e792643edd HTTP/1.0" 202 Accepted 2026-03-27 08:17:52,170 - lava_callback - INFO - Starting async job submit for node 69c39a1cbbc022e792643edd 2026-03-27 08:17:52,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:17:52,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39a1cbbc022e792643edd HTTP/1.1" 200 None 2026-03-27 08:17:52,229 - lava_callback - DEBUG - Processing node 69c39a1cbbc022e792643edd: baseline-arm64-baylibre 2026-03-27 08:17:52,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39a1cbbc022e792643edd 2026-03-27 08:17:52,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:17:52,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39a1cbbc022e792643edd HTTP/1.1" 200 None 2026-03-27 08:17:52,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:17:52,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 08:17:52,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:17:52,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39a1cbbc022e792643edd HTTP/1.1" 200 None 2026-03-27 08:17:52,291 - lava_callback - ERROR - Error processing callback for node 69c39a1cbbc022e792643edd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42210 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42238 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:19:39,532 - lava_callback - INFO - Received callback for node 69c39a8abbc022e792643ee5 INFO: 172.20.0.1:42248 - "POST /node/69c39a8abbc022e792643ee5 HTTP/1.0" 202 Accepted 2026-03-27 08:19:39,538 - lava_callback - INFO - Starting async job submit for node 69c39a8abbc022e792643ee5 2026-03-27 08:19:39,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:19:39,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39a8abbc022e792643ee5 HTTP/1.1" 200 None 2026-03-27 08:19:39,553 - lava_callback - DEBUG - Processing node 69c39a8abbc022e792643ee5: baseline-arm64-clabbe 2026-03-27 08:19:39,553 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39a8abbc022e792643ee5 2026-03-27 08:19:39,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:19:39,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39a8abbc022e792643ee5 HTTP/1.1" 200 None 2026-03-27 08:19:39,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:19:39,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 08:19:39,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:19:39,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39a8abbc022e792643ee5 HTTP/1.1" 200 None 2026-03-27 08:19:39,609 - lava_callback - ERROR - Error processing callback for node 69c39a8abbc022e792643ee5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40094 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:20:59,572 - lava_callback - INFO - Received callback for node 69c39ad8bbc022e792643ef4 INFO: 172.20.0.1:40110 - "POST /node/69c39ad8bbc022e792643ef4 HTTP/1.0" 202 Accepted 2026-03-27 08:20:59,578 - lava_callback - INFO - Starting async job submit for node 69c39ad8bbc022e792643ef4 2026-03-27 08:20:59,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:20:59,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39ad8bbc022e792643ef4 HTTP/1.1" 200 None 2026-03-27 08:20:59,594 - lava_callback - DEBUG - Processing node 69c39ad8bbc022e792643ef4: baseline-arm64-clabbe 2026-03-27 08:20:59,594 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39ad8bbc022e792643ef4 2026-03-27 08:20:59,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:20:59,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39ad8bbc022e792643ef4 HTTP/1.1" 200 None 2026-03-27 08:20:59,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:20:59,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 08:20:59,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:20:59,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39ad8bbc022e792643ef4 HTTP/1.1" 200 None 2026-03-27 08:20:59,664 - lava_callback - ERROR - Error processing callback for node 69c39ad8bbc022e792643ef4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44358 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:22:19,632 - lava_callback - INFO - Received callback for node 69c39b25bbc022e792643eff INFO: 172.20.0.1:44362 - "POST /node/69c39b25bbc022e792643eff HTTP/1.0" 202 Accepted 2026-03-27 08:22:19,638 - lava_callback - INFO - Starting async job submit for node 69c39b25bbc022e792643eff 2026-03-27 08:22:19,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:22:19,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39b25bbc022e792643eff HTTP/1.1" 200 None 2026-03-27 08:22:19,654 - lava_callback - DEBUG - Processing node 69c39b25bbc022e792643eff: baseline-arm64-clabbe 2026-03-27 08:22:19,654 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39b25bbc022e792643eff 2026-03-27 08:22:19,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:22:19,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39b25bbc022e792643eff HTTP/1.1" 200 None 2026-03-27 08:22:19,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:22:19,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 08:22:19,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:22:19,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39b25bbc022e792643eff HTTP/1.1" 200 None 2026-03-27 08:22:19,717 - lava_callback - ERROR - Error processing callback for node 69c39b25bbc022e792643eff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47934 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:23:39,664 - lava_callback - INFO - Received callback for node 69c39b7abbc022e792643f0d INFO: 172.20.0.1:47940 - "POST /node/69c39b7abbc022e792643f0d HTTP/1.0" 202 Accepted 2026-03-27 08:23:39,669 - lava_callback - INFO - Starting async job submit for node 69c39b7abbc022e792643f0d 2026-03-27 08:23:39,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:23:39,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39b7abbc022e792643f0d HTTP/1.1" 200 None 2026-03-27 08:23:39,685 - lava_callback - DEBUG - Processing node 69c39b7abbc022e792643f0d: baseline-arm64-clabbe 2026-03-27 08:23:39,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39b7abbc022e792643f0d 2026-03-27 08:23:39,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:23:39,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39b7abbc022e792643f0d HTTP/1.1" 200 None 2026-03-27 08:23:39,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:23:39,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 08:23:39,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:23:39,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39b7abbc022e792643f0d HTTP/1.1" 200 None 2026-03-27 08:23:39,738 - lava_callback - ERROR - Error processing callback for node 69c39b7abbc022e792643f0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39324 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:24:59,752 - lava_callback - INFO - Received callback for node 69c39bc7bbc022e792643f1f INFO: 172.20.0.1:47098 - "POST /node/69c39bc7bbc022e792643f1f HTTP/1.0" 202 Accepted 2026-03-27 08:24:59,757 - lava_callback - INFO - Starting async job submit for node 69c39bc7bbc022e792643f1f 2026-03-27 08:24:59,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:24:59,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39bc7bbc022e792643f1f HTTP/1.1" 200 None 2026-03-27 08:24:59,773 - lava_callback - DEBUG - Processing node 69c39bc7bbc022e792643f1f: baseline-arm64-clabbe 2026-03-27 08:24:59,773 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39bc7bbc022e792643f1f 2026-03-27 08:24:59,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:24:59,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39bc7bbc022e792643f1f HTTP/1.1" 200 None 2026-03-27 08:24:59,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:24:59,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 08:24:59,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:24:59,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39bc7bbc022e792643f1f HTTP/1.1" 200 None 2026-03-27 08:24:59,876 - lava_callback - ERROR - Error processing callback for node 69c39bc7bbc022e792643f1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46052 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:25:31,405 - lava_callback - INFO - Received callback for node 69c5b895abf66a8f4d9bccec INFO: 172.20.0.1:45834 - "POST /node/69c5b895abf66a8f4d9bccec HTTP/1.0" 202 Accepted 2026-03-27 08:25:31,411 - lava_callback - INFO - Starting async job submit for node 69c5b895abf66a8f4d9bccec 2026-03-27 08:25:31,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:31,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b895abf66a8f4d9bccec HTTP/1.1" 200 None 2026-03-27 08:25:31,451 - lava_callback - DEBUG - Processing node 69c5b895abf66a8f4d9bccec: baseline-arm64-broonie 2026-03-27 08:25:31,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:25:31,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:25:31,971 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b895abf66a8f4d9bccec/log.txt.gz 2026-03-27 08:25:32,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:25:32,360 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:25:32,362 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b895abf66a8f4d9bccec/lava-logs.yaml 2026-03-27 08:25:32,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:25:32,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:25:32,512 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5b895abf66a8f4d9bccec/lava_callback.json.gz 2026-03-27 08:25:32,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:32,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b895abf66a8f4d9bccec HTTP/1.1" 200 None 2026-03-27 08:25:32,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:32,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 08:25:32,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:32,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5b895abf66a8f4d9bccec HTTP/1.1" 200 None 2026-03-27 08:25:32,596 - lava_callback - INFO - Completed processing callback for node 69c5b895abf66a8f4d9bccec INFO: 172.20.0.1:45850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53922 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:25:51,356 - lava_callback - INFO - Received callback for node 69c5d59e2cb62322ad966304 INFO: 172.20.0.1:59984 - "POST /node/69c5d59e2cb62322ad966304 HTTP/1.0" 202 Accepted 2026-03-27 08:25:51,362 - lava_callback - INFO - Starting async job submit for node 69c5d59e2cb62322ad966304 2026-03-27 08:25:51,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:51,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d59e2cb62322ad966304 HTTP/1.1" 200 None 2026-03-27 08:25:51,400 - lava_callback - DEBUG - Processing node 69c5d59e2cb62322ad966304: baseline-arm64-broonie 2026-03-27 08:25:51,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:25:51,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:25:51,760 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d59e2cb62322ad966304/log.txt.gz 2026-03-27 08:25:51,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:25:52,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:25:52,216 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d59e2cb62322ad966304/lava-logs.yaml 2026-03-27 08:25:52,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:25:52,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:25:52,390 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c5d59e2cb62322ad966304/lava_callback.json.gz 2026-03-27 08:25:52,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:52,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5d59e2cb62322ad966304 HTTP/1.1" 200 None 2026-03-27 08:25:52,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:52,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 08:25:52,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:52,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5d59e2cb62322ad966304 HTTP/1.1" 200 None 2026-03-27 08:25:52,478 - lava_callback - INFO - Completed processing callback for node 69c5d59e2cb62322ad966304 INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:25:59,803 - lava_callback - INFO - Received callback for node 69c39c14bbc022e792643f27 INFO: 172.20.0.1:45686 - "POST /node/69c39c14bbc022e792643f27 HTTP/1.0" 202 Accepted 2026-03-27 08:25:59,808 - lava_callback - INFO - Starting async job submit for node 69c39c14bbc022e792643f27 2026-03-27 08:25:59,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:59,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39c14bbc022e792643f27 HTTP/1.1" 200 None 2026-03-27 08:25:59,825 - lava_callback - DEBUG - Processing node 69c39c14bbc022e792643f27: baseline-arm64-clabbe 2026-03-27 08:25:59,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39c14bbc022e792643f27 2026-03-27 08:25:59,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:59,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39c14bbc022e792643f27 HTTP/1.1" 200 None 2026-03-27 08:25:59,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:59,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 08:25:59,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:25:59,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39c14bbc022e792643f27 HTTP/1.1" 200 None 2026-03-27 08:25:59,885 - lava_callback - ERROR - Error processing callback for node 69c39c14bbc022e792643f27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:26:02,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:26:02,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53688 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:28:31,754 - lava_callback - INFO - Received callback for node 69c611782cb62322ad969535 INFO: 172.20.0.1:41218 - "POST /node/69c611782cb62322ad969535 HTTP/1.0" 202 Accepted 2026-03-27 08:28:31,760 - lava_callback - INFO - Starting async job submit for node 69c611782cb62322ad969535 2026-03-27 08:28:31,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:28:31,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c611782cb62322ad969535 HTTP/1.1" 200 None 2026-03-27 08:28:31,810 - lava_callback - DEBUG - Processing node 69c611782cb62322ad969535: baseline-arm64-broonie 2026-03-27 08:28:32,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:28:32,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:28:32,364 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c611782cb62322ad969535/log.txt.gz 2026-03-27 08:28:32,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:28:32,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:28:32,718 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c611782cb62322ad969535/lava-logs.yaml 2026-03-27 08:28:32,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:28:32,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:28:32,902 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c611782cb62322ad969535/lava_callback.json.gz 2026-03-27 08:28:32,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:28:32,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c611782cb62322ad969535 HTTP/1.1" 200 None 2026-03-27 08:28:32,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:28:33,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 08:28:33,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:28:33,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c611782cb62322ad969535 HTTP/1.1" 200 None 2026-03-27 08:28:33,161 - lava_callback - INFO - Completed processing callback for node 69c611782cb62322ad969535 2026-03-27 08:28:33,566 - lava_callback - INFO - Received callback for node 69c62e072cb62322ad969b13 INFO: 172.20.0.1:41234 - "POST /node/69c62e072cb62322ad969b13 HTTP/1.0" 202 Accepted 2026-03-27 08:28:33,572 - lava_callback - INFO - Starting async job submit for node 69c62e072cb62322ad969b13 2026-03-27 08:28:33,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:28:33,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62e072cb62322ad969b13 HTTP/1.1" 200 None 2026-03-27 08:28:33,619 - lava_callback - DEBUG - Processing node 69c62e072cb62322ad969b13: baseline-arm64-broonie 2026-03-27 08:28:33,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:28:33,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:28:33,981 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c62e072cb62322ad969b13/log.txt.gz 2026-03-27 08:28:34,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:28:34,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:28:34,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c62e072cb62322ad969b13/lava-logs.yaml 2026-03-27 08:28:34,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:28:34,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:28:34,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c62e072cb62322ad969b13/lava_callback.json.gz 2026-03-27 08:28:34,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:28:34,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62e072cb62322ad969b13 HTTP/1.1" 200 None 2026-03-27 08:28:34,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:28:34,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 08:28:34,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:28:34,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62e072cb62322ad969b13 HTTP/1.1" 200 None 2026-03-27 08:28:34,561 - lava_callback - INFO - Completed processing callback for node 69c62e072cb62322ad969b13 INFO: 172.20.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43878 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:29:02,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:29:02,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49718 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:33:11,973 - lava_callback - INFO - Received callback for node 69c64042b98d4d3c02068458 INFO: 172.20.0.1:37906 - "POST /node/69c64042b98d4d3c02068458 HTTP/1.0" 202 Accepted 2026-03-27 08:33:11,979 - lava_callback - INFO - Starting async job submit for node 69c64042b98d4d3c02068458 2026-03-27 08:33:12,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:12,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64042b98d4d3c02068458 HTTP/1.1" 200 None 2026-03-27 08:33:12,019 - lava_callback - DEBUG - Processing node 69c64042b98d4d3c02068458: baseline-arm 2026-03-27 08:33:12,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:33:12,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:33:12,446 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c64042b98d4d3c02068458/log.txt.gz 2026-03-27 08:33:12,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:33:12,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:33:12,828 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c64042b98d4d3c02068458/lava-logs.yaml 2026-03-27 08:33:12,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:33:12,998 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:33:13,013 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c64042b98d4d3c02068458/lava_callback.json.gz 2026-03-27 08:33:13,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:13,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64042b98d4d3c02068458 HTTP/1.1" 200 None 2026-03-27 08:33:13,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:13,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce8b98d4d3c02067dcb HTTP/1.1" 200 None 2026-03-27 08:33:13,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:13,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64042b98d4d3c02068458 HTTP/1.1" 200 None 2026-03-27 08:33:13,127 - lava_callback - INFO - Completed processing callback for node 69c64042b98d4d3c02068458 INFO: 172.20.0.1:37920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:33:32,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:32,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55476 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:33:45,320 - lava_callback - INFO - Received callback for node 69c640cab98d4d3c02068487 INFO: 172.20.0.1:37658 - "POST /node/69c640cab98d4d3c02068487 HTTP/1.0" 202 Accepted 2026-03-27 08:33:45,327 - lava_callback - INFO - Starting async job submit for node 69c640cab98d4d3c02068487 2026-03-27 08:33:45,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:45,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c640cab98d4d3c02068487 HTTP/1.1" 200 None 2026-03-27 08:33:45,362 - lava_callback - DEBUG - Processing node 69c640cab98d4d3c02068487: baseline-x86 2026-03-27 08:33:45,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:33:45,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:33:45,726 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c640cab98d4d3c02068487/log.txt.gz 2026-03-27 08:33:45,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:33:45,979 - lava_callback - INFO - Received callback for node 69c640c7b98d4d3c0206847a 2026-03-27 08:33:45,989 - lava_callback - INFO - Starting async job submit for node 69c640c7b98d4d3c0206847a INFO: 172.20.0.1:37666 - "POST /node/69c640c7b98d4d3c0206847a HTTP/1.0" 202 Accepted 2026-03-27 08:33:46,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:46,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c640c7b98d4d3c0206847a HTTP/1.1" 200 None 2026-03-27 08:33:46,034 - lava_callback - DEBUG - Processing node 69c640c7b98d4d3c0206847a: baseline-x86 2026-03-27 08:33:46,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:33:46,072 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c640cab98d4d3c02068487/lava-logs.yaml 2026-03-27 08:33:46,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:33:46,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:33:46,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:33:46,366 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c640c7b98d4d3c0206847a/log.txt.gz 2026-03-27 08:33:46,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:33:46,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:33:46,480 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c640cab98d4d3c02068487/lava_callback.json.gz 2026-03-27 08:33:46,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:37676 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:33:46,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c640cab98d4d3c02068487 HTTP/1.1" 200 None 2026-03-27 08:33:46,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:46,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63d02b98d4d3c02067dea HTTP/1.1" 200 None 2026-03-27 08:33:46,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:46,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c640cab98d4d3c02068487 HTTP/1.1" 200 None 2026-03-27 08:33:46,601 - lava_callback - INFO - Completed processing callback for node 69c640cab98d4d3c02068487 2026-03-27 08:33:46,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:33:46,749 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c640c7b98d4d3c0206847a/lava-logs.yaml 2026-03-27 08:33:46,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:33:46,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:33:46,909 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c640c7b98d4d3c0206847a/lava_callback.json.gz 2026-03-27 08:33:46,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:46,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c640c7b98d4d3c0206847a HTTP/1.1" 200 None 2026-03-27 08:33:46,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:46,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63d02b98d4d3c02067dea HTTP/1.1" 200 None 2026-03-27 08:33:46,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:33:46,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c640c7b98d4d3c0206847a HTTP/1.1" 200 None 2026-03-27 08:33:46,995 - lava_callback - INFO - Completed processing callback for node 69c640c7b98d4d3c0206847a INFO: 172.20.0.1:43604 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:34:02,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:34:02,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54094 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:35:06,536 - lava_callback - INFO - Received callback for node 69c640c9b98d4d3c02068484 INFO: 172.20.0.1:54110 - "POST /node/69c640c9b98d4d3c02068484 HTTP/1.0" 202 Accepted 2026-03-27 08:35:06,545 - lava_callback - INFO - Starting async job submit for node 69c640c9b98d4d3c02068484 2026-03-27 08:35:06,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:35:06,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c640c9b98d4d3c02068484 HTTP/1.1" 200 None 2026-03-27 08:35:06,620 - lava_callback - DEBUG - Processing node 69c640c9b98d4d3c02068484: baseline-x86 2026-03-27 08:35:07,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:35:07,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:35:07,498 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c640c9b98d4d3c02068484/log.txt.gz 2026-03-27 08:35:07,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:35:08,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:35:08,373 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c640c9b98d4d3c02068484/lava-logs.yaml 2026-03-27 08:35:08,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:35:08,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:35:08,711 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c640c9b98d4d3c02068484/lava_callback.json.gz 2026-03-27 08:35:08,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:35:08,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c640c9b98d4d3c02068484 HTTP/1.1" 200 None 2026-03-27 08:35:08,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:35:08,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63d02b98d4d3c02067dea HTTP/1.1" 200 None 2026-03-27 08:35:08,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:35:08,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c640c9b98d4d3c02068484 HTTP/1.1" 200 None 2026-03-27 08:35:08,856 - lava_callback - INFO - Completed processing callback for node 69c640c9b98d4d3c02068484 2026-03-27 08:35:10,769 - lava_callback - INFO - Received callback for node 69c640c8b98d4d3c0206847d INFO: 172.20.0.1:60898 - "POST /node/69c640c8b98d4d3c0206847d HTTP/1.0" 202 Accepted 2026-03-27 08:35:10,778 - lava_callback - INFO - Starting async job submit for node 69c640c8b98d4d3c0206847d 2026-03-27 08:35:10,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:35:10,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c640c8b98d4d3c0206847d HTTP/1.1" 200 None 2026-03-27 08:35:10,853 - lava_callback - DEBUG - Processing node 69c640c8b98d4d3c0206847d: baseline-x86 2026-03-27 08:35:11,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:35:11,838 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:35:11,840 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c640c8b98d4d3c0206847d/log.txt.gz 2026-03-27 08:35:12,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:35:12,749 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:35:12,752 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c640c8b98d4d3c0206847d/lava-logs.yaml 2026-03-27 08:35:12,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:35:13,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:35:13,433 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c640c8b98d4d3c0206847d/lava_callback.json.gz 2026-03-27 08:35:13,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:35:13,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c640c8b98d4d3c0206847d HTTP/1.1" 200 None 2026-03-27 08:35:13,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:35:13,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63d02b98d4d3c02067dea HTTP/1.1" 200 None 2026-03-27 08:35:13,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:35:13,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c640c8b98d4d3c0206847d HTTP/1.1" 200 None 2026-03-27 08:35:13,573 - lava_callback - INFO - Completed processing callback for node 69c640c8b98d4d3c0206847d INFO: 172.20.0.1:60902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38666 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:35:32,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:35:32,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55190 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:36:33,305 - lava_callback - INFO - Received callback for node 69c39e7fbbc022e79264451b INFO: 172.20.0.1:53728 - "POST /node/69c39e7fbbc022e79264451b HTTP/1.0" 202 Accepted 2026-03-27 08:36:33,314 - lava_callback - INFO - Starting async job submit for node 69c39e7fbbc022e79264451b 2026-03-27 08:36:33,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:33,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e7fbbc022e79264451b HTTP/1.1" 200 None 2026-03-27 08:36:33,352 - lava_callback - DEBUG - Processing node 69c39e7fbbc022e79264451b: baseline-arm-baylibre 2026-03-27 08:36:33,352 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e7fbbc022e79264451b 2026-03-27 08:36:33,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:33,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e7fbbc022e79264451b HTTP/1.1" 200 None 2026-03-27 08:36:33,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:33,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:36:33,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:33,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e7fbbc022e79264451b HTTP/1.1" 200 None 2026-03-27 08:36:33,442 - lava_callback - ERROR - Error processing callback for node 69c39e7fbbc022e79264451b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:36:33,997 - lava_callback - INFO - Received callback for node 69c39e81bbc022e79264451e INFO: 172.20.0.1:53734 - "POST /node/69c39e81bbc022e79264451e HTTP/1.0" 202 Accepted 2026-03-27 08:36:34,006 - lava_callback - INFO - Starting async job submit for node 69c39e81bbc022e79264451e 2026-03-27 08:36:34,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:34,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e81bbc022e79264451e HTTP/1.1" 200 None 2026-03-27 08:36:34,030 - lava_callback - DEBUG - Processing node 69c39e81bbc022e79264451e: baseline-arm-baylibre 2026-03-27 08:36:34,030 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e81bbc022e79264451e 2026-03-27 08:36:34,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:34,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e81bbc022e79264451e HTTP/1.1" 200 None 2026-03-27 08:36:34,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:34,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:36:34,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:34,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e81bbc022e79264451e HTTP/1.1" 200 None 2026-03-27 08:36:34,133 - lava_callback - ERROR - Error processing callback for node 69c39e81bbc022e79264451e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:36:34,676 - lava_callback - INFO - Received callback for node 69c39e82bbc022e792644521 INFO: 172.20.0.1:53740 - "POST /node/69c39e82bbc022e792644521 HTTP/1.0" 202 Accepted 2026-03-27 08:36:34,681 - lava_callback - INFO - Starting async job submit for node 69c39e82bbc022e792644521 2026-03-27 08:36:34,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:34,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e82bbc022e792644521 HTTP/1.1" 200 None 2026-03-27 08:36:34,698 - lava_callback - DEBUG - Processing node 69c39e82bbc022e792644521: baseline-arm-baylibre 2026-03-27 08:36:34,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e82bbc022e792644521 2026-03-27 08:36:34,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:34,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e82bbc022e792644521 HTTP/1.1" 200 None 2026-03-27 08:36:34,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:34,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:36:34,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:34,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e82bbc022e792644521 HTTP/1.1" 200 None 2026-03-27 08:36:34,774 - lava_callback - ERROR - Error processing callback for node 69c39e82bbc022e792644521: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:36:35,331 - lava_callback - INFO - Received callback for node 69c39e83bbc022e792644524 INFO: 172.20.0.1:53752 - "POST /node/69c39e83bbc022e792644524 HTTP/1.0" 202 Accepted 2026-03-27 08:36:35,337 - lava_callback - INFO - Starting async job submit for node 69c39e83bbc022e792644524 2026-03-27 08:36:35,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:35,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e83bbc022e792644524 HTTP/1.1" 200 None 2026-03-27 08:36:35,360 - lava_callback - DEBUG - Processing node 69c39e83bbc022e792644524: baseline-arm-baylibre 2026-03-27 08:36:35,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e83bbc022e792644524 2026-03-27 08:36:35,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:35,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e83bbc022e792644524 HTTP/1.1" 200 None 2026-03-27 08:36:35,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:35,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:36:35,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:35,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e83bbc022e792644524 HTTP/1.1" 200 None 2026-03-27 08:36:35,416 - lava_callback - ERROR - Error processing callback for node 69c39e83bbc022e792644524: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:36:35,992 - lava_callback - INFO - Received callback for node 69c39e86bbc022e79264452f INFO: 172.20.0.1:53760 - "POST /node/69c39e86bbc022e79264452f HTTP/1.0" 202 Accepted 2026-03-27 08:36:36,001 - lava_callback - INFO - Starting async job submit for node 69c39e86bbc022e79264452f 2026-03-27 08:36:36,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:36,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e86bbc022e79264452f HTTP/1.1" 200 None 2026-03-27 08:36:36,019 - lava_callback - DEBUG - Processing node 69c39e86bbc022e79264452f: baseline-arm-baylibre 2026-03-27 08:36:36,020 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e86bbc022e79264452f 2026-03-27 08:36:36,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:36,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e86bbc022e79264452f HTTP/1.1" 200 None 2026-03-27 08:36:36,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:36,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:36:36,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:36,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e86bbc022e79264452f HTTP/1.1" 200 None 2026-03-27 08:36:36,089 - lava_callback - ERROR - Error processing callback for node 69c39e86bbc022e79264452f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53770 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:36:36,674 - lava_callback - INFO - Received callback for node 69c39e87bbc022e792644532 INFO: 172.20.0.1:53776 - "POST /node/69c39e87bbc022e792644532 HTTP/1.0" 202 Accepted 2026-03-27 08:36:36,681 - lava_callback - INFO - Starting async job submit for node 69c39e87bbc022e792644532 2026-03-27 08:36:36,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:36,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e87bbc022e792644532 HTTP/1.1" 200 None 2026-03-27 08:36:36,698 - lava_callback - DEBUG - Processing node 69c39e87bbc022e792644532: baseline-arm-baylibre 2026-03-27 08:36:36,698 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e87bbc022e792644532 2026-03-27 08:36:36,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:36,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e87bbc022e792644532 HTTP/1.1" 200 None 2026-03-27 08:36:36,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:36,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:36:36,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:36,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e87bbc022e792644532 HTTP/1.1" 200 None 2026-03-27 08:36:36,751 - lava_callback - ERROR - Error processing callback for node 69c39e87bbc022e792644532: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:36:40,299 - lava_callback - INFO - Received callback for node 69c39e8fbbc022e79264454d INFO: 172.20.0.1:43974 - "POST /node/69c39e8fbbc022e79264454d HTTP/1.0" 202 Accepted 2026-03-27 08:36:40,308 - lava_callback - INFO - Starting async job submit for node 69c39e8fbbc022e79264454d 2026-03-27 08:36:40,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:40,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e8fbbc022e79264454d HTTP/1.1" 200 None 2026-03-27 08:36:40,330 - lava_callback - DEBUG - Processing node 69c39e8fbbc022e79264454d: baseline-arm-clabbe 2026-03-27 08:36:40,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e8fbbc022e79264454d 2026-03-27 08:36:40,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:40,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e8fbbc022e79264454d HTTP/1.1" 200 None 2026-03-27 08:36:40,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:40,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:36:40,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:40,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e8fbbc022e79264454d HTTP/1.1" 200 None 2026-03-27 08:36:40,400 - lava_callback - ERROR - Error processing callback for node 69c39e8fbbc022e79264454d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:36:40,981 - lava_callback - INFO - Received callback for node 69c39e94bbc022e792644579 INFO: 172.20.0.1:43986 - "POST /node/69c39e94bbc022e792644579 HTTP/1.0" 202 Accepted 2026-03-27 08:36:40,987 - lava_callback - INFO - Starting async job submit for node 69c39e94bbc022e792644579 2026-03-27 08:36:40,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:41,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e94bbc022e792644579 HTTP/1.1" 200 None 2026-03-27 08:36:41,010 - lava_callback - DEBUG - Processing node 69c39e94bbc022e792644579: baseline-arm-clabbe 2026-03-27 08:36:41,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e94bbc022e792644579 2026-03-27 08:36:41,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:41,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e94bbc022e792644579 HTTP/1.1" 200 None 2026-03-27 08:36:41,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:41,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:36:41,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:41,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e94bbc022e792644579 HTTP/1.1" 200 None 2026-03-27 08:36:41,063 - lava_callback - ERROR - Error processing callback for node 69c39e94bbc022e792644579: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:36:41,653 - lava_callback - INFO - Received callback for node 69c39e95bbc022e79264457c INFO: 172.20.0.1:43998 - "POST /node/69c39e95bbc022e79264457c HTTP/1.0" 202 Accepted 2026-03-27 08:36:41,661 - lava_callback - INFO - Starting async job submit for node 69c39e95bbc022e79264457c 2026-03-27 08:36:41,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:41,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e95bbc022e79264457c HTTP/1.1" 200 None 2026-03-27 08:36:41,684 - lava_callback - DEBUG - Processing node 69c39e95bbc022e79264457c: baseline-arm-clabbe 2026-03-27 08:36:41,684 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e95bbc022e79264457c 2026-03-27 08:36:41,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:41,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e95bbc022e79264457c HTTP/1.1" 200 None 2026-03-27 08:36:41,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:41,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:36:41,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:41,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e95bbc022e79264457c HTTP/1.1" 200 None 2026-03-27 08:36:41,741 - lava_callback - ERROR - Error processing callback for node 69c39e95bbc022e79264457c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:36:42,349 - lava_callback - INFO - Received callback for node 69c39e96bbc022e79264457f INFO: 172.20.0.1:44012 - "POST /node/69c39e96bbc022e79264457f HTTP/1.0" 202 Accepted 2026-03-27 08:36:42,354 - lava_callback - INFO - Starting async job submit for node 69c39e96bbc022e79264457f 2026-03-27 08:36:42,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:42,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e96bbc022e79264457f HTTP/1.1" 200 None 2026-03-27 08:36:42,371 - lava_callback - DEBUG - Processing node 69c39e96bbc022e79264457f: baseline-arm-clabbe 2026-03-27 08:36:42,371 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e96bbc022e79264457f 2026-03-27 08:36:42,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:42,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e96bbc022e79264457f HTTP/1.1" 200 None 2026-03-27 08:36:42,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:42,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:36:42,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:42,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e96bbc022e79264457f HTTP/1.1" 200 None 2026-03-27 08:36:42,435 - lava_callback - ERROR - Error processing callback for node 69c39e96bbc022e79264457f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49616 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:36:58,394 - lava_callback - INFO - Received callback for node 69c6419db98d4d3c020684fa INFO: 172.20.0.1:49620 - "POST /node/69c6419db98d4d3c020684fa HTTP/1.0" 202 Accepted 2026-03-27 08:36:58,400 - lava_callback - INFO - Starting async job submit for node 69c6419db98d4d3c020684fa 2026-03-27 08:36:58,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:58,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6419db98d4d3c020684fa HTTP/1.1" 200 None 2026-03-27 08:36:58,418 - lava_callback - DEBUG - Processing node 69c6419db98d4d3c020684fa: baseline-arm 2026-03-27 08:36:58,424 - lava_callback - WARNING - Failed to upload log 2026-03-27 08:36:58,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:36:58,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:36:58,541 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419db98d4d3c020684fa/lava-logs.yaml 2026-03-27 08:36:58,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:36:58,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:36:58,657 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419db98d4d3c020684fa/lava_callback.json.gz 2026-03-27 08:36:58,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:58,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6419db98d4d3c020684fa HTTP/1.1" 200 None 2026-03-27 08:36:58,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:58,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:36:58,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:36:58,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6419db98d4d3c020684fa HTTP/1.1" 200 None 2026-03-27 08:36:58,736 - lava_callback - INFO - Completed processing callback for node 69c6419db98d4d3c020684fa 2026-03-27 08:37:00,225 - lava_callback - INFO - Received callback for node 69c39e99bbc022e79264458a INFO: 172.20.0.1:44318 - "POST /node/69c39e99bbc022e79264458a HTTP/1.0" 202 Accepted 2026-03-27 08:37:00,231 - lava_callback - INFO - Starting async job submit for node 69c39e99bbc022e79264458a 2026-03-27 08:37:00,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:00,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e99bbc022e79264458a HTTP/1.1" 200 None 2026-03-27 08:37:00,259 - lava_callback - DEBUG - Processing node 69c39e99bbc022e79264458a: baseline-arm-clabbe 2026-03-27 08:37:00,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c39e99bbc022e79264458a 2026-03-27 08:37:00,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:00,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c39e99bbc022e79264458a HTTP/1.1" 200 None 2026-03-27 08:37:00,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:00,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 08:37:00,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:00,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c39e99bbc022e79264458a HTTP/1.1" 200 None 2026-03-27 08:37:00,311 - lava_callback - ERROR - Error processing callback for node 69c39e99bbc022e79264458a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:37:02,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:02,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47486 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:37:29,883 - lava_callback - INFO - Received callback for node 69c6419db98d4d3c020684fe INFO: 172.20.0.1:36154 - "POST /node/69c6419db98d4d3c020684fe HTTP/1.0" 202 Accepted 2026-03-27 08:37:29,929 - lava_callback - INFO - Starting async job submit for node 69c6419db98d4d3c020684fe 2026-03-27 08:37:29,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:29,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6419db98d4d3c020684fe HTTP/1.1" 200 None 2026-03-27 08:37:29,963 - lava_callback - DEBUG - Processing node 69c6419db98d4d3c020684fe: baseline-arm 2026-03-27 08:37:30,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:37:30,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:37:30,223 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419db98d4d3c020684fe/log.txt.gz 2026-03-27 08:37:30,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:37:30,458 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:37:30,460 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419db98d4d3c020684fe/lava-logs.yaml 2026-03-27 08:37:30,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:37:30,592 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:37:30,594 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419db98d4d3c020684fe/lava_callback.json.gz 2026-03-27 08:37:30,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:30,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6419db98d4d3c020684fe HTTP/1.1" 200 None 2026-03-27 08:37:30,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:30,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:37:30,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:30,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6419db98d4d3c020684fe HTTP/1.1" 200 None 2026-03-27 08:37:30,686 - lava_callback - INFO - Completed processing callback for node 69c6419db98d4d3c020684fe 2026-03-27 08:37:31,525 - lava_callback - INFO - Received callback for node 69c641a0b98d4d3c02068507 INFO: 172.20.0.1:36158 - "POST /node/69c641a0b98d4d3c02068507 HTTP/1.0" 202 Accepted 2026-03-27 08:37:31,578 - lava_callback - INFO - Starting async job submit for node 69c641a0b98d4d3c02068507 2026-03-27 08:37:31,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:31,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641a0b98d4d3c02068507 HTTP/1.1" 200 None 2026-03-27 08:37:31,613 - lava_callback - DEBUG - Processing node 69c641a0b98d4d3c02068507: baseline-arm 2026-03-27 08:37:31,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:37:31,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:37:31,875 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c641a0b98d4d3c02068507/log.txt.gz 2026-03-27 08:37:31,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:37:32,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:32,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:37:32,178 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c641a0b98d4d3c02068507/lava-logs.yaml 2026-03-27 08:37:32,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:37:32,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 08:37:32,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:37:32,304 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c641a0b98d4d3c02068507/lava_callback.json.gz 2026-03-27 08:37:32,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:32,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641a0b98d4d3c02068507 HTTP/1.1" 200 None 2026-03-27 08:37:32,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:32,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:37:32,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:37:32,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641a0b98d4d3c02068507 HTTP/1.1" 200 None 2026-03-27 08:37:32,393 - lava_callback - INFO - Completed processing callback for node 69c641a0b98d4d3c02068507 INFO: 172.20.0.1:36168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55408 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:38:02,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:02,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 08:38:04,620 - lava_callback - INFO - Received callback for node 69c6419cb98d4d3c020684f6 INFO: 172.20.0.1:36520 - "POST /node/69c6419cb98d4d3c020684f6 HTTP/1.0" 202 Accepted 2026-03-27 08:38:04,626 - lava_callback - INFO - Starting async job submit for node 69c6419cb98d4d3c020684f6 2026-03-27 08:38:04,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:04,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6419cb98d4d3c020684f6 HTTP/1.1" 200 None 2026-03-27 08:38:04,662 - lava_callback - DEBUG - Processing node 69c6419cb98d4d3c020684f6: baseline-arm 2026-03-27 08:38:04,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:04,967 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:04,969 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419cb98d4d3c020684f6/log.txt.gz 2026-03-27 08:38:05,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:05,289 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:05,291 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419cb98d4d3c020684f6/lava-logs.yaml 2026-03-27 08:38:05,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:05,432 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:05,433 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419cb98d4d3c020684f6/lava_callback.json.gz 2026-03-27 08:38:05,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:05,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6419cb98d4d3c020684f6 HTTP/1.1" 200 None 2026-03-27 08:38:05,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:05,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:38:05,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:05,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6419cb98d4d3c020684f6 HTTP/1.1" 200 None 2026-03-27 08:38:05,517 - lava_callback - INFO - Completed processing callback for node 69c6419cb98d4d3c020684f6 2026-03-27 08:38:05,666 - lava_callback - INFO - Received callback for node 69c6419fb98d4d3c02068504 INFO: 172.20.0.1:36530 - "POST /node/69c6419fb98d4d3c02068504 HTTP/1.0" 202 Accepted 2026-03-27 08:38:05,676 - lava_callback - INFO - Starting async job submit for node 69c6419fb98d4d3c02068504 2026-03-27 08:38:05,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:05,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6419fb98d4d3c02068504 HTTP/1.1" 200 None 2026-03-27 08:38:05,737 - lava_callback - DEBUG - Processing node 69c6419fb98d4d3c02068504: baseline-arm 2026-03-27 08:38:05,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:06,098 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:06,100 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419fb98d4d3c02068504/log.txt.gz 2026-03-27 08:38:06,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:06,417 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:06,419 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419fb98d4d3c02068504/lava-logs.yaml 2026-03-27 08:38:06,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:36544 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:38:06,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:06,560 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419fb98d4d3c02068504/lava_callback.json.gz 2026-03-27 08:38:06,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:06,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6419fb98d4d3c02068504 HTTP/1.1" 200 None 2026-03-27 08:38:06,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:06,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:38:06,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:06,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6419fb98d4d3c02068504 HTTP/1.1" 200 None 2026-03-27 08:38:06,639 - lava_callback - INFO - Completed processing callback for node 69c6419fb98d4d3c02068504 INFO: 172.20.0.1:36898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58962 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:38:22,910 - lava_callback - INFO - Received callback for node 69c641c0b98d4d3c02068594 INFO: 172.20.0.1:58964 - "POST /node/69c641c0b98d4d3c02068594 HTTP/1.0" 202 Accepted 2026-03-27 08:38:22,961 - lava_callback - INFO - Starting async job submit for node 69c641c0b98d4d3c02068594 2026-03-27 08:38:22,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:22,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641c0b98d4d3c02068594 HTTP/1.1" 200 None 2026-03-27 08:38:22,996 - lava_callback - DEBUG - Processing node 69c641c0b98d4d3c02068594: baseline-arm-pengutronix 2026-03-27 08:38:23,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:23,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:23,230 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641c0b98d4d3c02068594/log.txt.gz 2026-03-27 08:38:23,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:23,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:23,455 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641c0b98d4d3c02068594/lava-logs.yaml 2026-03-27 08:38:23,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:23,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:23,595 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641c0b98d4d3c02068594/lava_callback.json.gz 2026-03-27 08:38:23,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:23,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641c0b98d4d3c02068594 HTTP/1.1" 200 None 2026-03-27 08:38:23,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:23,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:38:23,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:23,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641c0b98d4d3c02068594 HTTP/1.1" 200 None 2026-03-27 08:38:23,687 - lava_callback - INFO - Completed processing callback for node 69c641c0b98d4d3c02068594 2026-03-27 08:38:26,066 - lava_callback - INFO - Received callback for node 69c641bdb98d4d3c0206858b INFO: 172.20.0.1:58974 - "POST /node/69c641bdb98d4d3c0206858b HTTP/1.0" 202 Accepted 2026-03-27 08:38:26,072 - lava_callback - INFO - Starting async job submit for node 69c641bdb98d4d3c0206858b 2026-03-27 08:38:26,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:26,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641bdb98d4d3c0206858b HTTP/1.1" 200 None 2026-03-27 08:38:26,109 - lava_callback - DEBUG - Processing node 69c641bdb98d4d3c0206858b: baseline-arm-pengutronix 2026-03-27 08:38:26,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:26,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:26,378 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641bdb98d4d3c0206858b/log.txt.gz 2026-03-27 08:38:26,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:58988 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:38:26,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:26,650 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641bdb98d4d3c0206858b/lava-logs.yaml 2026-03-27 08:38:26,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:26,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:26,794 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641bdb98d4d3c0206858b/lava_callback.json.gz 2026-03-27 08:38:26,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:26,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641bdb98d4d3c0206858b HTTP/1.1" 200 None 2026-03-27 08:38:26,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:26,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:38:26,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:26,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641bdb98d4d3c0206858b HTTP/1.1" 200 None 2026-03-27 08:38:26,887 - lava_callback - INFO - Completed processing callback for node 69c641bdb98d4d3c0206858b 2026-03-27 08:38:31,920 - lava_callback - INFO - Received callback for node 69c641adb98d4d3c0206852f INFO: 172.20.0.1:60968 - "POST /node/69c641adb98d4d3c0206852f HTTP/1.0" 202 Accepted 2026-03-27 08:38:31,926 - lava_callback - INFO - Starting async job submit for node 69c641adb98d4d3c0206852f 2026-03-27 08:38:31,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:31,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641adb98d4d3c0206852f HTTP/1.1" 200 None 2026-03-27 08:38:31,964 - lava_callback - DEBUG - Processing node 69c641adb98d4d3c0206852f: baseline-arm-broonie 2026-03-27 08:38:32,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:32,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:32,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-27 08:38:32,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:32,335 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641adb98d4d3c0206852f/log.txt.gz 2026-03-27 08:38:32,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:32,657 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:32,658 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641adb98d4d3c0206852f/lava-logs.yaml 2026-03-27 08:38:32,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:32,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:32,809 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641adb98d4d3c0206852f/lava_callback.json.gz 2026-03-27 08:38:32,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:32,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641adb98d4d3c0206852f HTTP/1.1" 200 None 2026-03-27 08:38:32,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:32,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:38:32,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:32,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641adb98d4d3c0206852f HTTP/1.1" 200 None 2026-03-27 08:38:32,894 - lava_callback - INFO - Completed processing callback for node 69c641adb98d4d3c0206852f INFO: 172.20.0.1:60976 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:38:41,840 - lava_callback - INFO - Received callback for node 69c641b0b98d4d3c02068536 INFO: 172.20.0.1:59842 - "POST /node/69c641b0b98d4d3c02068536 HTTP/1.0" 202 Accepted 2026-03-27 08:38:41,846 - lava_callback - INFO - Starting async job submit for node 69c641b0b98d4d3c02068536 2026-03-27 08:38:41,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:41,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641b0b98d4d3c02068536 HTTP/1.1" 200 None 2026-03-27 08:38:41,885 - lava_callback - DEBUG - Processing node 69c641b0b98d4d3c02068536: baseline-arm-broonie 2026-03-27 08:38:42,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:42,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:42,237 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641b0b98d4d3c02068536/log.txt.gz 2026-03-27 08:38:42,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:42,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:42,559 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641b0b98d4d3c02068536/lava-logs.yaml 2026-03-27 08:38:42,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:42,700 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:42,702 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641b0b98d4d3c02068536/lava_callback.json.gz 2026-03-27 08:38:42,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:42,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641b0b98d4d3c02068536 HTTP/1.1" 200 None 2026-03-27 08:38:42,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:42,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:38:42,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:42,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641b0b98d4d3c02068536 HTTP/1.1" 200 None 2026-03-27 08:38:42,785 - lava_callback - INFO - Completed processing callback for node 69c641b0b98d4d3c02068536 INFO: 172.20.0.1:59858 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:38:51,336 - lava_callback - INFO - Received callback for node 69c641b3b98d4d3c0206854a INFO: 172.20.0.1:34294 - "POST /node/69c641b3b98d4d3c0206854a HTTP/1.0" 202 Accepted 2026-03-27 08:38:51,342 - lava_callback - INFO - Starting async job submit for node 69c641b3b98d4d3c0206854a 2026-03-27 08:38:51,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:51,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641b3b98d4d3c0206854a HTTP/1.1" 200 None 2026-03-27 08:38:51,384 - lava_callback - DEBUG - Processing node 69c641b3b98d4d3c0206854a: baseline-arm-clabbe 2026-03-27 08:38:51,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:51,770 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:51,772 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b3b98d4d3c0206854a/log.txt.gz 2026-03-27 08:38:51,871 - lava_callback - INFO - Received callback for node 69c641afb98d4d3c02068533 INFO: 172.20.0.1:34300 - "POST /node/69c641afb98d4d3c02068533 HTTP/1.0" 202 Accepted 2026-03-27 08:38:51,893 - lava_callback - INFO - Starting async job submit for node 69c641afb98d4d3c02068533 2026-03-27 08:38:51,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:51,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:51,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641afb98d4d3c02068533 HTTP/1.1" 200 None 2026-03-27 08:38:51,960 - lava_callback - DEBUG - Processing node 69c641afb98d4d3c02068533: baseline-arm-broonie 2026-03-27 08:38:52,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:52,223 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b3b98d4d3c0206854a/lava-logs.yaml 2026-03-27 08:38:52,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:52,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:52,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:52,404 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641afb98d4d3c02068533/log.txt.gz 2026-03-27 08:38:52,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:52,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b3b98d4d3c0206854a/lava_callback.json.gz 2026-03-27 08:38:52,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:52,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:52,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641b3b98d4d3c0206854a HTTP/1.1" 200 None 2026-03-27 08:38:52,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:52,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:38:52,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:52,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641b3b98d4d3c0206854a HTTP/1.1" 200 None 2026-03-27 08:38:52,624 - lava_callback - INFO - Completed processing callback for node 69c641b3b98d4d3c0206854a 2026-03-27 08:38:52,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:52,810 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641afb98d4d3c02068533/lava-logs.yaml 2026-03-27 08:38:52,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:38:52,967 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:38:52,969 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641afb98d4d3c02068533/lava_callback.json.gz 2026-03-27 08:38:52,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:53,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641afb98d4d3c02068533 HTTP/1.1" 200 None 2026-03-27 08:38:53,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:53,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:38:53,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:38:53,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641afb98d4d3c02068533 HTTP/1.1" 200 None 2026-03-27 08:38:53,058 - lava_callback - INFO - Completed processing callback for node 69c641afb98d4d3c02068533 INFO: 172.20.0.1:34306 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:39:02,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:02,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60642 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:39:27,946 - lava_callback - INFO - Received callback for node 69c641beb98d4d3c0206858e INFO: 172.20.0.1:60650 - "POST /node/69c641beb98d4d3c0206858e HTTP/1.0" 202 Accepted 2026-03-27 08:39:27,952 - lava_callback - INFO - Starting async job submit for node 69c641beb98d4d3c0206858e 2026-03-27 08:39:27,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:27,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641beb98d4d3c0206858e HTTP/1.1" 200 None 2026-03-27 08:39:27,988 - lava_callback - DEBUG - Processing node 69c641beb98d4d3c0206858e: baseline-arm-pengutronix 2026-03-27 08:39:28,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:39:28,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:39:28,263 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641beb98d4d3c0206858e/log.txt.gz 2026-03-27 08:39:28,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:39:28,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:39:28,555 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641beb98d4d3c0206858e/lava-logs.yaml 2026-03-27 08:39:28,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:39:28,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:39:28,698 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641beb98d4d3c0206858e/lava_callback.json.gz 2026-03-27 08:39:28,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:28,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641beb98d4d3c0206858e HTTP/1.1" 200 None 2026-03-27 08:39:28,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:28,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:39:28,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:28,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641beb98d4d3c0206858e HTTP/1.1" 200 None 2026-03-27 08:39:28,868 - lava_callback - INFO - Completed processing callback for node 69c641beb98d4d3c0206858e 2026-03-27 08:39:32,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:32,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43778 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:39:41,107 - lava_callback - INFO - Received callback for node 69c641d8b98d4d3c020685c2 INFO: 172.20.0.1:43188 - "POST /node/69c641d8b98d4d3c020685c2 HTTP/1.0" 202 Accepted 2026-03-27 08:39:41,114 - lava_callback - INFO - Starting async job submit for node 69c641d8b98d4d3c020685c2 2026-03-27 08:39:41,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:41,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d8b98d4d3c020685c2 HTTP/1.1" 200 None 2026-03-27 08:39:41,159 - lava_callback - DEBUG - Processing node 69c641d8b98d4d3c020685c2: ltp-dio 2026-03-27 08:39:41,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:39:41,678 - lava_callback - INFO - Received callback for node 69c641acb98d4d3c0206852c INFO: 172.20.0.1:43192 - "POST /node/69c641acb98d4d3c0206852c HTTP/1.0" 202 Accepted 2026-03-27 08:39:41,684 - lava_callback - INFO - Starting async job submit for node 69c641acb98d4d3c0206852c 2026-03-27 08:39:41,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:41,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641acb98d4d3c0206852c HTTP/1.1" 200 None 2026-03-27 08:39:41,723 - lava_callback - DEBUG - Processing node 69c641acb98d4d3c0206852c: baseline-arm-broonie 2026-03-27 08:39:41,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:39:41,785 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c641d8b98d4d3c020685c2/log.txt.gz 2026-03-27 08:39:42,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:39:42,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:39:42,276 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:39:42,278 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641acb98d4d3c0206852c/log.txt.gz 2026-03-27 08:39:42,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:39:42,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:39:42,416 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c641d8b98d4d3c020685c2/lava-logs.yaml 2026-03-27 08:39:42,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:39:42,602 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:39:42,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c641d8b98d4d3c020685c2/lava_callback.json.gz 2026-03-27 08:39:42,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:42,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d8b98d4d3c020685c2 HTTP/1.1" 200 None 2026-03-27 08:39:42,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:42,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:39:42,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:42,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641d8b98d4d3c020685c2 HTTP/1.1" 200 None 2026-03-27 08:39:42,696 - lava_callback - INFO - Completed processing callback for node 69c641d8b98d4d3c020685c2 2026-03-27 08:39:42,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:39:42,862 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641acb98d4d3c0206852c/lava-logs.yaml 2026-03-27 08:39:42,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:39:43,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:39:43,015 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c641acb98d4d3c0206852c/lava_callback.json.gz 2026-03-27 08:39:43,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:43,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641acb98d4d3c0206852c HTTP/1.1" 200 None 2026-03-27 08:39:43,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:43,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:39:43,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:39:43,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641acb98d4d3c0206852c HTTP/1.1" 200 None 2026-03-27 08:39:43,135 - lava_callback - INFO - Completed processing callback for node 69c641acb98d4d3c0206852c INFO: 172.20.0.1:43194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49018 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:40:02,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:40:02,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43776 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:40:42,390 - lava_callback - INFO - Received callback for node 69c641b4b98d4d3c0206854d INFO: 172.20.0.1:49596 - "POST /node/69c641b4b98d4d3c0206854d HTTP/1.0" 202 Accepted 2026-03-27 08:40:42,399 - lava_callback - INFO - Starting async job submit for node 69c641b4b98d4d3c0206854d 2026-03-27 08:40:42,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:40:42,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641b4b98d4d3c0206854d HTTP/1.1" 200 None 2026-03-27 08:40:42,469 - lava_callback - DEBUG - Processing node 69c641b4b98d4d3c0206854d: baseline-arm-clabbe 2026-03-27 08:40:42,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:40:42,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:40:42,908 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b4b98d4d3c0206854d/log.txt.gz 2026-03-27 08:40:43,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:40:43,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:40:43,379 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b4b98d4d3c0206854d/lava-logs.yaml 2026-03-27 08:40:43,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:40:43,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:40:43,892 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b4b98d4d3c0206854d/lava_callback.json.gz 2026-03-27 08:40:43,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:40:43,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641b4b98d4d3c0206854d HTTP/1.1" 200 None 2026-03-27 08:40:43,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:40:43,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:40:43,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:40:44,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641b4b98d4d3c0206854d HTTP/1.1" 200 None 2026-03-27 08:40:44,009 - lava_callback - INFO - Completed processing callback for node 69c641b4b98d4d3c0206854d INFO: 172.20.0.1:49598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60902 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:41:02,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:02,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59396 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:41:41,922 - lava_callback - INFO - Received callback for node 69c641d1b98d4d3c020685a6 INFO: 172.20.0.1:45776 - "POST /node/69c641d1b98d4d3c020685a6 HTTP/1.0" 202 Accepted 2026-03-27 08:41:41,936 - lava_callback - INFO - Starting async job submit for node 69c641d1b98d4d3c020685a6 2026-03-27 08:41:41,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:42,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d1b98d4d3c020685a6 HTTP/1.1" 200 None 2026-03-27 08:41:42,015 - lava_callback - DEBUG - Processing node 69c641d1b98d4d3c020685a6: ltp-cap_bounds 2026-03-27 08:41:42,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:41:42,831 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:41:42,833 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c641d1b98d4d3c020685a6/log.txt.gz 2026-03-27 08:41:43,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:41:43,530 - lava_callback - INFO - Received callback for node 69c641d4b98d4d3c020685ac INFO: 172.20.0.1:45782 - "POST /node/69c641d4b98d4d3c020685ac HTTP/1.0" 202 Accepted 2026-03-27 08:41:43,545 - lava_callback - INFO - Starting async job submit for node 69c641d4b98d4d3c020685ac 2026-03-27 08:41:43,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:43,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d4b98d4d3c020685ac HTTP/1.1" 200 None 2026-03-27 08:41:43,700 - lava_callback - DEBUG - Processing node 69c641d4b98d4d3c020685ac: ltp-controllers 2026-03-27 08:41:43,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:41:43,782 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c641d1b98d4d3c020685a6/lava-logs.yaml 2026-03-27 08:41:43,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:41:44,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:41:44,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:41:44,444 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c641d1b98d4d3c020685a6/lava_callback.json.gz 2026-03-27 08:41:44,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:44,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d1b98d4d3c020685a6 HTTP/1.1" 200 None 2026-03-27 08:41:44,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:44,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:41:44,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:44,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641d1b98d4d3c020685a6 HTTP/1.1" 200 None 2026-03-27 08:41:44,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:41:44,577 - lava_callback - INFO - Completed processing callback for node 69c641d1b98d4d3c020685a6 2026-03-27 08:41:44,587 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c641d4b98d4d3c020685ac/log.txt.gz 2026-03-27 08:41:45,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:41:45,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:41:45,823 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c641d4b98d4d3c020685ac/lava-logs.yaml 2026-03-27 08:41:45,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:41:46,028 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:41:46,031 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c641d4b98d4d3c020685ac/lava_callback.json.gz 2026-03-27 08:41:46,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:46,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d4b98d4d3c020685ac HTTP/1.1" 200 None 2026-03-27 08:41:46,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:46,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:41:46,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:46,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641d4b98d4d3c020685ac HTTP/1.1" 200 None 2026-03-27 08:41:46,170 - lava_callback - INFO - Completed processing callback for node 69c641d4b98d4d3c020685ac INFO: 172.20.0.1:45786 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:41:52,453 - lava_callback - INFO - Received callback for node 69c641d2b98d4d3c020685a9 INFO: 172.20.0.1:54692 - "POST /node/69c641d2b98d4d3c020685a9 HTTP/1.0" 202 Accepted 2026-03-27 08:41:52,460 - lava_callback - INFO - Starting async job submit for node 69c641d2b98d4d3c020685a9 2026-03-27 08:41:52,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:52,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d2b98d4d3c020685a9 HTTP/1.1" 200 None 2026-03-27 08:41:52,500 - lava_callback - DEBUG - Processing node 69c641d2b98d4d3c020685a9: ltp-containers 2026-03-27 08:41:52,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:41:52,967 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:41:52,969 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c641d2b98d4d3c020685a9/log.txt.gz 2026-03-27 08:41:53,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:41:53,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:41:53,597 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c641d2b98d4d3c020685a9/lava-logs.yaml 2026-03-27 08:41:53,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:41:53,777 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:41:53,779 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c641d2b98d4d3c020685a9/lava_callback.json.gz 2026-03-27 08:41:53,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:53,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d2b98d4d3c020685a9 HTTP/1.1" 200 None 2026-03-27 08:41:53,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:53,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:41:53,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:41:53,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641d2b98d4d3c020685a9 HTTP/1.1" 200 None 2026-03-27 08:41:53,868 - lava_callback - INFO - Completed processing callback for node 69c641d2b98d4d3c020685a9 INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:42:02,268 - lava_callback - INFO - Received callback for node 69c641e4b98d4d3c020685db INFO: 172.20.0.1:58550 - "POST /node/69c641e4b98d4d3c020685db HTTP/1.0" 202 Accepted 2026-03-27 08:42:02,275 - lava_callback - INFO - Starting async job submit for node 69c641e4b98d4d3c020685db 2026-03-27 08:42:02,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:42:02,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e4b98d4d3c020685db HTTP/1.1" 200 None 2026-03-27 08:42:02,316 - lava_callback - DEBUG - Processing node 69c641e4b98d4d3c020685db: ltp-syscalls 2026-03-27 08:42:02,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:42:02,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 08:42:03,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:42:03,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:42:03,180 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c641e4b98d4d3c020685db/log.txt.gz 2026-03-27 08:42:03,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:42:03,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:42:03,808 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c641e4b98d4d3c020685db/lava-logs.yaml 2026-03-27 08:42:03,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:42:04,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:42:04,005 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c641e4b98d4d3c020685db/lava_callback.json.gz 2026-03-27 08:42:04,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:42:04,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e4b98d4d3c020685db HTTP/1.1" 200 None 2026-03-27 08:42:04,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:42:04,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:42:04,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:42:04,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641e4b98d4d3c020685db HTTP/1.1" 200 None 2026-03-27 08:42:04,116 - lava_callback - INFO - Completed processing callback for node 69c641e4b98d4d3c020685db INFO: 172.20.0.1:58562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47714 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41516 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:42:32,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:42:32,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36524 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:42:41,500 - lava_callback - INFO - Received callback for node 69c641b7b98d4d3c0206857a INFO: 172.20.0.1:41766 - "POST /node/69c641b7b98d4d3c0206857a HTTP/1.0" 202 Accepted 2026-03-27 08:42:41,507 - lava_callback - INFO - Starting async job submit for node 69c641b7b98d4d3c0206857a 2026-03-27 08:42:41,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:42:41,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641b7b98d4d3c0206857a HTTP/1.1" 200 None 2026-03-27 08:42:41,545 - lava_callback - DEBUG - Processing node 69c641b7b98d4d3c0206857a: baseline-arm-clabbe 2026-03-27 08:42:41,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:42:41,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:42:41,865 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b7b98d4d3c0206857a/log.txt.gz 2026-03-27 08:42:41,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:42:42,179 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:42:42,180 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b7b98d4d3c0206857a/lava-logs.yaml 2026-03-27 08:42:42,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:42:42,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:42:42,320 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b7b98d4d3c0206857a/lava_callback.json.gz 2026-03-27 08:42:42,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:42:42,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641b7b98d4d3c0206857a HTTP/1.1" 200 None 2026-03-27 08:42:42,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:42:42,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:42:42,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:42:42,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641b7b98d4d3c0206857a HTTP/1.1" 200 None 2026-03-27 08:42:42,420 - lava_callback - INFO - Completed processing callback for node 69c641b7b98d4d3c0206857a INFO: 172.20.0.1:41772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49780 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:43:02,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:43:02,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44056 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37256 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:44:02,068 - lava_callback - INFO - Received callback for node 69c641d6b98d4d3c020685be INFO: 172.20.0.1:35396 - "POST /node/69c641d6b98d4d3c020685be HTTP/1.0" 202 Accepted 2026-03-27 08:44:02,074 - lava_callback - INFO - Starting async job submit for node 69c641d6b98d4d3c020685be 2026-03-27 08:44:02,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:44:02,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d6b98d4d3c020685be HTTP/1.1" 200 None 2026-03-27 08:44:02,115 - lava_callback - DEBUG - Processing node 69c641d6b98d4d3c020685be: ltp-cve 2026-03-27 08:44:02,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:44:02,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:44:02,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c641d6b98d4d3c020685be/log.txt.gz 2026-03-27 08:44:02,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:44:03,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:44:03,145 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c641d6b98d4d3c020685be/lava-logs.yaml 2026-03-27 08:44:03,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:44:03,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:44:03,336 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c641d6b98d4d3c020685be/lava_callback.json.gz 2026-03-27 08:44:03,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:44:03,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d6b98d4d3c020685be HTTP/1.1" 200 None 2026-03-27 08:44:03,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:44:03,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:44:03,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:44:03,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641d6b98d4d3c020685be HTTP/1.1" 200 None 2026-03-27 08:44:03,419 - lava_callback - INFO - Completed processing callback for node 69c641d6b98d4d3c020685be INFO: 172.20.0.1:35402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39694 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:44:32,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:44:32,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58754 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:44:40,555 - lava_callback - INFO - Received callback for node 69c641b8b98d4d3c0206857d INFO: 172.20.0.1:32856 - "POST /node/69c641b8b98d4d3c0206857d HTTP/1.0" 202 Accepted 2026-03-27 08:44:40,564 - lava_callback - INFO - Starting async job submit for node 69c641b8b98d4d3c0206857d 2026-03-27 08:44:40,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:44:40,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641b8b98d4d3c0206857d HTTP/1.1" 200 None 2026-03-27 08:44:40,635 - lava_callback - DEBUG - Processing node 69c641b8b98d4d3c0206857d: baseline-arm-clabbe 2026-03-27 08:44:41,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:44:41,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:44:41,436 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b8b98d4d3c0206857d/log.txt.gz 2026-03-27 08:44:41,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:44:42,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:44:42,158 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b8b98d4d3c0206857d/lava-logs.yaml 2026-03-27 08:44:42,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:44:42,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:44:42,615 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c641b8b98d4d3c0206857d/lava_callback.json.gz 2026-03-27 08:44:42,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:44:42,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641b8b98d4d3c0206857d HTTP/1.1" 200 None 2026-03-27 08:44:42,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:44:42,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:44:42,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:44:42,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641b8b98d4d3c0206857d HTTP/1.1" 200 None 2026-03-27 08:44:42,776 - lava_callback - INFO - Completed processing callback for node 69c641b8b98d4d3c0206857d INFO: 172.20.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41994 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:45:02,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:02,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40840 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:45:13,663 - lava_callback - INFO - Received callback for node 69c3a092bbc022e792644754 INFO: 172.20.0.1:51552 - "POST /node/69c3a092bbc022e792644754 HTTP/1.0" 202 Accepted 2026-03-27 08:45:13,672 - lava_callback - INFO - Starting async job submit for node 69c3a092bbc022e792644754 2026-03-27 08:45:13,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:13,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a092bbc022e792644754 HTTP/1.1" 200 None 2026-03-27 08:45:13,705 - lava_callback - DEBUG - Processing node 69c3a092bbc022e792644754: baseline-arm-baylibre 2026-03-27 08:45:13,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a092bbc022e792644754 2026-03-27 08:45:13,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:13,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a092bbc022e792644754 HTTP/1.1" 200 None 2026-03-27 08:45:13,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:13,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 08:45:13,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:13,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a092bbc022e792644754 HTTP/1.1" 200 None 2026-03-27 08:45:13,793 - lava_callback - ERROR - Error processing callback for node 69c3a092bbc022e792644754: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48912 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:45:42,064 - lava_callback - INFO - Received callback for node 69c641d9b98d4d3c020685c6 INFO: 172.20.0.1:40878 - "POST /node/69c641d9b98d4d3c020685c6 HTTP/1.0" 202 Accepted 2026-03-27 08:45:42,071 - lava_callback - INFO - Starting async job submit for node 69c641d9b98d4d3c020685c6 2026-03-27 08:45:42,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:42,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d9b98d4d3c020685c6 HTTP/1.1" 200 None 2026-03-27 08:45:42,115 - lava_callback - DEBUG - Processing node 69c641d9b98d4d3c020685c6: ltp-fs_bind 2026-03-27 08:45:42,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:45:42,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:45:42,843 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c641d9b98d4d3c020685c6/log.txt.gz 2026-03-27 08:45:43,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:45:43,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:45:43,551 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c641d9b98d4d3c020685c6/lava-logs.yaml 2026-03-27 08:45:43,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:45:43,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:45:43,759 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c641d9b98d4d3c020685c6/lava_callback.json.gz 2026-03-27 08:45:43,790 - lava_callback - INFO - Received callback for node 69c641dbb98d4d3c020685c9 INFO: 172.20.0.1:40888 - "POST /node/69c641dbb98d4d3c020685c9 HTTP/1.0" 202 Accepted 2026-03-27 08:45:43,838 - lava_callback - INFO - Starting async job submit for node 69c641dbb98d4d3c020685c9 2026-03-27 08:45:43,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:43,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:43,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641d9b98d4d3c020685c6 HTTP/1.1" 200 None 2026-03-27 08:45:43,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:43,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641dbb98d4d3c020685c9 HTTP/1.1" 200 None 2026-03-27 08:45:44,005 - lava_callback - DEBUG - Processing node 69c641dbb98d4d3c020685c9: ltp-fsx 2026-03-27 08:45:44,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:45:44,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:44,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641d9b98d4d3c020685c6 HTTP/1.1" 200 None 2026-03-27 08:45:44,105 - lava_callback - INFO - Completed processing callback for node 69c641d9b98d4d3c020685c6 2026-03-27 08:45:44,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:45:44,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:45:44,804 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c641dbb98d4d3c020685c9/log.txt.gz 2026-03-27 08:45:45,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:45:45,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:45:45,422 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c641dbb98d4d3c020685c9/lava-logs.yaml 2026-03-27 08:45:45,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:45:45,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:45:45,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c641dbb98d4d3c020685c9/lava_callback.json.gz 2026-03-27 08:45:45,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:45,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641dbb98d4d3c020685c9 HTTP/1.1" 200 None 2026-03-27 08:45:45,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:45,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:45:45,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:45:45,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641dbb98d4d3c020685c9 HTTP/1.1" 200 None 2026-03-27 08:45:45,745 - lava_callback - INFO - Completed processing callback for node 69c641dbb98d4d3c020685c9 INFO: 172.20.0.1:40904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:46:02,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:46:02,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41268 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:46:12,344 - lava_callback - INFO - Received callback for node 69c641ddb98d4d3c020685cc INFO: 172.20.0.1:52656 - "POST /node/69c641ddb98d4d3c020685cc HTTP/1.0" 202 Accepted 2026-03-27 08:46:12,355 - lava_callback - INFO - Starting async job submit for node 69c641ddb98d4d3c020685cc 2026-03-27 08:46:12,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:46:12,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641ddb98d4d3c020685cc HTTP/1.1" 200 None 2026-03-27 08:46:12,431 - lava_callback - DEBUG - Processing node 69c641ddb98d4d3c020685cc: ltp-input 2026-03-27 08:46:12,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:46:13,314 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:46:13,316 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c641ddb98d4d3c020685cc/log.txt.gz 2026-03-27 08:46:13,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:46:14,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:46:14,020 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c641ddb98d4d3c020685cc/lava-logs.yaml 2026-03-27 08:46:14,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:46:14,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:46:14,179 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c641ddb98d4d3c020685cc/lava_callback.json.gz 2026-03-27 08:46:14,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:46:14,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641ddb98d4d3c020685cc HTTP/1.1" 200 None 2026-03-27 08:46:14,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:46:14,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:46:14,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:46:14,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641ddb98d4d3c020685cc HTTP/1.1" 200 None 2026-03-27 08:46:14,261 - lava_callback - INFO - Completed processing callback for node 69c641ddb98d4d3c020685cc INFO: 172.20.0.1:52664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41978 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:46:32,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:46:32,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36206 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:47:53,765 - lava_callback - INFO - Received callback for node 69c3a124bbc022e7926447ba INFO: 172.20.0.1:56848 - "POST /node/69c3a124bbc022e7926447ba HTTP/1.0" 202 Accepted 2026-03-27 08:47:53,771 - lava_callback - INFO - Starting async job submit for node 69c3a124bbc022e7926447ba 2026-03-27 08:47:53,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:47:53,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a124bbc022e7926447ba HTTP/1.1" 200 None 2026-03-27 08:47:53,792 - lava_callback - DEBUG - Processing node 69c3a124bbc022e7926447ba: baseline-arm-baylibre 2026-03-27 08:47:53,792 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a124bbc022e7926447ba 2026-03-27 08:47:53,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:47:53,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a124bbc022e7926447ba HTTP/1.1" 200 None 2026-03-27 08:47:53,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:47:53,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 08:47:53,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:47:53,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a124bbc022e7926447ba HTTP/1.1" 200 None 2026-03-27 08:47:53,869 - lava_callback - ERROR - Error processing callback for node 69c3a124bbc022e7926447ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37782 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:48:22,114 - lava_callback - INFO - Received callback for node 69c641deb98d4d3c020685cf INFO: 172.20.0.1:37784 - "POST /node/69c641deb98d4d3c020685cf HTTP/1.0" 202 Accepted 2026-03-27 08:48:22,121 - lava_callback - INFO - Starting async job submit for node 69c641deb98d4d3c020685cf 2026-03-27 08:48:22,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:48:22,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641deb98d4d3c020685cf HTTP/1.1" 200 None 2026-03-27 08:48:22,177 - lava_callback - DEBUG - Processing node 69c641deb98d4d3c020685cf: ltp-ipc 2026-03-27 08:48:22,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:48:23,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:48:23,465 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c641deb98d4d3c020685cf/log.txt.gz 2026-03-27 08:48:23,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:48:25,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:48:25,073 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c641deb98d4d3c020685cf/lava-logs.yaml 2026-03-27 08:48:25,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:48:26,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:48:26,199 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c641deb98d4d3c020685cf/lava_callback.json.gz 2026-03-27 08:48:26,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:48:26,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641deb98d4d3c020685cf HTTP/1.1" 200 None 2026-03-27 08:48:26,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:48:26,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:48:26,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:48:26,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641deb98d4d3c020685cf HTTP/1.1" 200 None 2026-03-27 08:48:26,352 - lava_callback - INFO - Completed processing callback for node 69c641deb98d4d3c020685cf INFO: 172.20.0.1:37790 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:48:32,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:48:32,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50032 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:48:53,788 - lava_callback - INFO - Received callback for node 69c3a16ebbc022e7926447ef INFO: 172.20.0.1:58404 - "POST /node/69c3a16ebbc022e7926447ef HTTP/1.0" 202 Accepted 2026-03-27 08:48:53,798 - lava_callback - INFO - Starting async job submit for node 69c3a16ebbc022e7926447ef 2026-03-27 08:48:53,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:48:53,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a16ebbc022e7926447ef HTTP/1.1" 200 None 2026-03-27 08:48:53,822 - lava_callback - DEBUG - Processing node 69c3a16ebbc022e7926447ef: baseline-arm-baylibre 2026-03-27 08:48:53,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a16ebbc022e7926447ef 2026-03-27 08:48:53,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:48:53,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a16ebbc022e7926447ef HTTP/1.1" 200 None 2026-03-27 08:48:53,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:48:53,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 08:48:53,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:48:53,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a16ebbc022e7926447ef HTTP/1.1" 200 None 2026-03-27 08:48:53,893 - lava_callback - ERROR - Error processing callback for node 69c3a16ebbc022e7926447ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37038 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:49:42,198 - lava_callback - INFO - Received callback for node 69c641e0b98d4d3c020685d2 INFO: 172.20.0.1:52068 - "POST /node/69c641e0b98d4d3c020685d2 HTTP/1.0" 202 Accepted 2026-03-27 08:49:42,208 - lava_callback - INFO - Starting async job submit for node 69c641e0b98d4d3c020685d2 2026-03-27 08:49:42,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:42,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e0b98d4d3c020685d2 HTTP/1.1" 200 None 2026-03-27 08:49:42,261 - lava_callback - DEBUG - Processing node 69c641e0b98d4d3c020685d2: ltp-pty 2026-03-27 08:49:42,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:49:43,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:49:43,537 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c641e0b98d4d3c020685d2/log.txt.gz 2026-03-27 08:49:43,852 - lava_callback - INFO - Received callback for node 69c641e1b98d4d3c020685d5 INFO: 172.20.0.1:52074 - "POST /node/69c641e1b98d4d3c020685d5 HTTP/1.0" 202 Accepted 2026-03-27 08:49:43,879 - lava_callback - INFO - Starting async job submit for node 69c641e1b98d4d3c020685d5 2026-03-27 08:49:43,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:49:43,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:43,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e1b98d4d3c020685d5 HTTP/1.1" 200 None 2026-03-27 08:49:43,952 - lava_callback - DEBUG - Processing node 69c641e1b98d4d3c020685d5: ltp-sched 2026-03-27 08:49:44,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:49:44,864 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:49:44,866 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c641e0b98d4d3c020685d2/lava-logs.yaml 2026-03-27 08:49:44,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:49:45,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:49:45,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c641e1b98d4d3c020685d5/log.txt.gz 2026-03-27 08:49:45,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:49:45,745 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:49:45,747 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c641e0b98d4d3c020685d2/lava_callback.json.gz 2026-03-27 08:49:45,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:45,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e0b98d4d3c020685d2 HTTP/1.1" 200 None 2026-03-27 08:49:45,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:45,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:49:45,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:45,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641e0b98d4d3c020685d2 HTTP/1.1" 200 None 2026-03-27 08:49:45,874 - lava_callback - INFO - Completed processing callback for node 69c641e0b98d4d3c020685d2 2026-03-27 08:49:46,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:49:46,475 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c641e1b98d4d3c020685d5/lava-logs.yaml 2026-03-27 08:49:46,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:52082 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:49:47,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:49:47,370 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c641e1b98d4d3c020685d5/lava_callback.json.gz 2026-03-27 08:49:47,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:47,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e1b98d4d3c020685d5 HTTP/1.1" 200 None 2026-03-27 08:49:47,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:47,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:49:47,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:47,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641e1b98d4d3c020685d5 HTTP/1.1" 200 None 2026-03-27 08:49:47,485 - lava_callback - INFO - Completed processing callback for node 69c641e1b98d4d3c020685d5 2026-03-27 08:49:49,806 - lava_callback - INFO - Received callback for node 69c5ea112cb62322ad968f77 INFO: 172.20.0.1:34234 - "POST /node/69c5ea112cb62322ad968f77 HTTP/1.0" 202 Accepted 2026-03-27 08:49:49,814 - lava_callback - INFO - Starting async job submit for node 69c5ea112cb62322ad968f77 2026-03-27 08:49:49,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:49,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ea112cb62322ad968f77 HTTP/1.1" 200 None 2026-03-27 08:49:49,881 - lava_callback - DEBUG - Processing node 69c5ea112cb62322ad968f77: baseline-nfs-arm64 2026-03-27 08:49:50,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:49:51,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:49:51,533 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5ea112cb62322ad968f77/log.txt.gz 2026-03-27 08:49:51,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:49:52,687 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:49:52,690 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5ea112cb62322ad968f77/lava-logs.yaml 2026-03-27 08:49:52,703 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:49:53,433 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:49:53,435 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5ea112cb62322ad968f77/lava_callback.json.gz 2026-03-27 08:49:53,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:53,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ea112cb62322ad968f77 HTTP/1.1" 200 None 2026-03-27 08:49:53,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:53,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 08:49:53,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:49:53,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ea112cb62322ad968f77 HTTP/1.1" 200 None 2026-03-27 08:49:53,654 - lava_callback - INFO - Completed processing callback for node 69c5ea112cb62322ad968f77 INFO: 172.20.0.1:34250 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:50:02,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:02,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50068 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:50:13,988 - lava_callback - INFO - Received callback for node 69c3a1b7bbc022e792644811 INFO: 172.20.0.1:58774 - "POST /node/69c3a1b7bbc022e792644811 HTTP/1.0" 202 Accepted 2026-03-27 08:50:13,994 - lava_callback - INFO - Starting async job submit for node 69c3a1b7bbc022e792644811 2026-03-27 08:50:13,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:14,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a1b7bbc022e792644811 HTTP/1.1" 200 None 2026-03-27 08:50:14,010 - lava_callback - DEBUG - Processing node 69c3a1b7bbc022e792644811: baseline-arm-baylibre 2026-03-27 08:50:14,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a1b7bbc022e792644811 2026-03-27 08:50:14,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:14,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a1b7bbc022e792644811 HTTP/1.1" 200 None 2026-03-27 08:50:14,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:14,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 08:50:14,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:14,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a1b7bbc022e792644811 HTTP/1.1" 200 None 2026-03-27 08:50:14,083 - lava_callback - ERROR - Error processing callback for node 69c3a1b7bbc022e792644811: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39648 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:50:32,630 - lava_callback - INFO - Received callback for node 69c641e3b98d4d3c020685d8 INFO: 172.20.0.1:49486 - "POST /node/69c641e3b98d4d3c020685d8 HTTP/1.0" 202 Accepted 2026-03-27 08:50:32,637 - lava_callback - INFO - Starting async job submit for node 69c641e3b98d4d3c020685d8 2026-03-27 08:50:32,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:32,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e3b98d4d3c020685d8 HTTP/1.1" 200 None 2026-03-27 08:50:32,677 - lava_callback - DEBUG - Processing node 69c641e3b98d4d3c020685d8: ltp-smoke 2026-03-27 08:50:33,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:50:33,159 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:50:33,161 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c641e3b98d4d3c020685d8/log.txt.gz 2026-03-27 08:50:33,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:50:33,589 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:50:33,591 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c641e3b98d4d3c020685d8/lava-logs.yaml 2026-03-27 08:50:33,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:50:33,775 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:50:33,776 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c641e3b98d4d3c020685d8/lava_callback.json.gz 2026-03-27 08:50:33,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:33,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e3b98d4d3c020685d8 HTTP/1.1" 200 None 2026-03-27 08:50:33,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:33,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:50:33,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:33,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641e3b98d4d3c020685d8 HTTP/1.1" 200 None 2026-03-27 08:50:33,864 - lava_callback - INFO - Completed processing callback for node 69c641e3b98d4d3c020685d8 INFO: 172.20.0.1:49488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49564 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:50:54,140 - lava_callback - INFO - Received callback for node 69c3a1ddbbc022e792644829 INFO: 172.20.0.1:36894 - "POST /node/69c3a1ddbbc022e792644829 HTTP/1.0" 202 Accepted 2026-03-27 08:50:54,145 - lava_callback - INFO - Starting async job submit for node 69c3a1ddbbc022e792644829 2026-03-27 08:50:54,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:54,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a1ddbbc022e792644829 HTTP/1.1" 200 None 2026-03-27 08:50:54,161 - lava_callback - DEBUG - Processing node 69c3a1ddbbc022e792644829: baseline-arm-baylibre 2026-03-27 08:50:54,162 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a1ddbbc022e792644829 2026-03-27 08:50:54,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:54,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a1ddbbc022e792644829 HTTP/1.1" 200 None 2026-03-27 08:50:54,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:54,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 08:50:54,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:50:54,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a1ddbbc022e792644829 HTTP/1.1" 200 None 2026-03-27 08:50:54,220 - lava_callback - ERROR - Error processing callback for node 69c3a1ddbbc022e792644829: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36908 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:51:02,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:51:02,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51810 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:51:12,557 - lava_callback - INFO - Received callback for node 69c64043b98d4d3c0206845b INFO: 172.20.0.1:60790 - "POST /node/69c64043b98d4d3c0206845b HTTP/1.0" 202 Accepted 2026-03-27 08:51:12,564 - lava_callback - INFO - Starting async job submit for node 69c64043b98d4d3c0206845b 2026-03-27 08:51:12,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:51:12,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64043b98d4d3c0206845b HTTP/1.1" 200 None 2026-03-27 08:51:12,655 - lava_callback - DEBUG - Processing node 69c64043b98d4d3c0206845b: ltp-smoke 2026-03-27 08:51:13,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:51:13,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:51:13,264 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c64043b98d4d3c0206845b/log.txt.gz 2026-03-27 08:51:13,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:51:13,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:51:13,750 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c64043b98d4d3c0206845b/lava-logs.yaml 2026-03-27 08:51:13,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:51:13,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:51:13,921 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c64043b98d4d3c0206845b/lava_callback.json.gz 2026-03-27 08:51:14,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:51:14,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64043b98d4d3c0206845b HTTP/1.1" 200 None 2026-03-27 08:51:14,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:51:14,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce8b98d4d3c02067dcb HTTP/1.1" 200 None 2026-03-27 08:51:14,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:51:14,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64043b98d4d3c0206845b HTTP/1.1" 200 None 2026-03-27 08:51:14,261 - lava_callback - INFO - Completed processing callback for node 69c64043b98d4d3c0206845b INFO: 172.20.0.1:60806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49278 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:51:32,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:51:32,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46678 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:51:53,995 - lava_callback - INFO - Received callback for node 69c3a225bbc022e792644842 INFO: 172.20.0.1:42606 - "POST /node/69c3a225bbc022e792644842 HTTP/1.0" 202 Accepted 2026-03-27 08:51:54,000 - lava_callback - INFO - Starting async job submit for node 69c3a225bbc022e792644842 2026-03-27 08:51:54,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:51:54,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a225bbc022e792644842 HTTP/1.1" 200 None 2026-03-27 08:51:54,016 - lava_callback - DEBUG - Processing node 69c3a225bbc022e792644842: baseline-arm-baylibre 2026-03-27 08:51:54,017 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a225bbc022e792644842 2026-03-27 08:51:54,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:51:54,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a225bbc022e792644842 HTTP/1.1" 200 None 2026-03-27 08:51:54,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:51:54,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 08:51:54,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:51:54,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a225bbc022e792644842 HTTP/1.1" 200 None 2026-03-27 08:51:54,083 - lava_callback - ERROR - Error processing callback for node 69c3a225bbc022e792644842: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55638 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:52:09,163 - lava_callback - INFO - Received callback for node 69c64515b98d4d3c020687b5 INFO: 172.20.0.1:55644 - "POST /node/69c64515b98d4d3c020687b5 HTTP/1.0" 202 Accepted 2026-03-27 08:52:09,172 - lava_callback - INFO - Starting async job submit for node 69c64515b98d4d3c020687b5 2026-03-27 08:52:09,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:09,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64515b98d4d3c020687b5 HTTP/1.1" 200 None 2026-03-27 08:52:09,191 - lava_callback - DEBUG - Processing node 69c64515b98d4d3c020687b5: baseline-arm 2026-03-27 08:52:09,196 - lava_callback - WARNING - Failed to upload log 2026-03-27 08:52:09,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:52:09,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:52:09,339 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c64515b98d4d3c020687b5/lava-logs.yaml 2026-03-27 08:52:09,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:52:09,452 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:52:09,453 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c64515b98d4d3c020687b5/lava_callback.json.gz 2026-03-27 08:52:09,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:09,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64515b98d4d3c020687b5 HTTP/1.1" 200 None 2026-03-27 08:52:09,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:09,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:52:09,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:09,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64515b98d4d3c020687b5 HTTP/1.1" 200 None 2026-03-27 08:52:09,519 - lava_callback - INFO - Completed processing callback for node 69c64515b98d4d3c020687b5 INFO: 172.20.0.1:59906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51718 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:52:32,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:32,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 08:52:33,989 - lava_callback - INFO - Received callback for node 69c3a250bbc022e7926448d2 INFO: 172.20.0.1:56864 - "POST /node/69c3a250bbc022e7926448d2 HTTP/1.0" 202 Accepted 2026-03-27 08:52:33,994 - lava_callback - INFO - Starting async job submit for node 69c3a250bbc022e7926448d2 2026-03-27 08:52:33,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:34,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a250bbc022e7926448d2 HTTP/1.1" 200 None 2026-03-27 08:52:34,011 - lava_callback - DEBUG - Processing node 69c3a250bbc022e7926448d2: baseline-arm64-baylibre 2026-03-27 08:52:34,012 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a250bbc022e7926448d2 2026-03-27 08:52:34,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:34,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a250bbc022e7926448d2 HTTP/1.1" 200 None 2026-03-27 08:52:34,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:34,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:52:34,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:34,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a250bbc022e7926448d2 HTTP/1.1" 200 None 2026-03-27 08:52:34,076 - lava_callback - ERROR - Error processing callback for node 69c3a250bbc022e7926448d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56876 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:52:42,305 - lava_callback - INFO - Received callback for node 69c641e6b98d4d3c020685de INFO: 172.20.0.1:33450 - "POST /node/69c641e6b98d4d3c020685de HTTP/1.0" 202 Accepted 2026-03-27 08:52:42,312 - lava_callback - INFO - Starting async job submit for node 69c641e6b98d4d3c020685de 2026-03-27 08:52:42,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:42,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e6b98d4d3c020685de HTTP/1.1" 200 None 2026-03-27 08:52:42,354 - lava_callback - DEBUG - Processing node 69c641e6b98d4d3c020685de: ltp-syscalls-ipc 2026-03-27 08:52:42,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:52:42,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:52:42,891 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c641e6b98d4d3c020685de/log.txt.gz 2026-03-27 08:52:43,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:52:43,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:52:43,333 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c641e6b98d4d3c020685de/lava-logs.yaml 2026-03-27 08:52:43,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:52:43,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:52:43,537 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c641e6b98d4d3c020685de/lava_callback.json.gz 2026-03-27 08:52:43,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:43,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e6b98d4d3c020685de HTTP/1.1" 200 None 2026-03-27 08:52:43,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:43,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:52:43,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:43,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641e6b98d4d3c020685de HTTP/1.1" 200 None 2026-03-27 08:52:43,650 - lava_callback - INFO - Completed processing callback for node 69c641e6b98d4d3c020685de INFO: 172.20.0.1:33462 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:52:54,005 - lava_callback - INFO - Received callback for node 69c3a251bbc022e7926448f5 INFO: 172.20.0.1:49310 - "POST /node/69c3a251bbc022e7926448f5 HTTP/1.0" 202 Accepted 2026-03-27 08:52:54,010 - lava_callback - INFO - Starting async job submit for node 69c3a251bbc022e7926448f5 2026-03-27 08:52:54,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:54,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a251bbc022e7926448f5 HTTP/1.1" 200 None 2026-03-27 08:52:54,026 - lava_callback - DEBUG - Processing node 69c3a251bbc022e7926448f5: baseline-arm64-baylibre 2026-03-27 08:52:54,026 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a251bbc022e7926448f5 2026-03-27 08:52:54,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:54,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a251bbc022e7926448f5 HTTP/1.1" 200 None 2026-03-27 08:52:54,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:54,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:52:54,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:54,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a251bbc022e7926448f5 HTTP/1.1" 200 None 2026-03-27 08:52:54,085 - lava_callback - ERROR - Error processing callback for node 69c3a251bbc022e7926448f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:52:54,719 - lava_callback - INFO - Received callback for node 69c3a252bbc022e7926448f8 INFO: 172.20.0.1:49322 - "POST /node/69c3a252bbc022e7926448f8 HTTP/1.0" 202 Accepted 2026-03-27 08:52:54,725 - lava_callback - INFO - Starting async job submit for node 69c3a252bbc022e7926448f8 2026-03-27 08:52:54,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:54,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a252bbc022e7926448f8 HTTP/1.1" 200 None 2026-03-27 08:52:54,741 - lava_callback - DEBUG - Processing node 69c3a252bbc022e7926448f8: baseline-arm64-baylibre 2026-03-27 08:52:54,742 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a252bbc022e7926448f8 2026-03-27 08:52:54,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:54,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a252bbc022e7926448f8 HTTP/1.1" 200 None 2026-03-27 08:52:54,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:54,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:52:54,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:54,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a252bbc022e7926448f8 HTTP/1.1" 200 None 2026-03-27 08:52:54,799 - lava_callback - ERROR - Error processing callback for node 69c3a252bbc022e7926448f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:52:55,380 - lava_callback - INFO - Received callback for node 69c3a254bbc022e7926448ff INFO: 172.20.0.1:49334 - "POST /node/69c3a254bbc022e7926448ff HTTP/1.0" 202 Accepted 2026-03-27 08:52:55,385 - lava_callback - INFO - Starting async job submit for node 69c3a254bbc022e7926448ff 2026-03-27 08:52:55,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:55,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a254bbc022e7926448ff HTTP/1.1" 200 None 2026-03-27 08:52:55,401 - lava_callback - DEBUG - Processing node 69c3a254bbc022e7926448ff: baseline-arm64-baylibre 2026-03-27 08:52:55,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a254bbc022e7926448ff 2026-03-27 08:52:55,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:55,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a254bbc022e7926448ff HTTP/1.1" 200 None 2026-03-27 08:52:55,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:55,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:52:55,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:55,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a254bbc022e7926448ff HTTP/1.1" 200 None 2026-03-27 08:52:55,454 - lava_callback - ERROR - Error processing callback for node 69c3a254bbc022e7926448ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:52:56,024 - lava_callback - INFO - Received callback for node 69c3a256bbc022e792644902 INFO: 172.20.0.1:49338 - "POST /node/69c3a256bbc022e792644902 HTTP/1.0" 202 Accepted 2026-03-27 08:52:56,029 - lava_callback - INFO - Starting async job submit for node 69c3a256bbc022e792644902 2026-03-27 08:52:56,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:56,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a256bbc022e792644902 HTTP/1.1" 200 None 2026-03-27 08:52:56,045 - lava_callback - DEBUG - Processing node 69c3a256bbc022e792644902: baseline-arm64-baylibre 2026-03-27 08:52:56,045 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a256bbc022e792644902 2026-03-27 08:52:56,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:56,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a256bbc022e792644902 HTTP/1.1" 200 None 2026-03-27 08:52:56,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:56,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:52:56,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:56,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a256bbc022e792644902 HTTP/1.1" 200 None 2026-03-27 08:52:56,108 - lava_callback - ERROR - Error processing callback for node 69c3a256bbc022e792644902: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49350 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:52:56,684 - lava_callback - INFO - Received callback for node 69c3a257bbc022e792644905 INFO: 172.20.0.1:49366 - "POST /node/69c3a257bbc022e792644905 HTTP/1.0" 202 Accepted 2026-03-27 08:52:56,689 - lava_callback - INFO - Starting async job submit for node 69c3a257bbc022e792644905 2026-03-27 08:52:56,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:56,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a257bbc022e792644905 HTTP/1.1" 200 None 2026-03-27 08:52:56,704 - lava_callback - DEBUG - Processing node 69c3a257bbc022e792644905: baseline-arm64-baylibre 2026-03-27 08:52:56,705 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a257bbc022e792644905 2026-03-27 08:52:56,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:56,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a257bbc022e792644905 HTTP/1.1" 200 None 2026-03-27 08:52:56,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:56,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:52:56,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:52:56,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a257bbc022e792644905 HTTP/1.1" 200 None 2026-03-27 08:52:56,755 - lava_callback - ERROR - Error processing callback for node 69c3a257bbc022e792644905: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:53:00,973 - lava_callback - INFO - Received callback for node 69c3a268bbc022e792644936 INFO: 172.20.0.1:44182 - "POST /node/69c3a268bbc022e792644936 HTTP/1.0" 202 Accepted 2026-03-27 08:53:00,979 - lava_callback - INFO - Starting async job submit for node 69c3a268bbc022e792644936 2026-03-27 08:53:00,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:01,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a268bbc022e792644936 HTTP/1.1" 200 None 2026-03-27 08:53:01,002 - lava_callback - DEBUG - Processing node 69c3a268bbc022e792644936: baseline-arm64-clabbe 2026-03-27 08:53:01,003 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a268bbc022e792644936 2026-03-27 08:53:01,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:01,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a268bbc022e792644936 HTTP/1.1" 200 None 2026-03-27 08:53:01,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:01,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:53:01,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:01,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a268bbc022e792644936 HTTP/1.1" 200 None 2026-03-27 08:53:01,084 - lava_callback - ERROR - Error processing callback for node 69c3a268bbc022e792644936: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:53:01,653 - lava_callback - INFO - Received callback for node 69c3a26abbc022e792644939 INFO: 172.20.0.1:44186 - "POST /node/69c3a26abbc022e792644939 HTTP/1.0" 202 Accepted 2026-03-27 08:53:01,659 - lava_callback - INFO - Starting async job submit for node 69c3a26abbc022e792644939 2026-03-27 08:53:01,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:01,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a26abbc022e792644939 HTTP/1.1" 200 None 2026-03-27 08:53:01,675 - lava_callback - DEBUG - Processing node 69c3a26abbc022e792644939: baseline-arm64-clabbe 2026-03-27 08:53:01,675 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a26abbc022e792644939 2026-03-27 08:53:01,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:01,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a26abbc022e792644939 HTTP/1.1" 200 None 2026-03-27 08:53:01,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:01,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:53:01,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:01,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a26abbc022e792644939 HTTP/1.1" 200 None 2026-03-27 08:53:01,738 - lava_callback - ERROR - Error processing callback for node 69c3a26abbc022e792644939: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:53:02,332 - lava_callback - INFO - Received callback for node 69c3a26bbbc022e79264493c INFO: 172.20.0.1:44188 - "POST /node/69c3a26bbbc022e79264493c HTTP/1.0" 202 Accepted 2026-03-27 08:53:02,337 - lava_callback - INFO - Starting async job submit for node 69c3a26bbbc022e79264493c 2026-03-27 08:53:02,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:02,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a26bbbc022e79264493c HTTP/1.1" 200 None 2026-03-27 08:53:02,353 - lava_callback - DEBUG - Processing node 69c3a26bbbc022e79264493c: baseline-arm64-clabbe 2026-03-27 08:53:02,353 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a26bbbc022e79264493c 2026-03-27 08:53:02,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:02,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a26bbbc022e79264493c HTTP/1.1" 200 None 2026-03-27 08:53:02,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:02,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:53:02,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:02,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a26bbbc022e79264493c HTTP/1.1" 200 None 2026-03-27 08:53:02,412 - lava_callback - ERROR - Error processing callback for node 69c3a26bbbc022e79264493c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:53:02,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:02,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53674 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:53:20,940 - lava_callback - INFO - Received callback for node 69c3a26cbbc022e792644943 INFO: 172.20.0.1:53678 - "POST /node/69c3a26cbbc022e792644943 HTTP/1.0" 202 Accepted 2026-03-27 08:53:20,947 - lava_callback - INFO - Starting async job submit for node 69c3a26cbbc022e792644943 2026-03-27 08:53:20,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:20,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a26cbbc022e792644943 HTTP/1.1" 200 None 2026-03-27 08:53:20,965 - lava_callback - DEBUG - Processing node 69c3a26cbbc022e792644943: baseline-arm64-clabbe 2026-03-27 08:53:20,965 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a26cbbc022e792644943 2026-03-27 08:53:20,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:20,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a26cbbc022e792644943 HTTP/1.1" 200 None 2026-03-27 08:53:20,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:20,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:53:20,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:21,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a26cbbc022e792644943 HTTP/1.1" 200 None 2026-03-27 08:53:21,016 - lava_callback - ERROR - Error processing callback for node 69c3a26cbbc022e792644943: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:53:21,619 - lava_callback - INFO - Received callback for node 69c3a26ebbc022e79264494a INFO: 172.20.0.1:53694 - "POST /node/69c3a26ebbc022e79264494a HTTP/1.0" 202 Accepted 2026-03-27 08:53:21,624 - lava_callback - INFO - Starting async job submit for node 69c3a26ebbc022e79264494a 2026-03-27 08:53:21,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:21,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a26ebbc022e79264494a HTTP/1.1" 200 None 2026-03-27 08:53:21,640 - lava_callback - DEBUG - Processing node 69c3a26ebbc022e79264494a: baseline-arm64-clabbe 2026-03-27 08:53:21,640 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a26ebbc022e79264494a 2026-03-27 08:53:21,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:21,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a26ebbc022e79264494a HTTP/1.1" 200 None 2026-03-27 08:53:21,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:21,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:53:21,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:21,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a26ebbc022e79264494a HTTP/1.1" 200 None 2026-03-27 08:53:21,691 - lava_callback - ERROR - Error processing callback for node 69c3a26ebbc022e79264494a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:53:22,295 - lava_callback - INFO - Received callback for node 69c3a26ebbc022e79264494d INFO: 172.20.0.1:53698 - "POST /node/69c3a26ebbc022e79264494d HTTP/1.0" 202 Accepted 2026-03-27 08:53:22,301 - lava_callback - INFO - Starting async job submit for node 69c3a26ebbc022e79264494d 2026-03-27 08:53:22,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:22,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a26ebbc022e79264494d HTTP/1.1" 200 None 2026-03-27 08:53:22,317 - lava_callback - DEBUG - Processing node 69c3a26ebbc022e79264494d: baseline-arm64-clabbe 2026-03-27 08:53:22,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a26ebbc022e79264494d 2026-03-27 08:53:22,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:22,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a26ebbc022e79264494d HTTP/1.1" 200 None 2026-03-27 08:53:22,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:22,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 08:53:22,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:22,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a26ebbc022e79264494d HTTP/1.1" 200 None 2026-03-27 08:53:22,378 - lava_callback - ERROR - Error processing callback for node 69c3a26ebbc022e79264494d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53178 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:53:42,617 - lava_callback - INFO - Received callback for node 69c641e7b98d4d3c020685e1 INFO: 172.20.0.1:55690 - "POST /node/69c641e7b98d4d3c020685e1 HTTP/1.0" 202 Accepted 2026-03-27 08:53:42,624 - lava_callback - INFO - Starting async job submit for node 69c641e7b98d4d3c020685e1 2026-03-27 08:53:42,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:42,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e7b98d4d3c020685e1 HTTP/1.1" 200 None 2026-03-27 08:53:42,664 - lava_callback - DEBUG - Processing node 69c641e7b98d4d3c020685e1: ltp-timers 2026-03-27 08:53:43,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:53:43,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:53:43,168 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c641e7b98d4d3c020685e1/log.txt.gz 2026-03-27 08:53:43,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:53:43,581 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:53:43,583 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c641e7b98d4d3c020685e1/lava-logs.yaml 2026-03-27 08:53:43,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:53:43,757 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:53:43,758 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c641e7b98d4d3c020685e1/lava_callback.json.gz 2026-03-27 08:53:43,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:43,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e7b98d4d3c020685e1 HTTP/1.1" 200 None 2026-03-27 08:53:43,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:43,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:53:43,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:43,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641e7b98d4d3c020685e1 HTTP/1.1" 200 None 2026-03-27 08:53:43,842 - lava_callback - INFO - Completed processing callback for node 69c641e7b98d4d3c020685e1 2026-03-27 08:53:44,260 - lava_callback - INFO - Received callback for node 69c641e9b98d4d3c020685e4 INFO: 172.20.0.1:55704 - "POST /node/69c641e9b98d4d3c020685e4 HTTP/1.0" 202 Accepted 2026-03-27 08:53:44,267 - lava_callback - INFO - Starting async job submit for node 69c641e9b98d4d3c020685e4 2026-03-27 08:53:44,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:44,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e9b98d4d3c020685e4 HTTP/1.1" 200 None 2026-03-27 08:53:44,310 - lava_callback - DEBUG - Processing node 69c641e9b98d4d3c020685e4: ltp-watchqueue 2026-03-27 08:53:44,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:53:44,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:53:44,813 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c641e9b98d4d3c020685e4/log.txt.gz 2026-03-27 08:53:45,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:53:45,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:53:45,381 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c641e9b98d4d3c020685e4/lava-logs.yaml 2026-03-27 08:53:45,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:53:45,709 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:53:45,710 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c641e9b98d4d3c020685e4/lava_callback.json.gz 2026-03-27 08:53:45,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:45,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641e9b98d4d3c020685e4 HTTP/1.1" 200 None 2026-03-27 08:53:45,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:45,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:53:45,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:45,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641e9b98d4d3c020685e4 HTTP/1.1" 200 None 2026-03-27 08:53:45,807 - lava_callback - INFO - Completed processing callback for node 69c641e9b98d4d3c020685e4 INFO: 172.20.0.1:55712 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:53:47,096 - lava_callback - INFO - Received callback for node 69c6419eb98d4d3c02068501 INFO: 172.20.0.1:55720 - "POST /node/69c6419eb98d4d3c02068501 HTTP/1.0" 202 Accepted 2026-03-27 08:53:47,104 - lava_callback - INFO - Starting async job submit for node 69c6419eb98d4d3c02068501 2026-03-27 08:53:47,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:47,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6419eb98d4d3c02068501 HTTP/1.1" 200 None 2026-03-27 08:53:47,235 - lava_callback - DEBUG - Processing node 69c6419eb98d4d3c02068501: baseline-arm 2026-03-27 08:53:47,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:53:47,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:53:47,996 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419eb98d4d3c02068501/log.txt.gz 2026-03-27 08:53:48,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:53:48,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:53:48,577 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419eb98d4d3c02068501/lava-logs.yaml 2026-03-27 08:53:48,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:53:48,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:53:48,745 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c6419eb98d4d3c02068501/lava_callback.json.gz DEBUG: Setup failed for 69c6419eb98d4d3c02068501. Transitting job node result: incomplete -> fail 2026-03-27 08:53:48,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:48,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6419eb98d4d3c02068501 HTTP/1.1" 200 None 2026-03-27 08:53:48,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:48,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:53:48,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:53:48,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6419eb98d4d3c02068501 HTTP/1.1" 200 None 2026-03-27 08:53:49,045 - lava_callback - INFO - Completed processing callback for node 69c6419eb98d4d3c02068501 INFO: 172.20.0.1:43352 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:54:01,036 - lava_callback - INFO - Received callback for node 69c3a294bbc022e792644996 INFO: 172.20.0.1:56730 - "POST /node/69c3a294bbc022e792644996 HTTP/1.0" 202 Accepted 2026-03-27 08:54:01,042 - lava_callback - INFO - Starting async job submit for node 69c3a294bbc022e792644996 2026-03-27 08:54:01,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:54:01,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a294bbc022e792644996 HTTP/1.1" 200 None 2026-03-27 08:54:01,058 - lava_callback - DEBUG - Processing node 69c3a294bbc022e792644996: baseline-arm-clabbe 2026-03-27 08:54:01,059 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a294bbc022e792644996 2026-03-27 08:54:01,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:54:01,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a294bbc022e792644996 HTTP/1.1" 200 None 2026-03-27 08:54:01,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:54:01,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 08:54:01,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:54:01,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a294bbc022e792644996 HTTP/1.1" 200 None 2026-03-27 08:54:01,114 - lava_callback - ERROR - Error processing callback for node 69c3a294bbc022e792644996: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 08:54:02,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:54:02,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54594 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:54:41,029 - lava_callback - INFO - Received callback for node 69c3a2bcbbc022e792644a22 INFO: 172.20.0.1:44460 - "POST /node/69c3a2bcbbc022e792644a22 HTTP/1.0" 202 Accepted 2026-03-27 08:54:41,037 - lava_callback - INFO - Starting async job submit for node 69c3a2bcbbc022e792644a22 2026-03-27 08:54:41,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:54:41,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a2bcbbc022e792644a22 HTTP/1.1" 200 None 2026-03-27 08:54:41,059 - lava_callback - DEBUG - Processing node 69c3a2bcbbc022e792644a22: baseline-arm-clabbe 2026-03-27 08:54:41,059 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a2bcbbc022e792644a22 2026-03-27 08:54:41,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:54:41,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a2bcbbc022e792644a22 HTTP/1.1" 200 None 2026-03-27 08:54:41,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:54:41,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 08:54:41,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:54:41,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a2bcbbc022e792644a22 HTTP/1.1" 200 None 2026-03-27 08:54:41,136 - lava_callback - ERROR - Error processing callback for node 69c3a2bcbbc022e792644a22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59296 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:55:01,055 - lava_callback - INFO - Received callback for node 69c3a2e0bbc022e792644a7d INFO: 172.20.0.1:51378 - "POST /node/69c3a2e0bbc022e792644a7d HTTP/1.0" 202 Accepted 2026-03-27 08:55:01,064 - lava_callback - INFO - Starting async job submit for node 69c3a2e0bbc022e792644a7d 2026-03-27 08:55:01,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:55:01,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a2e0bbc022e792644a7d HTTP/1.1" 200 None 2026-03-27 08:55:01,095 - lava_callback - DEBUG - Processing node 69c3a2e0bbc022e792644a7d: baseline-arm-clabbe 2026-03-27 08:55:01,096 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a2e0bbc022e792644a7d 2026-03-27 08:55:01,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:55:01,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a2e0bbc022e792644a7d HTTP/1.1" 200 None 2026-03-27 08:55:01,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:55:01,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 08:55:01,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:55:01,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a2e0bbc022e792644a7d HTTP/1.1" 200 None 2026-03-27 08:55:01,202 - lava_callback - ERROR - Error processing callback for node 69c3a2e0bbc022e792644a7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59618 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:55:34,163 - lava_callback - INFO - Received callback for node 69c3a2ffbbc022e792644a98 INFO: 172.20.0.1:54548 - "POST /node/69c3a2ffbbc022e792644a98 HTTP/1.0" 202 Accepted 2026-03-27 08:55:34,172 - lava_callback - INFO - Starting async job submit for node 69c3a2ffbbc022e792644a98 2026-03-27 08:55:34,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:55:34,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a2ffbbc022e792644a98 HTTP/1.1" 200 None 2026-03-27 08:55:34,195 - lava_callback - DEBUG - Processing node 69c3a2ffbbc022e792644a98: baseline-arm-baylibre 2026-03-27 08:55:34,196 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a2ffbbc022e792644a98 2026-03-27 08:55:34,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:55:34,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a2ffbbc022e792644a98 HTTP/1.1" 200 None 2026-03-27 08:55:34,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:55:34,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 08:55:34,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:55:34,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a2ffbbc022e792644a98 HTTP/1.1" 200 None 2026-03-27 08:55:34,254 - lava_callback - ERROR - Error processing callback for node 69c3a2ffbbc022e792644a98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41658 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:56:14,244 - lava_callback - INFO - Received callback for node 69c3a324bbc022e792644aac INFO: 172.20.0.1:57386 - "POST /node/69c3a324bbc022e792644aac HTTP/1.0" 202 Accepted 2026-03-27 08:56:14,250 - lava_callback - INFO - Starting async job submit for node 69c3a324bbc022e792644aac 2026-03-27 08:56:14,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:14,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a324bbc022e792644aac HTTP/1.1" 200 None 2026-03-27 08:56:14,269 - lava_callback - DEBUG - Processing node 69c3a324bbc022e792644aac: baseline-arm-baylibre 2026-03-27 08:56:14,269 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a324bbc022e792644aac 2026-03-27 08:56:14,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:14,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a324bbc022e792644aac HTTP/1.1" 200 None 2026-03-27 08:56:14,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:14,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 08:56:14,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:14,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a324bbc022e792644aac HTTP/1.1" 200 None 2026-03-27 08:56:14,332 - lava_callback - ERROR - Error processing callback for node 69c3a324bbc022e792644aac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38456 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:56:21,664 - lava_callback - INFO - Received callback for node 69c645f6b98d4d3c02068839 INFO: 172.20.0.1:38458 - "POST /node/69c645f6b98d4d3c02068839 HTTP/1.0" 202 Accepted 2026-03-27 08:56:21,670 - lava_callback - INFO - Starting async job submit for node 69c645f6b98d4d3c02068839 2026-03-27 08:56:21,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:21,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c645f6b98d4d3c02068839 HTTP/1.1" 200 None 2026-03-27 08:56:21,706 - lava_callback - DEBUG - Processing node 69c645f6b98d4d3c02068839: baseline-arm-pengutronix 2026-03-27 08:56:21,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:56:22,025 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:56:22,027 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c645f6b98d4d3c02068839/log.txt.gz 2026-03-27 08:56:22,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:56:22,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:56:22,373 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c645f6b98d4d3c02068839/lava-logs.yaml 2026-03-27 08:56:22,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:56:22,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:56:22,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c645f6b98d4d3c02068839/lava_callback.json.gz 2026-03-27 08:56:22,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:22,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c645f6b98d4d3c02068839 HTTP/1.1" 200 None 2026-03-27 08:56:22,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:22,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:56:22,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:22,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c645f6b98d4d3c02068839 HTTP/1.1" 200 None 2026-03-27 08:56:22,627 - lava_callback - INFO - Completed processing callback for node 69c645f6b98d4d3c02068839 INFO: 172.20.0.1:38462 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:56:32,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:32,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59480 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:56:51,369 - lava_callback - INFO - Received callback for node 69c641bfb98d4d3c02068591 INFO: 172.20.0.1:38174 - "POST /node/69c641bfb98d4d3c02068591 HTTP/1.0" 202 Accepted 2026-03-27 08:56:51,417 - lava_callback - INFO - Starting async job submit for node 69c641bfb98d4d3c02068591 2026-03-27 08:56:51,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:51,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641bfb98d4d3c02068591 HTTP/1.1" 200 None 2026-03-27 08:56:51,456 - lava_callback - DEBUG - Processing node 69c641bfb98d4d3c02068591: baseline-arm-pengutronix 2026-03-27 08:56:51,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:56:51,698 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:56:51,700 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641bfb98d4d3c02068591/log.txt.gz 2026-03-27 08:56:51,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:56:51,942 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:56:51,944 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641bfb98d4d3c02068591/lava-logs.yaml 2026-03-27 08:56:51,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 08:56:52,057 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 08:56:52,059 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c641bfb98d4d3c02068591/lava_callback.json.gz DEBUG: Setup failed for 69c641bfb98d4d3c02068591. Transitting job node result: incomplete -> fail 2026-03-27 08:56:52,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:52,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c641bfb98d4d3c02068591 HTTP/1.1" 200 None 2026-03-27 08:56:52,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:52,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 08:56:52,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:52,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c641bfb98d4d3c02068591 HTTP/1.1" 200 None 2026-03-27 08:56:52,217 - lava_callback - INFO - Completed processing callback for node 69c641bfb98d4d3c02068591 2026-03-27 08:56:54,250 - lava_callback - INFO - Received callback for node 69c3a349bbc022e792644ac8 INFO: 172.20.0.1:38178 - "POST /node/69c3a349bbc022e792644ac8 HTTP/1.0" 202 Accepted 2026-03-27 08:56:54,255 - lava_callback - INFO - Starting async job submit for node 69c3a349bbc022e792644ac8 2026-03-27 08:56:54,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:54,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a349bbc022e792644ac8 HTTP/1.1" 200 None 2026-03-27 08:56:54,271 - lava_callback - DEBUG - Processing node 69c3a349bbc022e792644ac8: baseline-arm-baylibre 2026-03-27 08:56:54,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a349bbc022e792644ac8 2026-03-27 08:56:54,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:54,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a349bbc022e792644ac8 HTTP/1.1" 200 None 2026-03-27 08:56:54,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:54,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 08:56:54,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:56:54,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a349bbc022e792644ac8 HTTP/1.1" 200 None 2026-03-27 08:56:54,331 - lava_callback - ERROR - Error processing callback for node 69c3a349bbc022e792644ac8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38182 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:57:02,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:57:02,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50834 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:57:34,256 - lava_callback - INFO - Received callback for node 69c3a36dbbc022e792644ae4 INFO: 172.20.0.1:44686 - "POST /node/69c3a36dbbc022e792644ae4 HTTP/1.0" 202 Accepted 2026-03-27 08:57:34,262 - lava_callback - INFO - Starting async job submit for node 69c3a36dbbc022e792644ae4 2026-03-27 08:57:34,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:57:34,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a36dbbc022e792644ae4 HTTP/1.1" 200 None 2026-03-27 08:57:34,291 - lava_callback - DEBUG - Processing node 69c3a36dbbc022e792644ae4: baseline-arm-baylibre 2026-03-27 08:57:34,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a36dbbc022e792644ae4 2026-03-27 08:57:34,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:57:34,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a36dbbc022e792644ae4 HTTP/1.1" 200 None 2026-03-27 08:57:34,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:57:34,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 08:57:34,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:57:34,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a36dbbc022e792644ae4 HTTP/1.1" 200 None 2026-03-27 08:57:34,364 - lava_callback - ERROR - Error processing callback for node 69c3a36dbbc022e792644ae4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33368 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:58:01,245 - lava_callback - INFO - Received callback for node 69c3a394bbc022e792644af2 INFO: 172.20.0.1:51656 - "POST /node/69c3a394bbc022e792644af2 HTTP/1.0" 202 Accepted 2026-03-27 08:58:01,250 - lava_callback - INFO - Starting async job submit for node 69c3a394bbc022e792644af2 2026-03-27 08:58:01,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:58:01,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a394bbc022e792644af2 HTTP/1.1" 200 None 2026-03-27 08:58:01,275 - lava_callback - DEBUG - Processing node 69c3a394bbc022e792644af2: baseline-arm-clabbe 2026-03-27 08:58:01,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a394bbc022e792644af2 2026-03-27 08:58:01,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:58:01,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a394bbc022e792644af2 HTTP/1.1" 200 None 2026-03-27 08:58:01,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:58:01,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 08:58:01,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:58:01,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a394bbc022e792644af2 HTTP/1.1" 200 None 2026-03-27 08:58:01,339 - lava_callback - ERROR - Error processing callback for node 69c3a394bbc022e792644af2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41716 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:58:34,341 - lava_callback - INFO - Received callback for node 69c3a3b6bbc022e792644b04 INFO: 172.20.0.1:49390 - "POST /node/69c3a3b6bbc022e792644b04 HTTP/1.0" 202 Accepted 2026-03-27 08:58:34,347 - lava_callback - INFO - Starting async job submit for node 69c3a3b6bbc022e792644b04 2026-03-27 08:58:34,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:58:34,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a3b6bbc022e792644b04 HTTP/1.1" 200 None 2026-03-27 08:58:34,364 - lava_callback - DEBUG - Processing node 69c3a3b6bbc022e792644b04: baseline-arm-baylibre 2026-03-27 08:58:34,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a3b6bbc022e792644b04 2026-03-27 08:58:34,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:58:34,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a3b6bbc022e792644b04 HTTP/1.1" 200 None 2026-03-27 08:58:34,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:58:34,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 08:58:34,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:58:34,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a3b6bbc022e792644b04 HTTP/1.1" 200 None 2026-03-27 08:58:34,430 - lava_callback - ERROR - Error processing callback for node 69c3a3b6bbc022e792644b04: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43318 - "GET / HTTP/1.0" 200 OK 2026-03-27 08:59:14,354 - lava_callback - INFO - Received callback for node 69c3a3d9bbc022e792644b14 INFO: 172.20.0.1:37580 - "POST /node/69c3a3d9bbc022e792644b14 HTTP/1.0" 202 Accepted 2026-03-27 08:59:14,359 - lava_callback - INFO - Starting async job submit for node 69c3a3d9bbc022e792644b14 2026-03-27 08:59:14,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:59:14,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a3d9bbc022e792644b14 HTTP/1.1" 200 None 2026-03-27 08:59:14,376 - lava_callback - DEBUG - Processing node 69c3a3d9bbc022e792644b14: baseline-arm-baylibre 2026-03-27 08:59:14,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a3d9bbc022e792644b14 2026-03-27 08:59:14,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:59:14,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a3d9bbc022e792644b14 HTTP/1.1" 200 None 2026-03-27 08:59:14,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:59:14,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 08:59:14,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 08:59:14,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a3d9bbc022e792644b14 HTTP/1.1" 200 None 2026-03-27 08:59:14,438 - lava_callback - ERROR - Error processing callback for node 69c3a3d9bbc022e792644b14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39212 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:00:01,344 - lava_callback - INFO - Received callback for node 69c3a3febbc022e792644b1f INFO: 172.20.0.1:60982 - "POST /node/69c3a3febbc022e792644b1f HTTP/1.0" 202 Accepted 2026-03-27 09:00:01,350 - lava_callback - INFO - Starting async job submit for node 69c3a3febbc022e792644b1f 2026-03-27 09:00:01,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:00:01,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a3febbc022e792644b1f HTTP/1.1" 200 None 2026-03-27 09:00:01,367 - lava_callback - DEBUG - Processing node 69c3a3febbc022e792644b1f: baseline-arm-clabbe 2026-03-27 09:00:01,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a3febbc022e792644b1f 2026-03-27 09:00:01,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:00:01,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a3febbc022e792644b1f HTTP/1.1" 200 None 2026-03-27 09:00:01,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:00:01,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 09:00:01,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:00:01,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a3febbc022e792644b1f HTTP/1.1" 200 None 2026-03-27 09:00:01,458 - lava_callback - ERROR - Error processing callback for node 69c3a3febbc022e792644b1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39514 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:00:41,337 - lava_callback - INFO - Received callback for node 69c3a423bbc022e792644b2a INFO: 172.20.0.1:58872 - "POST /node/69c3a423bbc022e792644b2a HTTP/1.0" 202 Accepted 2026-03-27 09:00:41,343 - lava_callback - INFO - Starting async job submit for node 69c3a423bbc022e792644b2a 2026-03-27 09:00:41,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:00:41,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a423bbc022e792644b2a HTTP/1.1" 200 None 2026-03-27 09:00:41,358 - lava_callback - DEBUG - Processing node 69c3a423bbc022e792644b2a: baseline-arm-clabbe 2026-03-27 09:00:41,359 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a423bbc022e792644b2a 2026-03-27 09:00:41,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:00:41,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a423bbc022e792644b2a HTTP/1.1" 200 None 2026-03-27 09:00:41,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:00:41,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 09:00:41,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:00:41,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a423bbc022e792644b2a HTTP/1.1" 200 None 2026-03-27 09:00:41,417 - lava_callback - ERROR - Error processing callback for node 69c3a423bbc022e792644b2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:01:01,383 - lava_callback - INFO - Received callback for node 69c3a447bbc022e792644b42 INFO: 172.20.0.1:56308 - "POST /node/69c3a447bbc022e792644b42 HTTP/1.0" 202 Accepted 2026-03-27 09:01:01,389 - lava_callback - INFO - Starting async job submit for node 69c3a447bbc022e792644b42 2026-03-27 09:01:01,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:01:01,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a447bbc022e792644b42 HTTP/1.1" 200 None 2026-03-27 09:01:01,405 - lava_callback - DEBUG - Processing node 69c3a447bbc022e792644b42: baseline-arm-clabbe 2026-03-27 09:01:01,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a447bbc022e792644b42 2026-03-27 09:01:01,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:01:01,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a447bbc022e792644b42 HTTP/1.1" 200 None 2026-03-27 09:01:01,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:01:01,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 09:01:01,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:01:01,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a447bbc022e792644b42 HTTP/1.1" 200 None 2026-03-27 09:01:01,468 - lava_callback - ERROR - Error processing callback for node 69c3a447bbc022e792644b42: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38752 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:01:41,411 - lava_callback - INFO - Received callback for node 69c3a46bbbc022e792644b4d INFO: 172.20.0.1:38368 - "POST /node/69c3a46bbbc022e792644b4d HTTP/1.0" 202 Accepted 2026-03-27 09:01:41,416 - lava_callback - INFO - Starting async job submit for node 69c3a46bbbc022e792644b4d 2026-03-27 09:01:41,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:01:41,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a46bbbc022e792644b4d HTTP/1.1" 200 None 2026-03-27 09:01:41,434 - lava_callback - DEBUG - Processing node 69c3a46bbbc022e792644b4d: baseline-arm-clabbe 2026-03-27 09:01:41,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a46bbbc022e792644b4d 2026-03-27 09:01:41,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:01:41,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a46bbbc022e792644b4d HTTP/1.1" 200 None 2026-03-27 09:01:41,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:01:41,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 09:01:41,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:01:41,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a46bbbc022e792644b4d HTTP/1.1" 200 None 2026-03-27 09:01:41,523 - lava_callback - ERROR - Error processing callback for node 69c3a46bbbc022e792644b4d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37142 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:02:19,772 - lava_callback - INFO - Received callback for node 69c5ed592cb62322ad9690ac INFO: 172.20.0.1:33884 - "POST /node/69c5ed592cb62322ad9690ac HTTP/1.0" 202 Accepted 2026-03-27 09:02:19,779 - lava_callback - INFO - Starting async job submit for node 69c5ed592cb62322ad9690ac 2026-03-27 09:02:19,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:02:19,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ed592cb62322ad9690ac HTTP/1.1" 200 None 2026-03-27 09:02:19,822 - lava_callback - DEBUG - Processing node 69c5ed592cb62322ad9690ac: baseline-arm64 2026-03-27 09:02:20,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:02:20,497 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:02:20,499 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5ed592cb62322ad9690ac/log.txt.gz 2026-03-27 09:02:20,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:33898 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:02:21,166 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:02:21,168 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5ed592cb62322ad9690ac/lava-logs.yaml 2026-03-27 09:02:21,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:02:21,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:02:21,354 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5ed592cb62322ad9690ac/lava_callback.json.gz 2026-03-27 09:02:21,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:02:21,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ed592cb62322ad9690ac HTTP/1.1" 200 None 2026-03-27 09:02:21,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:02:21,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 09:02:21,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:02:21,437 - lava_callback - INFO - Received callback for node 69c3a490bbc022e792644b5f INFO: 172.20.0.1:33902 - "POST /node/69c3a490bbc022e792644b5f HTTP/1.0" 202 Accepted 2026-03-27 09:02:21,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ed592cb62322ad9690ac HTTP/1.1" 200 None 2026-03-27 09:02:21,448 - lava_callback - INFO - Starting async job submit for node 69c3a490bbc022e792644b5f 2026-03-27 09:02:21,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:02:21,540 - lava_callback - INFO - Completed processing callback for node 69c5ed592cb62322ad9690ac 2026-03-27 09:02:21,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a490bbc022e792644b5f HTTP/1.1" 200 None 2026-03-27 09:02:21,550 - lava_callback - DEBUG - Processing node 69c3a490bbc022e792644b5f: baseline-arm-clabbe 2026-03-27 09:02:21,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a490bbc022e792644b5f 2026-03-27 09:02:21,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:02:21,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a490bbc022e792644b5f HTTP/1.1" 200 None 2026-03-27 09:02:21,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:02:21,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 09:02:21,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:02:21,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a490bbc022e792644b5f HTTP/1.1" 200 None 2026-03-27 09:02:21,623 - lava_callback - ERROR - Error processing callback for node 69c3a490bbc022e792644b5f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33904 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:02:32,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:02:32,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42634 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:03:01,451 - lava_callback - INFO - Received callback for node 69c3a4b4bbc022e792644b6a INFO: 172.20.0.1:50890 - "POST /node/69c3a4b4bbc022e792644b6a HTTP/1.0" 202 Accepted 2026-03-27 09:03:01,458 - lava_callback - INFO - Starting async job submit for node 69c3a4b4bbc022e792644b6a 2026-03-27 09:03:01,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:03:01,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a4b4bbc022e792644b6a HTTP/1.1" 200 None 2026-03-27 09:03:01,477 - lava_callback - DEBUG - Processing node 69c3a4b4bbc022e792644b6a: baseline-arm-clabbe 2026-03-27 09:03:01,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a4b4bbc022e792644b6a 2026-03-27 09:03:01,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:03:01,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a4b4bbc022e792644b6a HTTP/1.1" 200 None 2026-03-27 09:03:01,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:03:01,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 09:03:01,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:03:01,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a4b4bbc022e792644b6a HTTP/1.1" 200 None 2026-03-27 09:03:01,536 - lava_callback - ERROR - Error processing callback for node 69c3a4b4bbc022e792644b6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33336 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:04:54,670 - lava_callback - INFO - Received callback for node 69c3a526bbc022e792644b94 INFO: 172.20.0.1:34444 - "POST /node/69c3a526bbc022e792644b94 HTTP/1.0" 202 Accepted 2026-03-27 09:04:54,676 - lava_callback - INFO - Starting async job submit for node 69c3a526bbc022e792644b94 2026-03-27 09:04:54,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:04:54,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a526bbc022e792644b94 HTTP/1.1" 200 None 2026-03-27 09:04:54,696 - lava_callback - DEBUG - Processing node 69c3a526bbc022e792644b94: baseline-arm64-baylibre 2026-03-27 09:04:54,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a526bbc022e792644b94 2026-03-27 09:04:54,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:04:54,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a526bbc022e792644b94 HTTP/1.1" 200 None 2026-03-27 09:04:54,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:04:54,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:04:54,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:04:54,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a526bbc022e792644b94 HTTP/1.1" 200 None 2026-03-27 09:04:54,754 - lava_callback - ERROR - Error processing callback for node 69c3a526bbc022e792644b94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47662 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:05:35,691 - lava_callback - INFO - Received callback for node 69c64856b98d4d3c020688e6 INFO: 172.20.0.1:33728 - "POST /node/69c64856b98d4d3c020688e6 HTTP/1.0" 202 Accepted 2026-03-27 09:05:35,698 - lava_callback - INFO - Starting async job submit for node 69c64856b98d4d3c020688e6 2026-03-27 09:05:35,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:05:35,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64856b98d4d3c020688e6 HTTP/1.1" 200 None 2026-03-27 09:05:35,717 - lava_callback - DEBUG - Processing node 69c64856b98d4d3c020688e6: baseline-arm 2026-03-27 09:05:35,722 - lava_callback - WARNING - Failed to upload log 2026-03-27 09:05:35,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:05:35,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:05:35,870 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c64856b98d4d3c020688e6/lava-logs.yaml 2026-03-27 09:05:35,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:05:35,986 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:05:35,988 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c64856b98d4d3c020688e6/lava_callback.json.gz 2026-03-27 09:05:35,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:05:36,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64856b98d4d3c020688e6 HTTP/1.1" 200 None 2026-03-27 09:05:36,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:05:36,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 09:05:36,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:05:36,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64856b98d4d3c020688e6 HTTP/1.1" 200 None 2026-03-27 09:05:36,054 - lava_callback - INFO - Completed processing callback for node 69c64856b98d4d3c020688e6 INFO: 172.20.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:06:02,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:06:02,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36428 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:06:14,782 - lava_callback - INFO - Received callback for node 69c3a576bbc022e792644ba1 INFO: 172.20.0.1:51700 - "POST /node/69c3a576bbc022e792644ba1 HTTP/1.0" 202 Accepted 2026-03-27 09:06:14,787 - lava_callback - INFO - Starting async job submit for node 69c3a576bbc022e792644ba1 2026-03-27 09:06:14,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:06:14,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a576bbc022e792644ba1 HTTP/1.1" 200 None 2026-03-27 09:06:14,803 - lava_callback - DEBUG - Processing node 69c3a576bbc022e792644ba1: baseline-arm64-baylibre 2026-03-27 09:06:14,803 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a576bbc022e792644ba1 2026-03-27 09:06:14,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:06:14,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a576bbc022e792644ba1 HTTP/1.1" 200 None 2026-03-27 09:06:14,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:06:14,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:06:14,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:06:14,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a576bbc022e792644ba1 HTTP/1.1" 200 None 2026-03-27 09:06:14,859 - lava_callback - ERROR - Error processing callback for node 69c3a576bbc022e792644ba1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54754 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:08:54,902 - lava_callback - INFO - Received callback for node 69c3a611bbc022e792644bc9 INFO: 172.20.0.1:46430 - "POST /node/69c3a611bbc022e792644bc9 HTTP/1.0" 202 Accepted 2026-03-27 09:08:54,908 - lava_callback - INFO - Starting async job submit for node 69c3a611bbc022e792644bc9 2026-03-27 09:08:54,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:08:54,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a611bbc022e792644bc9 HTTP/1.1" 200 None 2026-03-27 09:08:54,925 - lava_callback - DEBUG - Processing node 69c3a611bbc022e792644bc9: baseline-arm64-baylibre 2026-03-27 09:08:54,926 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a611bbc022e792644bc9 2026-03-27 09:08:54,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:08:54,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a611bbc022e792644bc9 HTTP/1.1" 200 None 2026-03-27 09:08:54,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:08:54,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:08:54,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:08:55,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a611bbc022e792644bc9 HTTP/1.1" 200 None 2026-03-27 09:08:55,026 - lava_callback - ERROR - Error processing callback for node 69c3a611bbc022e792644bc9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59406 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:09:54,901 - lava_callback - INFO - Received callback for node 69c3a65ebbc022e792644bd7 INFO: 172.20.0.1:60464 - "POST /node/69c3a65ebbc022e792644bd7 HTTP/1.0" 202 Accepted 2026-03-27 09:09:54,907 - lava_callback - INFO - Starting async job submit for node 69c3a65ebbc022e792644bd7 2026-03-27 09:09:54,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:09:54,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a65ebbc022e792644bd7 HTTP/1.1" 200 None 2026-03-27 09:09:54,963 - lava_callback - DEBUG - Processing node 69c3a65ebbc022e792644bd7: baseline-arm64-baylibre 2026-03-27 09:09:54,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a65ebbc022e792644bd7 2026-03-27 09:09:54,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:09:54,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a65ebbc022e792644bd7 HTTP/1.1" 200 None 2026-03-27 09:09:55,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:09:55,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:09:55,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:09:55,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a65ebbc022e792644bd7 HTTP/1.1" 200 None 2026-03-27 09:09:55,049 - lava_callback - ERROR - Error processing callback for node 69c3a65ebbc022e792644bd7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50792 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:10:09,325 - lava_callback - INFO - Received callback for node 69c6495bb98d4d3c0206892c INFO: 172.20.0.1:50804 - "POST /node/69c6495bb98d4d3c0206892c HTTP/1.0" 202 Accepted 2026-03-27 09:10:09,332 - lava_callback - INFO - Starting async job submit for node 69c6495bb98d4d3c0206892c 2026-03-27 09:10:09,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:09,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6495bb98d4d3c0206892c HTTP/1.1" 200 None 2026-03-27 09:10:09,383 - lava_callback - DEBUG - Processing node 69c6495bb98d4d3c0206892c: baseline-arm64 2026-03-27 09:10:09,393 - lava_callback - WARNING - Failed to upload log 2026-03-27 09:10:09,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:10:09,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:10:09,537 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495bb98d4d3c0206892c/lava-logs.yaml 2026-03-27 09:10:09,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:10:09,645 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:10:09,646 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495bb98d4d3c0206892c/lava_callback.json.gz 2026-03-27 09:10:09,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:09,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6495bb98d4d3c0206892c HTTP/1.1" 200 None 2026-03-27 09:10:09,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:09,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:10:09,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:09,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6495bb98d4d3c0206892c HTTP/1.1" 200 None 2026-03-27 09:10:09,712 - lava_callback - INFO - Completed processing callback for node 69c6495bb98d4d3c0206892c INFO: 172.20.0.1:49300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40468 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:10:32,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:32,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46522 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:10:51,236 - lava_callback - INFO - Received callback for node 69c64964b98d4d3c02068946 INFO: 172.20.0.1:38970 - "POST /node/69c64964b98d4d3c02068946 HTTP/1.0" 202 Accepted 2026-03-27 09:10:51,243 - lava_callback - INFO - Starting async job submit for node 69c64964b98d4d3c02068946 2026-03-27 09:10:51,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:51,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64964b98d4d3c02068946 HTTP/1.1" 200 None 2026-03-27 09:10:51,280 - lava_callback - DEBUG - Processing node 69c64964b98d4d3c02068946: baseline-arm64 2026-03-27 09:10:51,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:10:51,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:10:51,782 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64964b98d4d3c02068946/log.txt.gz 2026-03-27 09:10:52,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:10:52,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:10:52,270 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64964b98d4d3c02068946/lava-logs.yaml 2026-03-27 09:10:52,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:10:52,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:10:52,425 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64964b98d4d3c02068946/lava_callback.json.gz 2026-03-27 09:10:52,456 - lava_callback - INFO - Received callback for node 69c64979b98d4d3c02068993 INFO: 172.20.0.1:38974 - "POST /node/69c64979b98d4d3c02068993 HTTP/1.0" 202 Accepted 2026-03-27 09:10:52,484 - lava_callback - INFO - Starting async job submit for node 69c64979b98d4d3c02068993 2026-03-27 09:10:52,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:52,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:52,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64964b98d4d3c02068946 HTTP/1.1" 200 None 2026-03-27 09:10:52,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:52,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64979b98d4d3c02068993 HTTP/1.1" 200 None 2026-03-27 09:10:52,554 - lava_callback - DEBUG - Processing node 69c64979b98d4d3c02068993: baseline-nfs-arm64 2026-03-27 09:10:52,567 - lava_callback - WARNING - Failed to upload log 2026-03-27 09:10:52,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:10:52,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:10:52,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:52,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64964b98d4d3c02068946 HTTP/1.1" 200 None 2026-03-27 09:10:52,685 - lava_callback - INFO - Completed processing callback for node 69c64964b98d4d3c02068946 2026-03-27 09:10:52,888 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:10:52,890 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64979b98d4d3c02068993/lava-logs.yaml 2026-03-27 09:10:52,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:10:53,020 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:10:53,022 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64979b98d4d3c02068993/lava_callback.json.gz 2026-03-27 09:10:53,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:53,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64979b98d4d3c02068993 HTTP/1.1" 200 None 2026-03-27 09:10:53,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:53,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:10:53,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:10:53,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64979b98d4d3c02068993 HTTP/1.1" 200 None 2026-03-27 09:10:53,106 - lava_callback - INFO - Completed processing callback for node 69c64979b98d4d3c02068993 INFO: 172.20.0.1:38976 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:11:00,605 - lava_callback - INFO - Received callback for node 69c6495cb98d4d3c02068932 INFO: 172.20.0.1:36552 - "POST /node/69c6495cb98d4d3c02068932 HTTP/1.0" 202 Accepted 2026-03-27 09:11:00,612 - lava_callback - INFO - Starting async job submit for node 69c6495cb98d4d3c02068932 2026-03-27 09:11:00,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:00,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6495cb98d4d3c02068932 HTTP/1.1" 200 None 2026-03-27 09:11:00,651 - lava_callback - DEBUG - Processing node 69c6495cb98d4d3c02068932: baseline-arm64 2026-03-27 09:11:00,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:01,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:01,116 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495cb98d4d3c02068932/log.txt.gz 2026-03-27 09:11:01,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:01,462 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:01,464 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495cb98d4d3c02068932/lava-logs.yaml 2026-03-27 09:11:01,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:01,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:01,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495cb98d4d3c02068932/lava_callback.json.gz 2026-03-27 09:11:01,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:01,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6495cb98d4d3c02068932 HTTP/1.1" 200 None 2026-03-27 09:11:01,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:01,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:01,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:01,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6495cb98d4d3c02068932 HTTP/1.1" 200 None 2026-03-27 09:11:01,759 - lava_callback - INFO - Completed processing callback for node 69c6495cb98d4d3c02068932 2026-03-27 09:11:02,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:02,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36556 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:11:06,664 - lava_callback - INFO - Received callback for node 69c64968b98d4d3c0206894f INFO: 172.20.0.1:36572 - "POST /node/69c64968b98d4d3c0206894f HTTP/1.0" 202 Accepted 2026-03-27 09:11:06,671 - lava_callback - INFO - Starting async job submit for node 69c64968b98d4d3c0206894f 2026-03-27 09:11:06,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:06,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64968b98d4d3c0206894f HTTP/1.1" 200 None 2026-03-27 09:11:06,706 - lava_callback - DEBUG - Processing node 69c64968b98d4d3c0206894f: baseline-arm64 2026-03-27 09:11:06,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:06,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:06,983 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64968b98d4d3c0206894f/log.txt.gz 2026-03-27 09:11:07,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:07,270 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:07,272 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64968b98d4d3c0206894f/lava-logs.yaml 2026-03-27 09:11:07,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:07,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:07,408 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64968b98d4d3c0206894f/lava_callback.json.gz 2026-03-27 09:11:07,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:07,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64968b98d4d3c0206894f HTTP/1.1" 200 None 2026-03-27 09:11:07,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:07,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:07,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:07,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64968b98d4d3c0206894f HTTP/1.1" 200 None 2026-03-27 09:11:07,487 - lava_callback - INFO - Completed processing callback for node 69c64968b98d4d3c0206894f 2026-03-27 09:11:07,827 - lava_callback - INFO - Received callback for node 69c64962b98d4d3c02068941 INFO: 172.20.0.1:36576 - "POST /node/69c64962b98d4d3c02068941 HTTP/1.0" 202 Accepted 2026-03-27 09:11:07,834 - lava_callback - INFO - Starting async job submit for node 69c64962b98d4d3c02068941 2026-03-27 09:11:07,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:07,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64962b98d4d3c02068941 HTTP/1.1" 200 None 2026-03-27 09:11:07,875 - lava_callback - DEBUG - Processing node 69c64962b98d4d3c02068941: baseline-arm64 2026-03-27 09:11:08,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:08,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:08,298 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64962b98d4d3c02068941/log.txt.gz 2026-03-27 09:11:08,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:08,623 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:08,625 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64962b98d4d3c02068941/lava-logs.yaml 2026-03-27 09:11:08,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:08,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:08,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64962b98d4d3c02068941/lava_callback.json.gz 2026-03-27 09:11:08,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:08,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64962b98d4d3c02068941 HTTP/1.1" 200 None 2026-03-27 09:11:08,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:08,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:08,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:08,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64962b98d4d3c02068941 HTTP/1.1" 200 None 2026-03-27 09:11:08,854 - lava_callback - INFO - Completed processing callback for node 69c64962b98d4d3c02068941 2026-03-27 09:11:11,834 - lava_callback - INFO - Received callback for node 69c64969b98d4d3c02068952 INFO: 172.20.0.1:36510 - "POST /node/69c64969b98d4d3c02068952 HTTP/1.0" 202 Accepted 2026-03-27 09:11:11,840 - lava_callback - INFO - Starting async job submit for node 69c64969b98d4d3c02068952 2026-03-27 09:11:11,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:11,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64969b98d4d3c02068952 HTTP/1.1" 200 None 2026-03-27 09:11:11,879 - lava_callback - DEBUG - Processing node 69c64969b98d4d3c02068952: baseline-arm64 2026-03-27 09:11:12,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:12,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:12,218 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64969b98d4d3c02068952/log.txt.gz 2026-03-27 09:11:12,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:12,518 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:12,520 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64969b98d4d3c02068952/lava-logs.yaml 2026-03-27 09:11:12,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:12,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:12,661 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64969b98d4d3c02068952/lava_callback.json.gz 2026-03-27 09:11:12,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:12,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64969b98d4d3c02068952 HTTP/1.1" 200 None 2026-03-27 09:11:12,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:12,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:12,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:12,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64969b98d4d3c02068952 HTTP/1.1" 200 None 2026-03-27 09:11:12,750 - lava_callback - INFO - Completed processing callback for node 69c64969b98d4d3c02068952 2026-03-27 09:11:15,013 - lava_callback - INFO - Received callback for node 69c3a6aabbc022e792644bed INFO: 172.20.0.1:36520 - "POST /node/69c3a6aabbc022e792644bed HTTP/1.0" 202 Accepted 2026-03-27 09:11:15,019 - lava_callback - INFO - Starting async job submit for node 69c3a6aabbc022e792644bed 2026-03-27 09:11:15,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:15,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a6aabbc022e792644bed HTTP/1.1" 200 None 2026-03-27 09:11:15,035 - lava_callback - DEBUG - Processing node 69c3a6aabbc022e792644bed: baseline-arm64-baylibre 2026-03-27 09:11:15,035 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a6aabbc022e792644bed 2026-03-27 09:11:15,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:15,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a6aabbc022e792644bed HTTP/1.1" 200 None 2026-03-27 09:11:15,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:15,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:11:15,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:15,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a6aabbc022e792644bed HTTP/1.1" 200 None 2026-03-27 09:11:15,097 - lava_callback - ERROR - Error processing callback for node 69c3a6aabbc022e792644bed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36528 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:11:21,034 - lava_callback - INFO - Received callback for node 69c6495db98d4d3c02068935 INFO: 172.20.0.1:55992 - "POST /node/69c6495db98d4d3c02068935 HTTP/1.0" 202 Accepted 2026-03-27 09:11:21,040 - lava_callback - INFO - Starting async job submit for node 69c6495db98d4d3c02068935 2026-03-27 09:11:21,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:56004 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:11:21,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6495db98d4d3c02068935 HTTP/1.1" 200 None 2026-03-27 09:11:21,077 - lava_callback - DEBUG - Processing node 69c6495db98d4d3c02068935: baseline-arm64 2026-03-27 09:11:21,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:21,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:21,420 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495db98d4d3c02068935/log.txt.gz 2026-03-27 09:11:21,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:21,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:21,773 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495db98d4d3c02068935/lava-logs.yaml 2026-03-27 09:11:21,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:21,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:21,908 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495db98d4d3c02068935/lava_callback.json.gz 2026-03-27 09:11:21,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:21,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6495db98d4d3c02068935 HTTP/1.1" 200 None 2026-03-27 09:11:21,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:21,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:21,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:21,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6495db98d4d3c02068935 HTTP/1.1" 200 None 2026-03-27 09:11:21,996 - lava_callback - INFO - Completed processing callback for node 69c6495db98d4d3c02068935 2026-03-27 09:11:23,442 - lava_callback - INFO - Received callback for node 69c6495eb98d4d3c02068938 INFO: 172.20.0.1:56020 - "POST /node/69c6495eb98d4d3c02068938 HTTP/1.0" 202 Accepted 2026-03-27 09:11:23,451 - lava_callback - INFO - Starting async job submit for node 69c6495eb98d4d3c02068938 2026-03-27 09:11:23,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:23,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6495eb98d4d3c02068938 HTTP/1.1" 200 None 2026-03-27 09:11:23,497 - lava_callback - DEBUG - Processing node 69c6495eb98d4d3c02068938: baseline-arm64 2026-03-27 09:11:23,642 - lava_callback - WARNING - Failed to upload log 2026-03-27 09:11:23,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:23,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:23,887 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495eb98d4d3c02068938/lava-logs.yaml 2026-03-27 09:11:23,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:24,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:24,004 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495eb98d4d3c02068938/lava_callback.json.gz 2026-03-27 09:11:24,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:24,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6495eb98d4d3c02068938 HTTP/1.1" 200 None 2026-03-27 09:11:24,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:24,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:24,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:24,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6495eb98d4d3c02068938 HTTP/1.1" 200 None 2026-03-27 09:11:24,203 - lava_callback - INFO - Completed processing callback for node 69c6495eb98d4d3c02068938 INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:11:31,719 - lava_callback - INFO - Received callback for node 69c6496db98d4d3c02068958 INFO: 172.20.0.1:49946 - "POST /node/69c6496db98d4d3c02068958 HTTP/1.0" 202 Accepted 2026-03-27 09:11:31,725 - lava_callback - INFO - Starting async job submit for node 69c6496db98d4d3c02068958 2026-03-27 09:11:31,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:31,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6496db98d4d3c02068958 HTTP/1.1" 200 None 2026-03-27 09:11:31,768 - lava_callback - DEBUG - Processing node 69c6496db98d4d3c02068958: baseline-arm64 2026-03-27 09:11:32,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:32,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:32,187 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6496db98d4d3c02068958/log.txt.gz 2026-03-27 09:11:32,262 - lava_callback - INFO - Received callback for node 69c6495bb98d4d3c0206892f INFO: 172.20.0.1:49962 - "POST /node/69c6495bb98d4d3c0206892f HTTP/1.0" 202 Accepted 2026-03-27 09:11:32,287 - lava_callback - INFO - Starting async job submit for node 69c6495bb98d4d3c0206892f 2026-03-27 09:11:32,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:32,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:32,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6495bb98d4d3c0206892f HTTP/1.1" 200 None 2026-03-27 09:11:32,408 - lava_callback - DEBUG - Processing node 69c6495bb98d4d3c0206892f: baseline-arm64 2026-03-27 09:11:32,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:32,703 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6496db98d4d3c02068958/lava-logs.yaml 2026-03-27 09:11:32,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:32,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:32,848 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:32,850 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495bb98d4d3c0206892f/log.txt.gz 2026-03-27 09:11:32,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:32,912 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6496db98d4d3c02068958/lava_callback.json.gz 2026-03-27 09:11:32,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:32,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:33,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:33,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6496db98d4d3c02068958 HTTP/1.1" 200 None 2026-03-27 09:11:33,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-27 09:11:33,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:33,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:33,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:33,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6496db98d4d3c02068958 HTTP/1.1" 200 None 2026-03-27 09:11:33,102 - lava_callback - INFO - Completed processing callback for node 69c6496db98d4d3c02068958 2026-03-27 09:11:33,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:33,264 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495bb98d4d3c0206892f/lava-logs.yaml 2026-03-27 09:11:33,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:33,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:33,413 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6495bb98d4d3c0206892f/lava_callback.json.gz 2026-03-27 09:11:33,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:33,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6495bb98d4d3c0206892f HTTP/1.1" 200 None 2026-03-27 09:11:33,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:33,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:33,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:33,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6495bb98d4d3c0206892f HTTP/1.1" 200 None 2026-03-27 09:11:33,534 - lava_callback - INFO - Completed processing callback for node 69c6495bb98d4d3c0206892f INFO: 172.20.0.1:49966 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:11:38,228 - lava_callback - INFO - Received callback for node 69c6496fb98d4d3c0206895f INFO: 172.20.0.1:49982 - "POST /node/69c6496fb98d4d3c0206895f HTTP/1.0" 202 Accepted 2026-03-27 09:11:38,234 - lava_callback - INFO - Starting async job submit for node 69c6496fb98d4d3c0206895f 2026-03-27 09:11:38,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:38,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6496fb98d4d3c0206895f HTTP/1.1" 200 None 2026-03-27 09:11:38,272 - lava_callback - DEBUG - Processing node 69c6496fb98d4d3c0206895f: baseline-arm64 2026-03-27 09:11:38,369 - lava_callback - INFO - Received callback for node 69c64977b98d4d3c0206898d INFO: 172.20.0.1:49986 - "POST /node/69c64977b98d4d3c0206898d HTTP/1.0" 202 Accepted 2026-03-27 09:11:38,391 - lava_callback - INFO - Starting async job submit for node 69c64977b98d4d3c0206898d 2026-03-27 09:11:38,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:38,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64977b98d4d3c0206898d HTTP/1.1" 200 None 2026-03-27 09:11:38,548 - lava_callback - DEBUG - Processing node 69c64977b98d4d3c0206898d: baseline-nfs-arm64 2026-03-27 09:11:38,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:38,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:39,010 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6496fb98d4d3c0206895f/log.txt.gz 2026-03-27 09:11:39,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:39,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:39,342 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:39,344 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64977b98d4d3c0206898d/log.txt.gz 2026-03-27 09:11:39,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:39,452 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6496fb98d4d3c0206895f/lava-logs.yaml 2026-03-27 09:11:39,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:39,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:39,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:39,743 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6496fb98d4d3c0206895f/lava_callback.json.gz 2026-03-27 09:11:39,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:39,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6496fb98d4d3c0206895f HTTP/1.1" 200 None 2026-03-27 09:11:39,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:39,798 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:39,800 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64977b98d4d3c0206898d/lava-logs.yaml 2026-03-27 09:11:39,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:39,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:39,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:39,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6496fb98d4d3c0206895f HTTP/1.1" 200 None 2026-03-27 09:11:39,839 - lava_callback - INFO - Completed processing callback for node 69c6496fb98d4d3c0206895f 2026-03-27 09:11:40,062 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:40,065 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64977b98d4d3c0206898d/lava_callback.json.gz 2026-03-27 09:11:40,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:40,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64977b98d4d3c0206898d HTTP/1.1" 200 None 2026-03-27 09:11:40,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:40,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:40,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:40,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64977b98d4d3c0206898d HTTP/1.1" 200 None 2026-03-27 09:11:40,165 - lava_callback - INFO - Completed processing callback for node 69c64977b98d4d3c0206898d INFO: 172.20.0.1:35252 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:11:46,997 - lava_callback - INFO - Received callback for node 69c64974b98d4d3c0206896d INFO: 172.20.0.1:35260 - "POST /node/69c64974b98d4d3c0206896d HTTP/1.0" 202 Accepted 2026-03-27 09:11:47,003 - lava_callback - INFO - Starting async job submit for node 69c64974b98d4d3c0206896d 2026-03-27 09:11:47,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:47,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64974b98d4d3c0206896d HTTP/1.1" 200 None 2026-03-27 09:11:47,043 - lava_callback - DEBUG - Processing node 69c64974b98d4d3c0206896d: baseline-nfs-arm64 2026-03-27 09:11:47,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:47,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:47,494 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64974b98d4d3c0206896d/log.txt.gz 2026-03-27 09:11:47,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:47,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:47,884 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64974b98d4d3c0206896d/lava-logs.yaml 2026-03-27 09:11:47,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:48,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:48,017 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64974b98d4d3c0206896d/lava_callback.json.gz 2026-03-27 09:11:48,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:48,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64974b98d4d3c0206896d HTTP/1.1" 200 None 2026-03-27 09:11:48,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:48,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:48,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:48,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64974b98d4d3c0206896d HTTP/1.1" 200 None 2026-03-27 09:11:48,107 - lava_callback - INFO - Completed processing callback for node 69c64974b98d4d3c0206896d 2026-03-27 09:11:52,387 - lava_callback - INFO - Received callback for node 69c649a6b98d4d3c02068a1b INFO: 172.20.0.1:51104 - "POST /node/69c649a6b98d4d3c02068a1b HTTP/1.0" 202 Accepted 2026-03-27 09:11:52,396 - lava_callback - INFO - Starting async job submit for node 69c649a6b98d4d3c02068a1b 2026-03-27 09:11:52,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:52,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c649a6b98d4d3c02068a1b HTTP/1.1" 200 None 2026-03-27 09:11:52,434 - lava_callback - DEBUG - Processing node 69c649a6b98d4d3c02068a1b: baseline-arm64-foundriesio 2026-03-27 09:11:52,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:52,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:52,809 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c649a6b98d4d3c02068a1b/log.txt.gz 2026-03-27 09:11:52,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:53,134 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:53,136 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c649a6b98d4d3c02068a1b/lava-logs.yaml 2026-03-27 09:11:53,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:53,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:53,313 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c649a6b98d4d3c02068a1b/lava_callback.json.gz 2026-03-27 09:11:53,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:53,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c649a6b98d4d3c02068a1b HTTP/1.1" 200 None 2026-03-27 09:11:53,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:53,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:53,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:53,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c649a6b98d4d3c02068a1b HTTP/1.1" 200 None 2026-03-27 09:11:53,428 - lava_callback - INFO - Completed processing callback for node 69c649a6b98d4d3c02068a1b 2026-03-27 09:11:54,455 - lava_callback - INFO - Received callback for node 69c649a6b98d4d3c02068a1e INFO: 172.20.0.1:51120 - "POST /node/69c649a6b98d4d3c02068a1e HTTP/1.0" 202 Accepted 2026-03-27 09:11:54,464 - lava_callback - INFO - Starting async job submit for node 69c649a6b98d4d3c02068a1e 2026-03-27 09:11:54,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:54,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c649a6b98d4d3c02068a1e HTTP/1.1" 200 None 2026-03-27 09:11:54,514 - lava_callback - DEBUG - Processing node 69c649a6b98d4d3c02068a1e: baseline-arm64-foundriesio 2026-03-27 09:11:54,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:55,069 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:55,071 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c649a6b98d4d3c02068a1e/log.txt.gz 2026-03-27 09:11:55,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:55,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:55,592 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c649a6b98d4d3c02068a1e/lava-logs.yaml 2026-03-27 09:11:55,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:11:55,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:11:55,787 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c649a6b98d4d3c02068a1e/lava_callback.json.gz 2026-03-27 09:11:55,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:55,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c649a6b98d4d3c02068a1e HTTP/1.1" 200 None 2026-03-27 09:11:55,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:55,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:11:55,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:11:55,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c649a6b98d4d3c02068a1e HTTP/1.1" 200 None 2026-03-27 09:11:55,893 - lava_callback - INFO - Completed processing callback for node 69c649a6b98d4d3c02068a1e INFO: 172.20.0.1:51126 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:12:01,845 - lava_callback - INFO - Received callback for node 69c649a3b98d4d3c02068a11 INFO: 172.20.0.1:36216 - "POST /node/69c649a3b98d4d3c02068a11 HTTP/1.0" 202 Accepted 2026-03-27 09:12:01,850 - lava_callback - INFO - Starting async job submit for node 69c649a3b98d4d3c02068a11 2026-03-27 09:12:01,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:01,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c649a3b98d4d3c02068a11 HTTP/1.1" 200 None 2026-03-27 09:12:01,904 - lava_callback - DEBUG - Processing node 69c649a3b98d4d3c02068a11: baseline-arm64-pengutronix 2026-03-27 09:12:02,016 - lava_callback - INFO - Received callback for node 69c6497fb98d4d3c020689a3 INFO: 172.20.0.1:36218 - "POST /node/69c6497fb98d4d3c020689a3 HTTP/1.0" 202 Accepted 2026-03-27 09:12:02,043 - lava_callback - INFO - Starting async job submit for node 69c6497fb98d4d3c020689a3 2026-03-27 09:12:02,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:02,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:02,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6497fb98d4d3c020689a3 HTTP/1.1" 200 None 2026-03-27 09:12:02,135 - lava_callback - DEBUG - Processing node 69c6497fb98d4d3c020689a3: baseline-nfs-arm64 2026-03-27 09:12:02,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:02,289 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c649a3b98d4d3c02068a11/log.txt.gz 2026-03-27 09:12:02,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:02,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:02,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:02,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c649a3b98d4d3c02068a11/lava-logs.yaml 2026-03-27 09:12:02,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:02,818 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497fb98d4d3c020689a3/log.txt.gz 2026-03-27 09:12:02,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:03,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:03,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:03,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:03,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c649a3b98d4d3c02068a11/lava_callback.json.gz 2026-03-27 09:12:03,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:03,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-27 09:12:03,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c649a3b98d4d3c02068a11 HTTP/1.1" 200 None 2026-03-27 09:12:03,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:03,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:03,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:03,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c649a3b98d4d3c02068a11 HTTP/1.1" 200 None 2026-03-27 09:12:03,170 - lava_callback - INFO - Completed processing callback for node 69c649a3b98d4d3c02068a11 2026-03-27 09:12:03,333 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:03,336 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497fb98d4d3c020689a3/lava-logs.yaml 2026-03-27 09:12:03,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:03,542 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:03,544 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497fb98d4d3c020689a3/lava_callback.json.gz 2026-03-27 09:12:03,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:03,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6497fb98d4d3c020689a3 HTTP/1.1" 200 None 2026-03-27 09:12:03,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:03,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:03,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:03,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6497fb98d4d3c020689a3 HTTP/1.1" 200 None 2026-03-27 09:12:03,640 - lava_callback - INFO - Completed processing callback for node 69c6497fb98d4d3c020689a3 2026-03-27 09:12:04,723 - lava_callback - INFO - Received callback for node 69c6497db98d4d3c0206899c INFO: 172.20.0.1:36228 - "POST /node/69c6497db98d4d3c0206899c HTTP/1.0" 202 Accepted 2026-03-27 09:12:04,730 - lava_callback - INFO - Starting async job submit for node 69c6497db98d4d3c0206899c 2026-03-27 09:12:04,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:04,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6497db98d4d3c0206899c HTTP/1.1" 200 None 2026-03-27 09:12:04,773 - lava_callback - DEBUG - Processing node 69c6497db98d4d3c0206899c: baseline-nfs-arm64 2026-03-27 09:12:05,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:05,319 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:05,321 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497db98d4d3c0206899c/log.txt.gz 2026-03-27 09:12:05,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:05,715 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:05,717 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497db98d4d3c0206899c/lava-logs.yaml 2026-03-27 09:12:05,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:05,872 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:05,874 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497db98d4d3c0206899c/lava_callback.json.gz 2026-03-27 09:12:05,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:05,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6497db98d4d3c0206899c HTTP/1.1" 200 None 2026-03-27 09:12:05,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:05,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:05,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:05,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6497db98d4d3c0206899c HTTP/1.1" 200 None 2026-03-27 09:12:05,959 - lava_callback - INFO - Completed processing callback for node 69c6497db98d4d3c0206899c 2026-03-27 09:12:06,477 - lava_callback - INFO - Received callback for node 69c6497ab98d4d3c02068996 INFO: 172.20.0.1:36230 - "POST /node/69c6497ab98d4d3c02068996 HTTP/1.0" 202 Accepted 2026-03-27 09:12:06,488 - lava_callback - INFO - Starting async job submit for node 69c6497ab98d4d3c02068996 INFO: 172.20.0.1:36240 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:12:06,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:06,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6497ab98d4d3c02068996 HTTP/1.1" 200 None 2026-03-27 09:12:06,536 - lava_callback - DEBUG - Processing node 69c6497ab98d4d3c02068996: baseline-nfs-arm64 2026-03-27 09:12:06,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:07,051 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:07,052 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497ab98d4d3c02068996/log.txt.gz 2026-03-27 09:12:07,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:07,445 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:07,447 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497ab98d4d3c02068996/lava-logs.yaml 2026-03-27 09:12:07,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:07,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:07,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497ab98d4d3c02068996/lava_callback.json.gz 2026-03-27 09:12:07,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:07,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6497ab98d4d3c02068996 HTTP/1.1" 200 None 2026-03-27 09:12:07,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:07,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:07,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:07,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6497ab98d4d3c02068996 HTTP/1.1" 200 None 2026-03-27 09:12:07,698 - lava_callback - INFO - Completed processing callback for node 69c6497ab98d4d3c02068996 2026-03-27 09:12:13,113 - lava_callback - INFO - Received callback for node 69c649a4b98d4d3c02068a14 INFO: 172.20.0.1:46610 - "POST /node/69c649a4b98d4d3c02068a14 HTTP/1.0" 202 Accepted 2026-03-27 09:12:13,119 - lava_callback - INFO - Starting async job submit for node 69c649a4b98d4d3c02068a14 2026-03-27 09:12:13,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:13,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c649a4b98d4d3c02068a14 HTTP/1.1" 200 None 2026-03-27 09:12:13,156 - lava_callback - DEBUG - Processing node 69c649a4b98d4d3c02068a14: baseline-arm64-pengutronix 2026-03-27 09:12:13,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:13,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:13,499 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c649a4b98d4d3c02068a14/log.txt.gz 2026-03-27 09:12:13,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:13,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:13,816 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c649a4b98d4d3c02068a14/lava-logs.yaml 2026-03-27 09:12:13,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:13,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:13,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c649a4b98d4d3c02068a14/lava_callback.json.gz 2026-03-27 09:12:13,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:13,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c649a4b98d4d3c02068a14 HTTP/1.1" 200 None 2026-03-27 09:12:13,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:14,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:14,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:14,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c649a4b98d4d3c02068a14 HTTP/1.1" 200 None 2026-03-27 09:12:14,032 - lava_callback - INFO - Completed processing callback for node 69c649a4b98d4d3c02068a14 INFO: 172.20.0.1:46618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46626 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:12:17,602 - lava_callback - INFO - Received callback for node 69c64965b98d4d3c02068949 INFO: 172.20.0.1:46634 - "POST /node/69c64965b98d4d3c02068949 HTTP/1.0" 202 Accepted 2026-03-27 09:12:17,609 - lava_callback - INFO - Starting async job submit for node 69c64965b98d4d3c02068949 2026-03-27 09:12:17,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:17,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64965b98d4d3c02068949 HTTP/1.1" 200 None 2026-03-27 09:12:17,647 - lava_callback - DEBUG - Processing node 69c64965b98d4d3c02068949: baseline-arm64 2026-03-27 09:12:17,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:18,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:18,081 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64965b98d4d3c02068949/log.txt.gz 2026-03-27 09:12:18,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:18,465 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:18,467 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64965b98d4d3c02068949/lava-logs.yaml 2026-03-27 09:12:18,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:18,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:18,607 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64965b98d4d3c02068949/lava_callback.json.gz 2026-03-27 09:12:18,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:18,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64965b98d4d3c02068949 HTTP/1.1" 200 None 2026-03-27 09:12:18,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:18,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:18,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:18,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64965b98d4d3c02068949 HTTP/1.1" 200 None 2026-03-27 09:12:18,806 - lava_callback - INFO - Completed processing callback for node 69c64965b98d4d3c02068949 2026-03-27 09:12:19,969 - lava_callback - INFO - Received callback for node 69c64971b98d4d3c02068967 INFO: 172.20.0.1:55390 - "POST /node/69c64971b98d4d3c02068967 HTTP/1.0" 202 Accepted 2026-03-27 09:12:19,975 - lava_callback - INFO - Starting async job submit for node 69c64971b98d4d3c02068967 2026-03-27 09:12:19,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:20,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64971b98d4d3c02068967 HTTP/1.1" 200 None 2026-03-27 09:12:20,018 - lava_callback - DEBUG - Processing node 69c64971b98d4d3c02068967: baseline-nfs-arm64 2026-03-27 09:12:20,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:20,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:20,463 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64971b98d4d3c02068967/log.txt.gz 2026-03-27 09:12:20,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:20,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:20,842 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64971b98d4d3c02068967/lava-logs.yaml 2026-03-27 09:12:20,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:20,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:20,982 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64971b98d4d3c02068967/lava_callback.json.gz 2026-03-27 09:12:21,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:21,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64971b98d4d3c02068967 HTTP/1.1" 200 None 2026-03-27 09:12:21,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:21,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:21,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:21,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64971b98d4d3c02068967 HTTP/1.1" 200 None 2026-03-27 09:12:21,075 - lava_callback - INFO - Completed processing callback for node 69c64971b98d4d3c02068967 INFO: 172.20.0.1:55394 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:12:22,848 - lava_callback - INFO - Received callback for node 69c64998b98d4d3c020689f0 INFO: 172.20.0.1:55396 - "POST /node/69c64998b98d4d3c020689f0 HTTP/1.0" 202 Accepted 2026-03-27 09:12:22,854 - lava_callback - INFO - Starting async job submit for node 69c64998b98d4d3c020689f0 2026-03-27 09:12:22,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:22,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64998b98d4d3c020689f0 HTTP/1.1" 200 None 2026-03-27 09:12:22,893 - lava_callback - DEBUG - Processing node 69c64998b98d4d3c020689f0: baseline-arm64-broonie 2026-03-27 09:12:23,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:23,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:23,341 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64998b98d4d3c020689f0/log.txt.gz 2026-03-27 09:12:23,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:23,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:23,666 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64998b98d4d3c020689f0/lava-logs.yaml 2026-03-27 09:12:23,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:23,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:23,802 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64998b98d4d3c020689f0/lava_callback.json.gz 2026-03-27 09:12:23,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:23,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64998b98d4d3c020689f0 HTTP/1.1" 200 None 2026-03-27 09:12:23,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:23,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:23,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:23,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64998b98d4d3c020689f0 HTTP/1.1" 200 None 2026-03-27 09:12:23,907 - lava_callback - INFO - Completed processing callback for node 69c64998b98d4d3c020689f0 INFO: 172.20.0.1:55412 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:12:33,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:33,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-03-27 09:12:33,276 - lava_callback - INFO - Received callback for node 69c6498db98d4d3c020689bd INFO: 172.20.0.1:56336 - "POST /node/69c6498db98d4d3c020689bd HTTP/1.0" 202 Accepted 2026-03-27 09:12:33,282 - lava_callback - INFO - Starting async job submit for node 69c6498db98d4d3c020689bd 2026-03-27 09:12:33,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:33,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6498db98d4d3c020689bd HTTP/1.1" 200 None 2026-03-27 09:12:33,341 - lava_callback - DEBUG - Processing node 69c6498db98d4d3c020689bd: baseline-arm64-broonie 2026-03-27 09:12:33,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:33,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:33,688 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c6498db98d4d3c020689bd/log.txt.gz 2026-03-27 09:12:33,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:33,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:33,991 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c6498db98d4d3c020689bd/lava-logs.yaml 2026-03-27 09:12:33,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:34,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:34,310 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c6498db98d4d3c020689bd/lava_callback.json.gz 2026-03-27 09:12:34,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:34,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6498db98d4d3c020689bd HTTP/1.1" 200 None 2026-03-27 09:12:34,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:34,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:34,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:34,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6498db98d4d3c020689bd HTTP/1.1" 200 None 2026-03-27 09:12:34,397 - lava_callback - INFO - Completed processing callback for node 69c6498db98d4d3c020689bd 2026-03-27 09:12:34,709 - lava_callback - INFO - Received callback for node 69c64994b98d4d3c020689ea INFO: 172.20.0.1:56346 - "POST /node/69c64994b98d4d3c020689ea HTTP/1.0" 202 Accepted 2026-03-27 09:12:34,716 - lava_callback - INFO - Starting async job submit for node 69c64994b98d4d3c020689ea 2026-03-27 09:12:34,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:34,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64994b98d4d3c020689ea HTTP/1.1" 200 None 2026-03-27 09:12:34,779 - lava_callback - DEBUG - Processing node 69c64994b98d4d3c020689ea: baseline-arm64-broonie 2026-03-27 09:12:35,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:35,141 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:35,143 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64994b98d4d3c020689ea/log.txt.gz 2026-03-27 09:12:35,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:35,453 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:35,455 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64994b98d4d3c020689ea/lava-logs.yaml 2026-03-27 09:12:35,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:35,599 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:35,601 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64994b98d4d3c020689ea/lava_callback.json.gz 2026-03-27 09:12:35,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:35,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64994b98d4d3c020689ea HTTP/1.1" 200 None 2026-03-27 09:12:35,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:35,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:35,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:35,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64994b98d4d3c020689ea HTTP/1.1" 200 None 2026-03-27 09:12:35,689 - lava_callback - INFO - Completed processing callback for node 69c64994b98d4d3c020689ea INFO: 172.20.0.1:56352 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:12:41,832 - lava_callback - INFO - Received callback for node 69c3a6fcbbc022e792644bfa INFO: 172.20.0.1:54344 - "POST /node/69c3a6fcbbc022e792644bfa HTTP/1.0" 202 Accepted 2026-03-27 09:12:41,837 - lava_callback - INFO - Starting async job submit for node 69c3a6fcbbc022e792644bfa 2026-03-27 09:12:41,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:41,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a6fcbbc022e792644bfa HTTP/1.1" 200 None 2026-03-27 09:12:41,855 - lava_callback - DEBUG - Processing node 69c3a6fcbbc022e792644bfa: baseline-arm64-clabbe 2026-03-27 09:12:41,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a6fcbbc022e792644bfa 2026-03-27 09:12:41,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:41,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a6fcbbc022e792644bfa HTTP/1.1" 200 None 2026-03-27 09:12:41,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:41,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:12:41,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:41,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a6fcbbc022e792644bfa HTTP/1.1" 200 None 2026-03-27 09:12:41,925 - lava_callback - ERROR - Error processing callback for node 69c3a6fcbbc022e792644bfa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 09:12:42,863 - lava_callback - INFO - Received callback for node 69c64990b98d4d3c020689c5 INFO: 172.20.0.1:54354 - "POST /node/69c64990b98d4d3c020689c5 HTTP/1.0" 202 Accepted 2026-03-27 09:12:42,870 - lava_callback - INFO - Starting async job submit for node 69c64990b98d4d3c020689c5 2026-03-27 09:12:42,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:42,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64990b98d4d3c020689c5 HTTP/1.1" 200 None 2026-03-27 09:12:42,923 - lava_callback - DEBUG - Processing node 69c64990b98d4d3c020689c5: baseline-arm64-broonie 2026-03-27 09:12:43,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:43,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:43,337 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64990b98d4d3c020689c5/log.txt.gz 2026-03-27 09:12:43,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:43,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:43,708 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64990b98d4d3c020689c5/lava-logs.yaml 2026-03-27 09:12:43,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:43,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:43,845 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64990b98d4d3c020689c5/lava_callback.json.gz 2026-03-27 09:12:43,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:43,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64990b98d4d3c020689c5 HTTP/1.1" 200 None 2026-03-27 09:12:43,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:43,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:43,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:43,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64990b98d4d3c020689c5 HTTP/1.1" 200 None 2026-03-27 09:12:43,936 - lava_callback - INFO - Completed processing callback for node 69c64990b98d4d3c020689c5 2026-03-27 09:12:44,444 - lava_callback - INFO - Received callback for node 69c64991b98d4d3c020689c9 INFO: 172.20.0.1:54358 - "POST /node/69c64991b98d4d3c020689c9 HTTP/1.0" 202 Accepted 2026-03-27 09:12:44,451 - lava_callback - INFO - Starting async job submit for node 69c64991b98d4d3c020689c9 2026-03-27 09:12:44,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:44,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64991b98d4d3c020689c9 HTTP/1.1" 200 None 2026-03-27 09:12:44,501 - lava_callback - DEBUG - Processing node 69c64991b98d4d3c020689c9: baseline-arm64-broonie 2026-03-27 09:12:44,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:44,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:44,889 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64991b98d4d3c020689c9/log.txt.gz 2026-03-27 09:12:45,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:45,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:45,231 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64991b98d4d3c020689c9/lava-logs.yaml 2026-03-27 09:12:45,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:45,256 - lava_callback - INFO - Received callback for node 69c6499fb98d4d3c02068a08 INFO: 172.20.0.1:54368 - "POST /node/69c6499fb98d4d3c02068a08 HTTP/1.0" 202 Accepted 2026-03-27 09:12:45,262 - lava_callback - INFO - Starting async job submit for node 69c6499fb98d4d3c02068a08 2026-03-27 09:12:45,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:45,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6499fb98d4d3c02068a08 HTTP/1.1" 200 None 2026-03-27 09:12:45,304 - lava_callback - DEBUG - Processing node 69c6499fb98d4d3c02068a08: baseline-arm64-clabbe 2026-03-27 09:12:45,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:45,432 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64991b98d4d3c020689c9/lava_callback.json.gz 2026-03-27 09:12:45,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:45,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64991b98d4d3c020689c9 HTTP/1.1" 200 None 2026-03-27 09:12:45,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:45,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:45,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:45,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:45,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64991b98d4d3c020689c9 HTTP/1.1" 200 None 2026-03-27 09:12:45,622 - lava_callback - INFO - Completed processing callback for node 69c64991b98d4d3c020689c9 2026-03-27 09:12:45,845 - lava_callback - INFO - Received callback for node 69c64996b98d4d3c020689ed INFO: 172.20.0.1:54380 - "POST /node/69c64996b98d4d3c020689ed HTTP/1.0" 202 Accepted 2026-03-27 09:12:45,853 - lava_callback - INFO - Starting async job submit for node 69c64996b98d4d3c020689ed 2026-03-27 09:12:45,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:45,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:45,897 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c6499fb98d4d3c02068a08/log.txt.gz 2026-03-27 09:12:45,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64996b98d4d3c020689ed HTTP/1.1" 200 None 2026-03-27 09:12:45,926 - lava_callback - DEBUG - Processing node 69c64996b98d4d3c020689ed: baseline-arm64-broonie 2026-03-27 09:12:46,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:46,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:54394 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:12:46,536 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:46,537 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64996b98d4d3c020689ed/log.txt.gz 2026-03-27 09:12:46,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:46,582 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c6499fb98d4d3c02068a08/lava-logs.yaml 2026-03-27 09:12:46,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:46,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:46,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:46,825 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c6499fb98d4d3c02068a08/lava_callback.json.gz 2026-03-27 09:12:46,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:46,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6499fb98d4d3c02068a08 HTTP/1.1" 200 None 2026-03-27 09:12:46,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:46,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:46,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:46,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:46,902 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64996b98d4d3c020689ed/lava-logs.yaml 2026-03-27 09:12:46,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:46,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6499fb98d4d3c02068a08 HTTP/1.1" 200 None 2026-03-27 09:12:46,918 - lava_callback - INFO - Completed processing callback for node 69c6499fb98d4d3c02068a08 2026-03-27 09:12:47,087 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:47,089 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64996b98d4d3c020689ed/lava_callback.json.gz 2026-03-27 09:12:47,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:47,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64996b98d4d3c020689ed HTTP/1.1" 200 None 2026-03-27 09:12:47,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:47,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:47,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:47,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64996b98d4d3c020689ed HTTP/1.1" 200 None 2026-03-27 09:12:47,188 - lava_callback - INFO - Completed processing callback for node 69c64996b98d4d3c020689ed 2026-03-27 09:12:47,261 - lava_callback - INFO - Received callback for node 69c64999b98d4d3c020689f3 INFO: 172.20.0.1:54408 - "POST /node/69c64999b98d4d3c020689f3 HTTP/1.0" 202 Accepted 2026-03-27 09:12:47,267 - lava_callback - INFO - Starting async job submit for node 69c64999b98d4d3c020689f3 2026-03-27 09:12:47,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:47,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64999b98d4d3c020689f3 HTTP/1.1" 200 None 2026-03-27 09:12:47,316 - lava_callback - DEBUG - Processing node 69c64999b98d4d3c020689f3: baseline-arm64-broonie 2026-03-27 09:12:47,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:47,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:47,819 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64999b98d4d3c020689f3/log.txt.gz 2026-03-27 09:12:48,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:48,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:48,185 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64999b98d4d3c020689f3/lava-logs.yaml 2026-03-27 09:12:48,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:12:48,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:12:48,313 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64999b98d4d3c020689f3/lava_callback.json.gz 2026-03-27 09:12:48,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:48,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64999b98d4d3c020689f3 HTTP/1.1" 200 None 2026-03-27 09:12:48,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:48,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:12:48,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:12:48,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64999b98d4d3c020689f3 HTTP/1.1" 200 None 2026-03-27 09:12:48,455 - lava_callback - INFO - Completed processing callback for node 69c64999b98d4d3c020689f3 INFO: 172.20.0.1:40386 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:13:03,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:13:03,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:55284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54938 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:13:33,436 - lava_callback - INFO - Received callback for node 69c6498eb98d4d3c020689c0 INFO: 172.20.0.1:37716 - "POST /node/69c6498eb98d4d3c020689c0 HTTP/1.0" 202 Accepted 2026-03-27 09:13:33,446 - lava_callback - INFO - Starting async job submit for node 69c6498eb98d4d3c020689c0 2026-03-27 09:13:33,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:13:33,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6498eb98d4d3c020689c0 HTTP/1.1" 200 None 2026-03-27 09:13:33,485 - lava_callback - DEBUG - Processing node 69c6498eb98d4d3c020689c0: baseline-arm64-broonie 2026-03-27 09:13:33,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:13:33,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:13:33,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c6498eb98d4d3c020689c0/log.txt.gz 2026-03-27 09:13:34,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:13:34,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:13:34,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c6498eb98d4d3c020689c0/lava-logs.yaml 2026-03-27 09:13:34,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:13:34,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:13:34,357 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c6498eb98d4d3c020689c0/lava_callback.json.gz 2026-03-27 09:13:34,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:13:34,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6498eb98d4d3c020689c0 HTTP/1.1" 200 None 2026-03-27 09:13:34,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:13:34,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:13:34,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:13:34,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6498eb98d4d3c020689c0 HTTP/1.1" 200 None 2026-03-27 09:13:34,437 - lava_callback - INFO - Completed processing callback for node 69c6498eb98d4d3c020689c0 INFO: 172.20.0.1:37730 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:13:46,346 - lava_callback - INFO - Received callback for node 69c64961b98d4d3c0206893e INFO: 172.20.0.1:56528 - "POST /node/69c64961b98d4d3c0206893e HTTP/1.0" 202 Accepted 2026-03-27 09:13:46,352 - lava_callback - INFO - Starting async job submit for node 69c64961b98d4d3c0206893e 2026-03-27 09:13:46,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:13:46,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64961b98d4d3c0206893e HTTP/1.1" 200 None 2026-03-27 09:13:46,385 - lava_callback - DEBUG - Processing node 69c64961b98d4d3c0206893e: baseline-arm64 INFO: 172.20.0.1:56530 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:13:46,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:13:46,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:13:46,865 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64961b98d4d3c0206893e/log.txt.gz 2026-03-27 09:13:47,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:13:47,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:13:47,251 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64961b98d4d3c0206893e/lava-logs.yaml 2026-03-27 09:13:47,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:13:47,403 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:13:47,405 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64961b98d4d3c0206893e/lava_callback.json.gz 2026-03-27 09:13:47,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:13:47,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64961b98d4d3c0206893e HTTP/1.1" 200 None 2026-03-27 09:13:47,438 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:13:47,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:13:47,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:13:47,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64961b98d4d3c0206893e HTTP/1.1" 200 None 2026-03-27 09:13:47,494 - lava_callback - INFO - Completed processing callback for node 69c64961b98d4d3c0206893e INFO: 172.20.0.1:52742 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:14:01,965 - lava_callback - INFO - Received callback for node 69c3a749bbc022e792644c15 INFO: 172.20.0.1:40166 - "POST /node/69c3a749bbc022e792644c15 HTTP/1.0" 202 Accepted 2026-03-27 09:14:01,971 - lava_callback - INFO - Starting async job submit for node 69c3a749bbc022e792644c15 2026-03-27 09:14:01,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:14:01,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a749bbc022e792644c15 HTTP/1.1" 200 None 2026-03-27 09:14:01,992 - lava_callback - DEBUG - Processing node 69c3a749bbc022e792644c15: baseline-arm64-clabbe 2026-03-27 09:14:01,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a749bbc022e792644c15 2026-03-27 09:14:01,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:14:02,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a749bbc022e792644c15 HTTP/1.1" 200 None 2026-03-27 09:14:02,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:14:02,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:14:02,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:14:02,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a749bbc022e792644c15 HTTP/1.1" 200 None 2026-03-27 09:14:02,050 - lava_callback - ERROR - Error processing callback for node 69c3a749bbc022e792644c15: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 09:14:03,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:14:03,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45292 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:14:33,241 - lava_callback - INFO - Received callback for node 69c64993b98d4d3c020689e7 INFO: 172.20.0.1:53190 - "POST /node/69c64993b98d4d3c020689e7 HTTP/1.0" 202 Accepted 2026-03-27 09:14:33,247 - lava_callback - INFO - Starting async job submit for node 69c64993b98d4d3c020689e7 2026-03-27 09:14:33,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:14:33,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64993b98d4d3c020689e7 HTTP/1.1" 200 None 2026-03-27 09:14:33,284 - lava_callback - DEBUG - Processing node 69c64993b98d4d3c020689e7: baseline-arm64-broonie 2026-03-27 09:14:33,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:14:33,720 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:14:33,721 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64993b98d4d3c020689e7/log.txt.gz 2026-03-27 09:14:33,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:14:34,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:14:34,087 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64993b98d4d3c020689e7/lava-logs.yaml 2026-03-27 09:14:34,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:14:34,272 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:14:34,274 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c64993b98d4d3c020689e7/lava_callback.json.gz 2026-03-27 09:14:34,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:14:34,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64993b98d4d3c020689e7 HTTP/1.1" 200 None 2026-03-27 09:14:34,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:14:34,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:14:34,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:14:34,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64993b98d4d3c020689e7 HTTP/1.1" 200 None 2026-03-27 09:14:34,355 - lava_callback - INFO - Completed processing callback for node 69c64993b98d4d3c020689e7 INFO: 172.20.0.1:53204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50798 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:15:03,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:15:03,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:32910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37670 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 09:15:16,270 - lava_callback - INFO - Received callback for node 69c5ee062cb62322ad9690d6 INFO: 172.20.0.1:37678 - "POST /node/69c5ee062cb62322ad9690d6 HTTP/1.0" 202 Accepted 2026-03-27 09:15:16,277 - lava_callback - INFO - Starting async job submit for node 69c5ee062cb62322ad9690d6 2026-03-27 09:15:16,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:15:16,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ee062cb62322ad9690d6 HTTP/1.1" 200 None 2026-03-27 09:15:16,317 - lava_callback - DEBUG - Processing node 69c5ee062cb62322ad9690d6: baseline-nfs-arm64 INFO: 172.20.0.1:37686 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:15:16,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:15:17,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:15:17,005 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5ee062cb62322ad9690d6/log.txt.gz 2026-03-27 09:15:17,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:15:17,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:15:17,502 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5ee062cb62322ad9690d6/lava-logs.yaml 2026-03-27 09:15:17,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:15:17,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:15:17,675 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5ee062cb62322ad9690d6/lava_callback.json.gz 2026-03-27 09:15:17,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:15:17,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ee062cb62322ad9690d6 HTTP/1.1" 200 None 2026-03-27 09:15:17,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:15:17,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 09:15:17,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:15:17,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ee062cb62322ad9690d6 HTTP/1.1" 200 None 2026-03-27 09:15:17,822 - lava_callback - INFO - Completed processing callback for node 69c5ee062cb62322ad9690d6 INFO: 172.20.0.1:50334 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:15:22,006 - lava_callback - INFO - Received callback for node 69c3a796bbc022e792644c22 INFO: 172.20.0.1:50338 - "POST /node/69c3a796bbc022e792644c22 HTTP/1.0" 202 Accepted 2026-03-27 09:15:22,012 - lava_callback - INFO - Starting async job submit for node 69c3a796bbc022e792644c22 2026-03-27 09:15:22,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:15:22,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a796bbc022e792644c22 HTTP/1.1" 200 None 2026-03-27 09:15:22,040 - lava_callback - DEBUG - Processing node 69c3a796bbc022e792644c22: baseline-arm64-clabbe 2026-03-27 09:15:22,040 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a796bbc022e792644c22 2026-03-27 09:15:22,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:15:22,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a796bbc022e792644c22 HTTP/1.1" 200 None 2026-03-27 09:15:22,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:15:22,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:15:22,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:15:22,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a796bbc022e792644c22 HTTP/1.1" 200 None 2026-03-27 09:15:22,095 - lava_callback - ERROR - Error processing callback for node 69c3a796bbc022e792644c22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50350 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:15:33,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:15:33,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39716 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:16:13,883 - lava_callback - INFO - Received callback for node 69c64960b98d4d3c0206893b INFO: 172.20.0.1:58790 - "POST /node/69c64960b98d4d3c0206893b HTTP/1.0" 202 Accepted 2026-03-27 09:16:13,890 - lava_callback - INFO - Starting async job submit for node 69c64960b98d4d3c0206893b 2026-03-27 09:16:13,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:16:13,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64960b98d4d3c0206893b HTTP/1.1" 200 None 2026-03-27 09:16:13,936 - lava_callback - DEBUG - Processing node 69c64960b98d4d3c0206893b: baseline-arm64 2026-03-27 09:16:14,084 - lava_callback - WARNING - Failed to upload log 2026-03-27 09:16:14,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:16:14,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:16:14,338 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64960b98d4d3c0206893b/lava-logs.yaml 2026-03-27 09:16:14,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:16:14,492 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:16:14,494 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64960b98d4d3c0206893b/lava_callback.json.gz 2026-03-27 09:16:14,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:16:14,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64960b98d4d3c0206893b HTTP/1.1" 200 None 2026-03-27 09:16:14,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:16:14,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:16:14,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:16:14,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64960b98d4d3c0206893b HTTP/1.1" 200 None 2026-03-27 09:16:14,642 - lava_callback - INFO - Completed processing callback for node 69c64960b98d4d3c0206893b INFO: 172.20.0.1:58802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33632 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:16:22,054 - lava_callback - INFO - Received callback for node 69c3a7e3bbc022e792644c35 INFO: 172.20.0.1:33646 - "POST /node/69c3a7e3bbc022e792644c35 HTTP/1.0" 202 Accepted 2026-03-27 09:16:22,060 - lava_callback - INFO - Starting async job submit for node 69c3a7e3bbc022e792644c35 2026-03-27 09:16:22,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:16:22,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a7e3bbc022e792644c35 HTTP/1.1" 200 None 2026-03-27 09:16:22,079 - lava_callback - DEBUG - Processing node 69c3a7e3bbc022e792644c35: baseline-arm64-clabbe 2026-03-27 09:16:22,079 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a7e3bbc022e792644c35 2026-03-27 09:16:22,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:16:22,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a7e3bbc022e792644c35 HTTP/1.1" 200 None 2026-03-27 09:16:22,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:16:22,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:16:22,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:16:22,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a7e3bbc022e792644c35 HTTP/1.1" 200 None 2026-03-27 09:16:22,164 - lava_callback - ERROR - Error processing callback for node 69c3a7e3bbc022e792644c35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33650 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:16:33,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:16:33,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37024 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:17:10,949 - lava_callback - INFO - Received callback for node 69c64973b98d4d3c0206896a INFO: 172.20.0.1:51972 - "POST /node/69c64973b98d4d3c0206896a HTTP/1.0" 202 Accepted 2026-03-27 09:17:10,956 - lava_callback - INFO - Starting async job submit for node 69c64973b98d4d3c0206896a 2026-03-27 09:17:10,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:10,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64973b98d4d3c0206896a HTTP/1.1" 200 None 2026-03-27 09:17:10,989 - lava_callback - DEBUG - Processing node 69c64973b98d4d3c0206896a: baseline-nfs-arm64 2026-03-27 09:17:11,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:17:11,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:17:11,594 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64973b98d4d3c0206896a/log.txt.gz 2026-03-27 09:17:11,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:17:12,043 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:17:12,045 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64973b98d4d3c0206896a/lava-logs.yaml 2026-03-27 09:17:12,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:17:12,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:17:12,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64973b98d4d3c0206896a/lava_callback.json.gz 2026-03-27 09:17:12,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:12,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64973b98d4d3c0206896a HTTP/1.1" 200 None 2026-03-27 09:17:12,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:12,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:17:12,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:12,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64973b98d4d3c0206896a HTTP/1.1" 200 None 2026-03-27 09:17:12,332 - lava_callback - INFO - Completed processing callback for node 69c64973b98d4d3c0206896a INFO: 172.20.0.1:51976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53340 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:17:31,072 - lava_callback - INFO - Received callback for node 69c64970b98d4d3c02068962 INFO: 172.20.0.1:57410 - "POST /node/69c64970b98d4d3c02068962 HTTP/1.0" 202 Accepted 2026-03-27 09:17:31,080 - lava_callback - INFO - Starting async job submit for node 69c64970b98d4d3c02068962 2026-03-27 09:17:31,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:31,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64970b98d4d3c02068962 HTTP/1.1" 200 None 2026-03-27 09:17:31,131 - lava_callback - DEBUG - Processing node 69c64970b98d4d3c02068962: baseline-arm64 2026-03-27 09:17:31,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:17:31,649 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:17:31,651 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64970b98d4d3c02068962/log.txt.gz 2026-03-27 09:17:31,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:17:32,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:17:32,102 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64970b98d4d3c02068962/lava-logs.yaml 2026-03-27 09:17:32,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:17:32,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:17:32,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64970b98d4d3c02068962/lava_callback.json.gz DEBUG: Setup failed for 69c64970b98d4d3c02068962. Transitting job node result: incomplete -> fail 2026-03-27 09:17:32,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:32,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64970b98d4d3c02068962 HTTP/1.1" 200 None 2026-03-27 09:17:32,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:32,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:17:32,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:32,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64970b98d4d3c02068962 HTTP/1.1" 200 None 2026-03-27 09:17:32,473 - lava_callback - INFO - Completed processing callback for node 69c64970b98d4d3c02068962 2026-03-27 09:17:33,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:33,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57424 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:17:42,152 - lava_callback - INFO - Received callback for node 69c3a830bbc022e792644c53 INFO: 172.20.0.1:49838 - "POST /node/69c3a830bbc022e792644c53 HTTP/1.0" 202 Accepted 2026-03-27 09:17:42,158 - lava_callback - INFO - Starting async job submit for node 69c3a830bbc022e792644c53 2026-03-27 09:17:42,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:42,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a830bbc022e792644c53 HTTP/1.1" 200 None 2026-03-27 09:17:42,174 - lava_callback - DEBUG - Processing node 69c3a830bbc022e792644c53: baseline-arm64-clabbe 2026-03-27 09:17:42,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a830bbc022e792644c53 2026-03-27 09:17:42,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:42,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a830bbc022e792644c53 HTTP/1.1" 200 None 2026-03-27 09:17:42,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:42,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:17:42,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:17:42,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a830bbc022e792644c53 HTTP/1.1" 200 None 2026-03-27 09:17:42,238 - lava_callback - ERROR - Error processing callback for node 69c3a830bbc022e792644c53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43622 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:19:02,197 - lava_callback - INFO - Received callback for node 69c3a87cbbc022e792644c5d INFO: 172.20.0.1:59604 - "POST /node/69c3a87cbbc022e792644c5d HTTP/1.0" 202 Accepted 2026-03-27 09:19:02,202 - lava_callback - INFO - Starting async job submit for node 69c3a87cbbc022e792644c5d 2026-03-27 09:19:02,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:19:02,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a87cbbc022e792644c5d HTTP/1.1" 200 None 2026-03-27 09:19:02,229 - lava_callback - DEBUG - Processing node 69c3a87cbbc022e792644c5d: baseline-arm64-clabbe 2026-03-27 09:19:02,229 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3a87cbbc022e792644c5d 2026-03-27 09:19:02,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:19:02,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3a87cbbc022e792644c5d HTTP/1.1" 200 None 2026-03-27 09:19:02,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:19:02,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 09:19:02,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:19:02,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3a87cbbc022e792644c5d HTTP/1.1" 200 None 2026-03-27 09:19:02,290 - lava_callback - ERROR - Error processing callback for node 69c3a87cbbc022e792644c5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60046 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:22:33,693 - lava_callback - INFO - Received callback for node 69c64831b98d4d3c020688d7 INFO: 172.20.0.1:33196 - "POST /node/69c64831b98d4d3c020688d7 HTTP/1.0" 202 Accepted 2026-03-27 09:22:33,700 - lava_callback - INFO - Starting async job submit for node 69c64831b98d4d3c020688d7 2026-03-27 09:22:33,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:22:33,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64831b98d4d3c020688d7 HTTP/1.1" 200 None 2026-03-27 09:22:33,773 - lava_callback - DEBUG - Processing node 69c64831b98d4d3c020688d7: ltp-smoke 2026-03-27 09:22:34,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:22:34,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:22:34,393 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c64831b98d4d3c020688d7/log.txt.gz 2026-03-27 09:22:34,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:22:34,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:22:34,907 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c64831b98d4d3c020688d7/lava-logs.yaml 2026-03-27 09:22:34,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:22:35,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:22:35,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c64831b98d4d3c020688d7/lava_callback.json.gz 2026-03-27 09:22:35,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:22:35,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64831b98d4d3c020688d7 HTTP/1.1" 200 None 2026-03-27 09:22:35,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:22:35,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce8b98d4d3c02067dcb HTTP/1.1" 200 None 2026-03-27 09:22:35,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:22:35,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64831b98d4d3c020688d7 HTTP/1.1" 200 None 2026-03-27 09:22:35,586 - lava_callback - INFO - Completed processing callback for node 69c64831b98d4d3c020688d7 INFO: 172.20.0.1:33204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46634 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:23:03,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:23:03,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46060 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:29:54,240 - lava_callback - INFO - Received callback for node 69c64975b98d4d3c0206898a INFO: 172.20.0.1:54642 - "POST /node/69c64975b98d4d3c0206898a HTTP/1.0" 202 Accepted 2026-03-27 09:29:54,249 - lava_callback - INFO - Starting async job submit for node 69c64975b98d4d3c0206898a 2026-03-27 09:29:54,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:29:54,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64975b98d4d3c0206898a HTTP/1.1" 200 None 2026-03-27 09:29:54,320 - lava_callback - DEBUG - Processing node 69c64975b98d4d3c0206898a: baseline-nfs-arm64 2026-03-27 09:29:55,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:29:55,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:29:55,692 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64975b98d4d3c0206898a/log.txt.gz 2026-03-27 09:29:56,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:54650 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:29:56,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:29:56,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64975b98d4d3c0206898a/lava-logs.yaml 2026-03-27 09:29:56,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:29:56,850 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:29:56,852 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64975b98d4d3c0206898a/lava_callback.json.gz DEBUG: Setup failed for 69c64975b98d4d3c0206898a. Transitting job node result: incomplete -> fail 2026-03-27 09:29:56,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:29:56,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64975b98d4d3c0206898a HTTP/1.1" 200 None 2026-03-27 09:29:56,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:29:56,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:29:56,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:29:57,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64975b98d4d3c0206898a HTTP/1.1" 200 None 2026-03-27 09:29:57,181 - lava_callback - INFO - Completed processing callback for node 69c64975b98d4d3c0206898a 2026-03-27 09:30:03,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:30:03,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55520 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:38:16,545 - lava_callback - INFO - Received callback for node 69c3ad05bbc022e792644da2 INFO: 172.20.0.1:55534 - "POST /node/69c3ad05bbc022e792644da2 HTTP/1.0" 202 Accepted 2026-03-27 09:38:16,551 - lava_callback - INFO - Starting async job submit for node 69c3ad05bbc022e792644da2 2026-03-27 09:38:16,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:16,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ad05bbc022e792644da2 HTTP/1.1" 200 None 2026-03-27 09:38:16,568 - lava_callback - DEBUG - Processing node 69c3ad05bbc022e792644da2: baseline-arm-baylibre 2026-03-27 09:38:16,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ad05bbc022e792644da2 2026-03-27 09:38:16,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:16,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ad05bbc022e792644da2 HTTP/1.1" 200 None 2026-03-27 09:38:16,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:16,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:38:16,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:16,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ad05bbc022e792644da2 HTTP/1.1" 200 None 2026-03-27 09:38:16,653 - lava_callback - ERROR - Error processing callback for node 69c3ad05bbc022e792644da2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38560 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:38:40,540 - lava_callback - INFO - Received callback for node 69c65012b98d4d3c02068cca INFO: 172.20.0.1:47924 - "POST /node/69c65012b98d4d3c02068cca HTTP/1.0" 202 Accepted 2026-03-27 09:38:40,547 - lava_callback - INFO - Starting async job submit for node 69c65012b98d4d3c02068cca 2026-03-27 09:38:40,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:40,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65012b98d4d3c02068cca HTTP/1.1" 200 None 2026-03-27 09:38:40,567 - lava_callback - DEBUG - Processing node 69c65012b98d4d3c02068cca: baseline-arm 2026-03-27 09:38:40,573 - lava_callback - WARNING - Failed to upload log 2026-03-27 09:38:40,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:38:40,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:38:40,731 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c65012b98d4d3c02068cca/lava-logs.yaml 2026-03-27 09:38:40,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:38:40,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:38:40,840 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c65012b98d4d3c02068cca/lava_callback.json.gz 2026-03-27 09:38:40,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:40,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65012b98d4d3c02068cca HTTP/1.1" 200 None 2026-03-27 09:38:40,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:40,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 09:38:40,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:40,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65012b98d4d3c02068cca HTTP/1.1" 200 None 2026-03-27 09:38:40,906 - lava_callback - INFO - Completed processing callback for node 69c65012b98d4d3c02068cca INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:38:56,492 - lava_callback - INFO - Received callback for node 69c3ad29bbc022e792644dac INFO: 172.20.0.1:55796 - "POST /node/69c3ad29bbc022e792644dac HTTP/1.0" 202 Accepted 2026-03-27 09:38:56,498 - lava_callback - INFO - Starting async job submit for node 69c3ad29bbc022e792644dac 2026-03-27 09:38:56,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:55810 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:38:56,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ad29bbc022e792644dac HTTP/1.1" 200 None 2026-03-27 09:38:56,518 - lava_callback - DEBUG - Processing node 69c3ad29bbc022e792644dac: baseline-arm-baylibre 2026-03-27 09:38:56,518 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ad29bbc022e792644dac 2026-03-27 09:38:56,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:56,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ad29bbc022e792644dac HTTP/1.1" 200 None 2026-03-27 09:38:56,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:56,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:38:56,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:38:56,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ad29bbc022e792644dac HTTP/1.1" 200 None 2026-03-27 09:38:56,573 - lava_callback - ERROR - Error processing callback for node 69c3ad29bbc022e792644dac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 09:39:03,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:39:03,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35984 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:39:36,546 - lava_callback - INFO - Received callback for node 69c3ad4ebbc022e792644db5 INFO: 172.20.0.1:36000 - "POST /node/69c3ad4ebbc022e792644db5 HTTP/1.0" 202 Accepted 2026-03-27 09:39:36,552 - lava_callback - INFO - Starting async job submit for node 69c3ad4ebbc022e792644db5 2026-03-27 09:39:36,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:39:36,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ad4ebbc022e792644db5 HTTP/1.1" 200 None 2026-03-27 09:39:36,568 - lava_callback - DEBUG - Processing node 69c3ad4ebbc022e792644db5: baseline-arm-baylibre 2026-03-27 09:39:36,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ad4ebbc022e792644db5 2026-03-27 09:39:36,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:39:36,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ad4ebbc022e792644db5 HTTP/1.1" 200 None 2026-03-27 09:39:36,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:39:36,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:39:36,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:39:36,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ad4ebbc022e792644db5 HTTP/1.1" 200 None 2026-03-27 09:39:36,628 - lava_callback - ERROR - Error processing callback for node 69c3ad4ebbc022e792644db5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43222 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:40:16,537 - lava_callback - INFO - Received callback for node 69c3ad71bbc022e792644dbd INFO: 172.20.0.1:43238 - "POST /node/69c3ad71bbc022e792644dbd HTTP/1.0" 202 Accepted 2026-03-27 09:40:16,545 - lava_callback - INFO - Starting async job submit for node 69c3ad71bbc022e792644dbd 2026-03-27 09:40:16,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:40:16,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ad71bbc022e792644dbd HTTP/1.1" 200 None 2026-03-27 09:40:16,566 - lava_callback - DEBUG - Processing node 69c3ad71bbc022e792644dbd: baseline-arm-baylibre 2026-03-27 09:40:16,567 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ad71bbc022e792644dbd 2026-03-27 09:40:16,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:40:16,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ad71bbc022e792644dbd HTTP/1.1" 200 None 2026-03-27 09:40:16,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:40:16,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:40:16,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:40:16,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ad71bbc022e792644dbd HTTP/1.1" 200 None 2026-03-27 09:40:16,631 - lava_callback - ERROR - Error processing callback for node 69c3ad71bbc022e792644dbd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41674 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:40:31,619 - lava_callback - INFO - Received callback for node 69c64c5fb98d4d3c02068b90 INFO: 172.20.0.1:41404 - "POST /node/69c64c5fb98d4d3c02068b90 HTTP/1.0" 202 Accepted 2026-03-27 09:40:31,625 - lava_callback - INFO - Starting async job submit for node 69c64c5fb98d4d3c02068b90 2026-03-27 09:40:31,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:40:31,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64c5fb98d4d3c02068b90 HTTP/1.1" 200 None 2026-03-27 09:40:31,698 - lava_callback - DEBUG - Processing node 69c64c5fb98d4d3c02068b90: baseline-arm 2026-03-27 09:40:32,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:40:32,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:40:32,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c64c5fb98d4d3c02068b90/log.txt.gz 2026-03-27 09:40:32,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:40:33,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:40:33,076 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c64c5fb98d4d3c02068b90/lava-logs.yaml 2026-03-27 09:40:33,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:40:33,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:40:33,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c64c5fb98d4d3c02068b90/lava_callback.json.gz DEBUG: Setup failed for 69c64c5fb98d4d3c02068b90. Transitting job node result: incomplete -> fail 2026-03-27 09:40:33,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:40:33,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64c5fb98d4d3c02068b90 HTTP/1.1" 200 None 2026-03-27 09:40:33,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:40:33,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 09:40:33,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:40:33,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64c5fb98d4d3c02068b90 HTTP/1.1" 200 None 2026-03-27 09:40:33,529 - lava_callback - INFO - Completed processing callback for node 69c64c5fb98d4d3c02068b90 INFO: 172.20.0.1:41410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52382 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:41:03,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:41:03,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43280 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:41:56,656 - lava_callback - INFO - Received callback for node 69c3addfbbc022e792644de1 INFO: 172.20.0.1:43282 - "POST /node/69c3addfbbc022e792644de1 HTTP/1.0" 202 Accepted 2026-03-27 09:41:56,661 - lava_callback - INFO - Starting async job submit for node 69c3addfbbc022e792644de1 2026-03-27 09:41:56,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:41:56,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3addfbbc022e792644de1 HTTP/1.1" 200 None 2026-03-27 09:41:56,678 - lava_callback - DEBUG - Processing node 69c3addfbbc022e792644de1: baseline-arm-baylibre 2026-03-27 09:41:56,678 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3addfbbc022e792644de1 2026-03-27 09:41:56,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:41:56,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3addfbbc022e792644de1 HTTP/1.1" 200 None 2026-03-27 09:41:56,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:41:56,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:41:56,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:41:56,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3addfbbc022e792644de1 HTTP/1.1" 200 None 2026-03-27 09:41:56,743 - lava_callback - ERROR - Error processing callback for node 69c3addfbbc022e792644de1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54192 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:42:54,906 - lava_callback - INFO - Received callback for node 69c5ee572cb62322ad9690dd INFO: 172.20.0.1:49564 - "POST /node/69c5ee572cb62322ad9690dd HTTP/1.0" 202 Accepted 2026-03-27 09:42:54,912 - lava_callback - INFO - Starting async job submit for node 69c5ee572cb62322ad9690dd 2026-03-27 09:42:54,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:42:54,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ee572cb62322ad9690dd HTTP/1.1" 200 None 2026-03-27 09:42:54,952 - lava_callback - DEBUG - Processing node 69c5ee572cb62322ad9690dd: baseline-arm64 2026-03-27 09:42:55,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:42:55,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:42:55,581 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5ee572cb62322ad9690dd/log.txt.gz 2026-03-27 09:42:55,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:42:56,137 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:42:56,139 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5ee572cb62322ad9690dd/lava-logs.yaml 2026-03-27 09:42:56,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:42:56,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:42:56,323 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5ee572cb62322ad9690dd/lava_callback.json.gz 2026-03-27 09:42:56,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:42:56,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5ee572cb62322ad9690dd HTTP/1.1" 200 None 2026-03-27 09:42:56,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:42:56,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 09:42:56,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:42:56,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5ee572cb62322ad9690dd HTTP/1.1" 200 None 2026-03-27 09:42:56,471 - lava_callback - INFO - Completed processing callback for node 69c5ee572cb62322ad9690dd INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:43:03,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:43:03,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39134 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 09:46:15,403 - lava_callback - INFO - Received callback for node 69c5eed42cb62322ad969107 INFO: 172.20.0.1:39140 - "POST /node/69c5eed42cb62322ad969107 HTTP/1.0" 202 Accepted 2026-03-27 09:46:15,410 - lava_callback - INFO - Starting async job submit for node 69c5eed42cb62322ad969107 2026-03-27 09:46:15,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:15,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eed42cb62322ad969107 HTTP/1.1" 200 None 2026-03-27 09:46:15,454 - lava_callback - DEBUG - Processing node 69c5eed42cb62322ad969107: baseline-nfs-arm64 2026-03-27 09:46:15,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:46:16,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:46:16,122 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5eed42cb62322ad969107/log.txt.gz 2026-03-27 09:46:16,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39152 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:46:16,797 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:46:16,799 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5eed42cb62322ad969107/lava-logs.yaml 2026-03-27 09:46:16,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:46:16,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:46:16,974 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5eed42cb62322ad969107/lava_callback.json.gz 2026-03-27 09:46:17,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:17,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eed42cb62322ad969107 HTTP/1.1" 200 None 2026-03-27 09:46:17,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:17,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 09:46:17,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:17,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5eed42cb62322ad969107 HTTP/1.1" 200 None 2026-03-27 09:46:17,141 - lava_callback - INFO - Completed processing callback for node 69c5eed42cb62322ad969107 INFO: 172.20.0.1:48038 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:46:23,342 - lava_callback - INFO - Received callback for node 69c3aee4bbc022e792644e1e INFO: 172.20.0.1:48050 - "POST /node/69c3aee4bbc022e792644e1e HTTP/1.0" 202 Accepted 2026-03-27 09:46:23,348 - lava_callback - INFO - Starting async job submit for node 69c3aee4bbc022e792644e1e 2026-03-27 09:46:23,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:23,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3aee4bbc022e792644e1e HTTP/1.1" 200 None 2026-03-27 09:46:23,364 - lava_callback - DEBUG - Processing node 69c3aee4bbc022e792644e1e: baseline-arm-clabbe 2026-03-27 09:46:23,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3aee4bbc022e792644e1e 2026-03-27 09:46:23,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:23,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3aee4bbc022e792644e1e HTTP/1.1" 200 None 2026-03-27 09:46:23,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:23,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:46:23,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:23,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3aee4bbc022e792644e1e HTTP/1.1" 200 None 2026-03-27 09:46:23,418 - lava_callback - ERROR - Error processing callback for node 69c3aee4bbc022e792644e1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48066 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:46:33,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:33,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 09:46:33,744 - lava_callback - INFO - Received callback for node 69c651d5b98d4d3c02068d34 INFO: 172.20.0.1:50892 - "POST /node/69c651d5b98d4d3c02068d34 HTTP/1.0" 202 Accepted 2026-03-27 09:46:33,750 - lava_callback - INFO - Starting async job submit for node 69c651d5b98d4d3c02068d34 2026-03-27 09:46:33,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:33,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c651d5b98d4d3c02068d34 HTTP/1.1" 200 None 2026-03-27 09:46:33,783 - lava_callback - DEBUG - Processing node 69c651d5b98d4d3c02068d34: baseline-arm64 2026-03-27 09:46:33,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:46:34,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:46:34,054 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c651d5b98d4d3c02068d34/log.txt.gz 2026-03-27 09:46:34,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:46:34,321 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:46:34,323 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c651d5b98d4d3c02068d34/lava-logs.yaml 2026-03-27 09:46:34,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:46:34,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:46:34,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c651d5b98d4d3c02068d34/lava_callback.json.gz 2026-03-27 09:46:34,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:34,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c651d5b98d4d3c02068d34 HTTP/1.1" 200 None 2026-03-27 09:46:34,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:34,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:46:34,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:34,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c651d5b98d4d3c02068d34 HTTP/1.1" 200 None 2026-03-27 09:46:34,556 - lava_callback - INFO - Completed processing callback for node 69c651d5b98d4d3c02068d34 INFO: 172.20.0.1:50908 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:46:38,190 - lava_callback - INFO - Received callback for node 69c651dab98d4d3c02068d37 INFO: 172.20.0.1:50912 - "POST /node/69c651dab98d4d3c02068d37 HTTP/1.0" 202 Accepted 2026-03-27 09:46:38,196 - lava_callback - INFO - Starting async job submit for node 69c651dab98d4d3c02068d37 2026-03-27 09:46:38,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:38,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c651dab98d4d3c02068d37 HTTP/1.1" 200 None 2026-03-27 09:46:38,230 - lava_callback - DEBUG - Processing node 69c651dab98d4d3c02068d37: baseline-arm64 2026-03-27 09:46:38,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:46:38,493 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:46:38,495 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c651dab98d4d3c02068d37/log.txt.gz 2026-03-27 09:46:38,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:46:38,781 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:46:38,783 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c651dab98d4d3c02068d37/lava-logs.yaml 2026-03-27 09:46:38,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:46:38,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:46:38,923 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c651dab98d4d3c02068d37/lava_callback.json.gz 2026-03-27 09:46:38,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:38,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c651dab98d4d3c02068d37 HTTP/1.1" 200 None 2026-03-27 09:46:38,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:38,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:46:38,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:46:38,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c651dab98d4d3c02068d37 HTTP/1.1" 200 None 2026-03-27 09:46:39,014 - lava_callback - INFO - Completed processing callback for node 69c651dab98d4d3c02068d37 INFO: 172.20.0.1:53938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36690 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:47:03,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:47:03,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57412 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:48:16,958 - lava_callback - INFO - Received callback for node 69c3af51bbc022e792644e35 INFO: 172.20.0.1:57418 - "POST /node/69c3af51bbc022e792644e35 HTTP/1.0" 202 Accepted 2026-03-27 09:48:16,963 - lava_callback - INFO - Starting async job submit for node 69c3af51bbc022e792644e35 2026-03-27 09:48:16,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:48:16,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3af51bbc022e792644e35 HTTP/1.1" 200 None 2026-03-27 09:48:16,986 - lava_callback - DEBUG - Processing node 69c3af51bbc022e792644e35: baseline-arm-baylibre 2026-03-27 09:48:16,986 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3af51bbc022e792644e35 2026-03-27 09:48:16,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:48:17,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3af51bbc022e792644e35 HTTP/1.1" 200 None 2026-03-27 09:48:17,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:48:17,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:48:17,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:48:17,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3af51bbc022e792644e35 HTTP/1.1" 200 None 2026-03-27 09:48:17,041 - lava_callback - ERROR - Error processing callback for node 69c3af51bbc022e792644e35: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 09:48:21,354 - lava_callback - INFO - Received callback for node 69c5eff92cb62322ad969175 INFO: 172.20.0.1:42236 - "POST /node/69c5eff92cb62322ad969175 HTTP/1.0" 202 Accepted 2026-03-27 09:48:21,361 - lava_callback - INFO - Starting async job submit for node 69c5eff92cb62322ad969175 2026-03-27 09:48:21,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:48:21,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eff92cb62322ad969175 HTTP/1.1" 200 None 2026-03-27 09:48:21,412 - lava_callback - DEBUG - Processing node 69c5eff92cb62322ad969175: baseline-arm64 INFO: 172.20.0.1:42238 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:48:21,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:48:22,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:48:22,106 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5eff92cb62322ad969175/log.txt.gz 2026-03-27 09:48:22,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:48:22,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:48:22,608 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5eff92cb62322ad969175/lava-logs.yaml 2026-03-27 09:48:22,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:48:22,788 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:48:22,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5eff92cb62322ad969175/lava_callback.json.gz 2026-03-27 09:48:22,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:48:22,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5eff92cb62322ad969175 HTTP/1.1" 200 None 2026-03-27 09:48:22,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:48:22,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 09:48:22,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:48:22,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5eff92cb62322ad969175 HTTP/1.1" 200 None 2026-03-27 09:48:22,938 - lava_callback - INFO - Completed processing callback for node 69c5eff92cb62322ad969175 INFO: 172.20.0.1:42252 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:48:33,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:48:33,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51920 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:50:02,019 - lava_callback - INFO - Received callback for node 69c652c3b98d4d3c02068d76 INFO: 172.20.0.1:41830 - "POST /node/69c652c3b98d4d3c02068d76 HTTP/1.0" 202 Accepted 2026-03-27 09:50:02,025 - lava_callback - INFO - Starting async job submit for node 69c652c3b98d4d3c02068d76 2026-03-27 09:50:02,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:50:02,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c652c3b98d4d3c02068d76 HTTP/1.1" 200 None 2026-03-27 09:50:02,044 - lava_callback - DEBUG - Processing node 69c652c3b98d4d3c02068d76: baseline-nfs-arm64 2026-03-27 09:50:02,046 - lava_callback - WARNING - Failed to upload log 2026-03-27 09:50:02,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:50:02,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:50:02,194 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c652c3b98d4d3c02068d76/lava-logs.yaml 2026-03-27 09:50:02,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:50:02,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:50:02,330 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c652c3b98d4d3c02068d76/lava_callback.json.gz 2026-03-27 09:50:02,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:50:02,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c652c3b98d4d3c02068d76 HTTP/1.1" 200 None 2026-03-27 09:50:02,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:50:02,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:50:02,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:50:02,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c652c3b98d4d3c02068d76 HTTP/1.1" 200 None 2026-03-27 09:50:02,396 - lava_callback - INFO - Completed processing callback for node 69c652c3b98d4d3c02068d76 2026-03-27 09:50:03,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:50:03,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53572 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:51:23,510 - lava_callback - INFO - Received callback for node 69c3b00dbbc022e792644e57 INFO: 172.20.0.1:53576 - "POST /node/69c3b00dbbc022e792644e57 HTTP/1.0" 202 Accepted 2026-03-27 09:51:23,516 - lava_callback - INFO - Starting async job submit for node 69c3b00dbbc022e792644e57 2026-03-27 09:51:23,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:51:23,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b00dbbc022e792644e57 HTTP/1.1" 200 None 2026-03-27 09:51:23,532 - lava_callback - DEBUG - Processing node 69c3b00dbbc022e792644e57: baseline-arm-clabbe 2026-03-27 09:51:23,532 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b00dbbc022e792644e57 2026-03-27 09:51:23,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:51:23,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b00dbbc022e792644e57 HTTP/1.1" 200 None 2026-03-27 09:51:23,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:51:23,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:51:23,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:51:23,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b00dbbc022e792644e57 HTTP/1.1" 200 None 2026-03-27 09:51:23,595 - lava_callback - ERROR - Error processing callback for node 69c3b00dbbc022e792644e57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39006 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:52:06,435 - lava_callback - INFO - Received callback for node 69c64d8eb98d4d3c02068be3 INFO: 172.20.0.1:53750 - "POST /node/69c64d8eb98d4d3c02068be3 HTTP/1.0" 202 Accepted 2026-03-27 09:52:06,481 - lava_callback - INFO - Starting async job submit for node 69c64d8eb98d4d3c02068be3 2026-03-27 09:52:06,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:53760 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:52:06,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64d8eb98d4d3c02068be3 HTTP/1.1" 200 None 2026-03-27 09:52:06,520 - lava_callback - DEBUG - Processing node 69c64d8eb98d4d3c02068be3: baseline-arm-pengutronix 2026-03-27 09:52:06,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:52:06,742 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:52:06,744 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c64d8eb98d4d3c02068be3/log.txt.gz 2026-03-27 09:52:06,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:52:06,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:52:06,989 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c64d8eb98d4d3c02068be3/lava-logs.yaml 2026-03-27 09:52:06,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:52:07,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:52:07,132 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c64d8eb98d4d3c02068be3/lava_callback.json.gz DEBUG: Setup failed for 69c64d8eb98d4d3c02068be3. Transitting job node result: incomplete -> fail 2026-03-27 09:52:07,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:52:07,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64d8eb98d4d3c02068be3 HTTP/1.1" 200 None 2026-03-27 09:52:07,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:52:07,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 09:52:07,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:52:07,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64d8eb98d4d3c02068be3 HTTP/1.1" 200 None 2026-03-27 09:52:07,302 - lava_callback - INFO - Completed processing callback for node 69c64d8eb98d4d3c02068be3 INFO: 172.20.0.1:34838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39718 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:52:33,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:52:33,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46734 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:53:17,113 - lava_callback - INFO - Received callback for node 69c65353b98d4d3c02068d98 INFO: 172.20.0.1:46744 - "POST /node/69c65353b98d4d3c02068d98 HTTP/1.0" 202 Accepted 2026-03-27 09:53:17,120 - lava_callback - INFO - Starting async job submit for node 69c65353b98d4d3c02068d98 2026-03-27 09:53:17,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:17,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65353b98d4d3c02068d98 HTTP/1.1" 200 None 2026-03-27 09:53:17,164 - lava_callback - DEBUG - Processing node 69c65353b98d4d3c02068d98: baseline-arm64 2026-03-27 09:53:17,304 - lava_callback - WARNING - Failed to upload log 2026-03-27 09:53:17,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:53:17,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:53:17,611 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65353b98d4d3c02068d98/lava-logs.yaml 2026-03-27 09:53:17,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:53:17,766 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:53:17,767 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65353b98d4d3c02068d98/lava_callback.json.gz 2026-03-27 09:53:17,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:17,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65353b98d4d3c02068d98 HTTP/1.1" 200 None 2026-03-27 09:53:17,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:17,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:53:17,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:17,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65353b98d4d3c02068d98 HTTP/1.1" 200 None 2026-03-27 09:53:17,917 - lava_callback - INFO - Completed processing callback for node 69c65353b98d4d3c02068d98 INFO: 172.20.0.1:38668 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:53:23,595 - lava_callback - INFO - Received callback for node 69c3b07dbbc022e792644e72 INFO: 172.20.0.1:38680 - "POST /node/69c3b07dbbc022e792644e72 HTTP/1.0" 202 Accepted 2026-03-27 09:53:23,601 - lava_callback - INFO - Starting async job submit for node 69c3b07dbbc022e792644e72 2026-03-27 09:53:23,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:23,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b07dbbc022e792644e72 HTTP/1.1" 200 None 2026-03-27 09:53:23,616 - lava_callback - DEBUG - Processing node 69c3b07dbbc022e792644e72: baseline-arm-clabbe 2026-03-27 09:53:23,617 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b07dbbc022e792644e72 2026-03-27 09:53:23,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:23,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b07dbbc022e792644e72 HTTP/1.1" 200 None 2026-03-27 09:53:23,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:23,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:53:23,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:23,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b07dbbc022e792644e72 HTTP/1.1" 200 None 2026-03-27 09:53:23,678 - lava_callback - ERROR - Error processing callback for node 69c3b07dbbc022e792644e72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38682 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:53:33,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:33,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50962 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:53:43,536 - lava_callback - INFO - Received callback for node 69c3b0a3bbc022e792644e82 INFO: 172.20.0.1:36612 - "POST /node/69c3b0a3bbc022e792644e82 HTTP/1.0" 202 Accepted 2026-03-27 09:53:43,541 - lava_callback - INFO - Starting async job submit for node 69c3b0a3bbc022e792644e82 2026-03-27 09:53:43,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:43,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b0a3bbc022e792644e82 HTTP/1.1" 200 None 2026-03-27 09:53:43,579 - lava_callback - DEBUG - Processing node 69c3b0a3bbc022e792644e82: baseline-arm-clabbe 2026-03-27 09:53:43,579 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b0a3bbc022e792644e82 2026-03-27 09:53:43,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:43,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b0a3bbc022e792644e82 HTTP/1.1" 200 None 2026-03-27 09:53:43,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:43,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:53:43,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:53:43,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b0a3bbc022e792644e82 HTTP/1.1" 200 None 2026-03-27 09:53:43,639 - lava_callback - ERROR - Error processing callback for node 69c3b0a3bbc022e792644e82: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46054 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:54:39,429 - lava_callback - INFO - Received callback for node 69c653a2b98d4d3c02068daa INFO: 172.20.0.1:46058 - "POST /node/69c653a2b98d4d3c02068daa HTTP/1.0" 202 Accepted 2026-03-27 09:54:39,436 - lava_callback - INFO - Starting async job submit for node 69c653a2b98d4d3c02068daa 2026-03-27 09:54:39,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:54:39,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c653a2b98d4d3c02068daa HTTP/1.1" 200 None 2026-03-27 09:54:39,473 - lava_callback - DEBUG - Processing node 69c653a2b98d4d3c02068daa: baseline-arm64 2026-03-27 09:54:39,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:54:40,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:54:40,163 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c653a2b98d4d3c02068daa/log.txt.gz 2026-03-27 09:54:40,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:54:40,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:54:40,614 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c653a2b98d4d3c02068daa/lava-logs.yaml 2026-03-27 09:54:40,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 09:54:40,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 09:54:40,770 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c653a2b98d4d3c02068daa/lava_callback.json.gz 2026-03-27 09:54:40,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:54:40,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c653a2b98d4d3c02068daa HTTP/1.1" 200 None 2026-03-27 09:54:40,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:54:40,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 09:54:40,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:54:40,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c653a2b98d4d3c02068daa HTTP/1.1" 200 None 2026-03-27 09:54:40,920 - lava_callback - INFO - Completed processing callback for node 69c653a2b98d4d3c02068daa INFO: 172.20.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41654 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:55:03,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:55:03,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53928 - "GET / HTTP/1.0" 200 OK 2026-03-27 09:57:03,713 - lava_callback - INFO - Received callback for node 69c3b15ebbc022e792644ea9 INFO: 172.20.0.1:44444 - "POST /node/69c3b15ebbc022e792644ea9 HTTP/1.0" 202 Accepted 2026-03-27 09:57:03,720 - lava_callback - INFO - Starting async job submit for node 69c3b15ebbc022e792644ea9 2026-03-27 09:57:03,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:57:03,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b15ebbc022e792644ea9 HTTP/1.1" 200 None 2026-03-27 09:57:03,736 - lava_callback - DEBUG - Processing node 69c3b15ebbc022e792644ea9: baseline-arm-clabbe 2026-03-27 09:57:03,736 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b15ebbc022e792644ea9 2026-03-27 09:57:03,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:57:03,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b15ebbc022e792644ea9 HTTP/1.1" 200 None 2026-03-27 09:57:03,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:57:03,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb130ec3f435d20d51a98 HTTP/1.1" 200 None 2026-03-27 09:57:03,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 09:57:03,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b15ebbc022e792644ea9 HTTP/1.1" 200 None 2026-03-27 09:57:03,801 - lava_callback - ERROR - Error processing callback for node 69c3b15ebbc022e792644ea9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60206 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:00:17,564 - lava_callback - INFO - Received callback for node 69c3b21fbbc022e792644ed0 INFO: 172.20.0.1:60210 - "POST /node/69c3b21fbbc022e792644ed0 HTTP/1.0" 202 Accepted 2026-03-27 10:00:17,570 - lava_callback - INFO - Starting async job submit for node 69c3b21fbbc022e792644ed0 2026-03-27 10:00:17,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:00:17,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b21fbbc022e792644ed0 HTTP/1.1" 200 None 2026-03-27 10:00:17,592 - lava_callback - DEBUG - Processing node 69c3b21fbbc022e792644ed0: baseline-arm64-baylibre 2026-03-27 10:00:17,592 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b21fbbc022e792644ed0 2026-03-27 10:00:17,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:00:17,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b21fbbc022e792644ed0 HTTP/1.1" 200 None 2026-03-27 10:00:17,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:00:17,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-27 10:00:17,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:00:17,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b21fbbc022e792644ed0 HTTP/1.1" 200 None 2026-03-27 10:00:17,659 - lava_callback - ERROR - Error processing callback for node 69c3b21fbbc022e792644ed0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36512 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:00:52,347 - lava_callback - INFO - Received callback for node 69c654d3b98d4d3c02068de8 INFO: 172.20.0.1:37968 - "POST /node/69c654d3b98d4d3c02068de8 HTTP/1.0" 202 Accepted 2026-03-27 10:00:52,354 - lava_callback - INFO - Starting async job submit for node 69c654d3b98d4d3c02068de8 2026-03-27 10:00:52,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:00:52,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c654d3b98d4d3c02068de8 HTTP/1.1" 200 None 2026-03-27 10:00:52,406 - lava_callback - DEBUG - Processing node 69c654d3b98d4d3c02068de8: baseline-arm64 2026-03-27 10:00:52,606 - lava_callback - WARNING - Failed to upload log 2026-03-27 10:00:52,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:00:52,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:00:52,937 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c654d3b98d4d3c02068de8/lava-logs.yaml 2026-03-27 10:00:52,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:00:53,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:00:53,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c654d3b98d4d3c02068de8/lava_callback.json.gz 2026-03-27 10:00:53,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:00:53,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c654d3b98d4d3c02068de8 HTTP/1.1" 200 None 2026-03-27 10:00:53,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:00:53,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:00:53,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:00:53,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c654d3b98d4d3c02068de8 HTTP/1.1" 200 None 2026-03-27 10:00:53,243 - lava_callback - INFO - Completed processing callback for node 69c654d3b98d4d3c02068de8 INFO: 172.20.0.1:37970 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:01:03,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:01:03,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43938 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:01:37,613 - lava_callback - INFO - Received callback for node 69c3b26dbbc022e792644ede INFO: 172.20.0.1:43948 - "POST /node/69c3b26dbbc022e792644ede HTTP/1.0" 202 Accepted 2026-03-27 10:01:37,618 - lava_callback - INFO - Starting async job submit for node 69c3b26dbbc022e792644ede 2026-03-27 10:01:37,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:01:37,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b26dbbc022e792644ede HTTP/1.1" 200 None 2026-03-27 10:01:37,634 - lava_callback - DEBUG - Processing node 69c3b26dbbc022e792644ede: baseline-arm64-baylibre 2026-03-27 10:01:37,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b26dbbc022e792644ede 2026-03-27 10:01:37,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:01:37,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b26dbbc022e792644ede HTTP/1.1" 200 None 2026-03-27 10:01:37,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:01:37,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-27 10:01:37,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:01:37,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b26dbbc022e792644ede HTTP/1.1" 200 None 2026-03-27 10:01:37,716 - lava_callback - ERROR - Error processing callback for node 69c3b26dbbc022e792644ede: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36436 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:05:17,873 - lava_callback - INFO - Received callback for node 69c3b34dbbc022e792644f0a INFO: 172.20.0.1:36442 - "POST /node/69c3b34dbbc022e792644f0a HTTP/1.0" 202 Accepted 2026-03-27 10:05:17,878 - lava_callback - INFO - Starting async job submit for node 69c3b34dbbc022e792644f0a 2026-03-27 10:05:17,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:05:17,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b34dbbc022e792644f0a HTTP/1.1" 200 None 2026-03-27 10:05:17,903 - lava_callback - DEBUG - Processing node 69c3b34dbbc022e792644f0a: baseline-arm64-baylibre 2026-03-27 10:05:17,904 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b34dbbc022e792644f0a 2026-03-27 10:05:17,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:05:17,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b34dbbc022e792644f0a HTTP/1.1" 200 None 2026-03-27 10:05:17,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:05:17,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-27 10:05:17,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:05:17,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b34dbbc022e792644f0a HTTP/1.1" 200 None 2026-03-27 10:05:17,956 - lava_callback - ERROR - Error processing callback for node 69c3b34dbbc022e792644f0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44126 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:08:59,597 - lava_callback - INFO - Received callback for node 69c65576b98d4d3c02068e08 INFO: 172.20.0.1:44134 - "POST /node/69c65576b98d4d3c02068e08 HTTP/1.0" 202 Accepted 2026-03-27 10:08:59,604 - lava_callback - INFO - Starting async job submit for node 69c65576b98d4d3c02068e08 2026-03-27 10:08:59,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:08:59,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65576b98d4d3c02068e08 HTTP/1.1" 200 None 2026-03-27 10:08:59,654 - lava_callback - DEBUG - Processing node 69c65576b98d4d3c02068e08: baseline-arm64 2026-03-27 10:09:00,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:09:00,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:09:00,180 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65576b98d4d3c02068e08/log.txt.gz 2026-03-27 10:09:00,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:09:00,592 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:09:00,594 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65576b98d4d3c02068e08/lava-logs.yaml 2026-03-27 10:09:00,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:09:00,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:09:00,811 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65576b98d4d3c02068e08/lava_callback.json.gz DEBUG: Setup failed for 69c65576b98d4d3c02068e08. Transitting job node result: incomplete -> fail 2026-03-27 10:09:00,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:09:00,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65576b98d4d3c02068e08 HTTP/1.1" 200 None 2026-03-27 10:09:00,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:09:00,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:09:00,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:09:00,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65576b98d4d3c02068e08 HTTP/1.1" 200 None 2026-03-27 10:09:01,031 - lava_callback - INFO - Completed processing callback for node 69c65576b98d4d3c02068e08 2026-03-27 10:09:03,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:09:03,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56832 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:10:13,389 - lava_callback - INFO - Received callback for node 69c5f44a2cb62322ad9691e5 INFO: 172.20.0.1:52038 - "POST /node/69c5f44a2cb62322ad9691e5 HTTP/1.0" 202 Accepted 2026-03-27 10:10:13,396 - lava_callback - INFO - Starting async job submit for node 69c5f44a2cb62322ad9691e5 2026-03-27 10:10:13,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:10:13,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f44a2cb62322ad9691e5 HTTP/1.1" 200 None 2026-03-27 10:10:13,436 - lava_callback - DEBUG - Processing node 69c5f44a2cb62322ad9691e5: baseline-nfs-arm64 2026-03-27 10:10:13,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:10:14,105 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:10:14,107 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5f44a2cb62322ad9691e5/log.txt.gz 2026-03-27 10:10:14,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:10:14,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:10:14,595 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5f44a2cb62322ad9691e5/lava-logs.yaml 2026-03-27 10:10:14,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:10:14,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:10:14,796 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5f44a2cb62322ad9691e5/lava_callback.json.gz 2026-03-27 10:10:14,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:10:14,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f44a2cb62322ad9691e5 HTTP/1.1" 200 None 2026-03-27 10:10:14,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:10:14,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 10:10:14,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:10:14,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f44a2cb62322ad9691e5 HTTP/1.1" 200 None 2026-03-27 10:10:14,955 - lava_callback - INFO - Completed processing callback for node 69c5f44a2cb62322ad9691e5 INFO: 172.20.0.1:52054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52070 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:10:18,133 - lava_callback - INFO - Received callback for node 69c3b476bbc022e792644f3b INFO: 172.20.0.1:52074 - "POST /node/69c3b476bbc022e792644f3b HTTP/1.0" 202 Accepted 2026-03-27 10:10:18,138 - lava_callback - INFO - Starting async job submit for node 69c3b476bbc022e792644f3b 2026-03-27 10:10:18,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:10:18,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b476bbc022e792644f3b HTTP/1.1" 200 None 2026-03-27 10:10:18,154 - lava_callback - DEBUG - Processing node 69c3b476bbc022e792644f3b: baseline-arm64-baylibre 2026-03-27 10:10:18,155 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b476bbc022e792644f3b 2026-03-27 10:10:18,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:10:18,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b476bbc022e792644f3b HTTP/1.1" 200 None 2026-03-27 10:10:18,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:10:18,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-27 10:10:18,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:10:18,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b476bbc022e792644f3b HTTP/1.1" 200 None 2026-03-27 10:10:18,213 - lava_callback - ERROR - Error processing callback for node 69c3b476bbc022e792644f3b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55408 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:10:33,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:10:33,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50448 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:12:18,177 - lava_callback - INFO - Received callback for node 69c5f65f2cb62322ad969223 INFO: 172.20.0.1:50454 - "POST /node/69c5f65f2cb62322ad969223 HTTP/1.0" 202 Accepted 2026-03-27 10:12:18,184 - lava_callback - INFO - Starting async job submit for node 69c5f65f2cb62322ad969223 2026-03-27 10:12:18,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:12:18,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f65f2cb62322ad969223 HTTP/1.1" 200 None 2026-03-27 10:12:18,223 - lava_callback - DEBUG - Processing node 69c5f65f2cb62322ad969223: baseline-arm64 2026-03-27 10:12:18,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:12:18,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:12:18,919 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5f65f2cb62322ad969223/log.txt.gz 2026-03-27 10:12:19,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:12:19,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:12:19,438 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5f65f2cb62322ad969223/lava-logs.yaml 2026-03-27 10:12:19,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:12:19,614 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:12:19,616 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5f65f2cb62322ad969223/lava_callback.json.gz 2026-03-27 10:12:19,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:12:19,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f65f2cb62322ad969223 HTTP/1.1" 200 None 2026-03-27 10:12:19,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:12:19,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 10:12:19,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:12:19,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f65f2cb62322ad969223 HTTP/1.1" 200 None 2026-03-27 10:12:19,758 - lava_callback - INFO - Completed processing callback for node 69c5f65f2cb62322ad969223 INFO: 172.20.0.1:40274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40284 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:12:33,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:12:33,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58998 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:14:23,530 - lava_callback - INFO - Received callback for node 69c6586bb98d4d3c02068e9a INFO: 172.20.0.1:59014 - "POST /node/69c6586bb98d4d3c02068e9a HTTP/1.0" 202 Accepted 2026-03-27 10:14:23,536 - lava_callback - INFO - Starting async job submit for node 69c6586bb98d4d3c02068e9a 2026-03-27 10:14:23,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:14:23,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6586bb98d4d3c02068e9a HTTP/1.1" 200 None 2026-03-27 10:14:23,554 - lava_callback - DEBUG - Processing node 69c6586bb98d4d3c02068e9a: baseline-nfs-arm64 2026-03-27 10:14:23,555 - lava_callback - WARNING - Failed to upload log 2026-03-27 10:14:23,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:14:23,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:14:23,868 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6586bb98d4d3c02068e9a/lava-logs.yaml 2026-03-27 10:14:23,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:14:24,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:14:24,011 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6586bb98d4d3c02068e9a/lava_callback.json.gz 2026-03-27 10:14:24,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:14:24,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6586bb98d4d3c02068e9a HTTP/1.1" 200 None 2026-03-27 10:14:24,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:14:24,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:14:24,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:14:24,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6586bb98d4d3c02068e9a HTTP/1.1" 200 None 2026-03-27 10:14:24,076 - lava_callback - INFO - Completed processing callback for node 69c6586bb98d4d3c02068e9a 2026-03-27 10:14:24,608 - lava_callback - INFO - Received callback for node 69c3b57dbbc022e792644f70 INFO: 172.20.0.1:59028 - "POST /node/69c3b57dbbc022e792644f70 HTTP/1.0" 202 Accepted 2026-03-27 10:14:24,614 - lava_callback - INFO - Starting async job submit for node 69c3b57dbbc022e792644f70 2026-03-27 10:14:24,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:14:24,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b57dbbc022e792644f70 HTTP/1.1" 200 None 2026-03-27 10:14:24,630 - lava_callback - DEBUG - Processing node 69c3b57dbbc022e792644f70: baseline-arm64-clabbe 2026-03-27 10:14:24,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b57dbbc022e792644f70 2026-03-27 10:14:24,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:14:24,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b57dbbc022e792644f70 HTTP/1.1" 200 None 2026-03-27 10:14:24,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:14:24,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-27 10:14:24,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:14:24,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b57dbbc022e792644f70 HTTP/1.1" 200 None 2026-03-27 10:14:24,692 - lava_callback - ERROR - Error processing callback for node 69c3b57dbbc022e792644f70: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59034 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:14:33,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:14:33,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44308 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:16:37,856 - lava_callback - INFO - Received callback for node 69c658d5b98d4d3c02068eb1 INFO: 172.20.0.1:44312 - "POST /node/69c658d5b98d4d3c02068eb1 HTTP/1.0" 202 Accepted 2026-03-27 10:16:37,863 - lava_callback - INFO - Starting async job submit for node 69c658d5b98d4d3c02068eb1 2026-03-27 10:16:37,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:16:37,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c658d5b98d4d3c02068eb1 HTTP/1.1" 200 None 2026-03-27 10:16:37,918 - lava_callback - DEBUG - Processing node 69c658d5b98d4d3c02068eb1: baseline-arm64 2026-03-27 10:16:38,053 - lava_callback - WARNING - Failed to upload log 2026-03-27 10:16:38,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:16:38,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:16:38,319 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c658d5b98d4d3c02068eb1/lava-logs.yaml 2026-03-27 10:16:38,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:16:38,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:16:38,479 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c658d5b98d4d3c02068eb1/lava_callback.json.gz 2026-03-27 10:16:38,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:16:38,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c658d5b98d4d3c02068eb1 HTTP/1.1" 200 None 2026-03-27 10:16:38,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:16:38,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:16:38,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:16:38,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c658d5b98d4d3c02068eb1 HTTP/1.1" 200 None 2026-03-27 10:16:38,652 - lava_callback - INFO - Completed processing callback for node 69c658d5b98d4d3c02068eb1 INFO: 172.20.0.1:45962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39634 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:17:03,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:17:03,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 10:17:04,564 - lava_callback - INFO - Received callback for node 69c3b613bbc022e792644f84 INFO: 172.20.0.1:44508 - "POST /node/69c3b613bbc022e792644f84 HTTP/1.0" 202 Accepted 2026-03-27 10:17:04,570 - lava_callback - INFO - Starting async job submit for node 69c3b613bbc022e792644f84 2026-03-27 10:17:04,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:17:04,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b613bbc022e792644f84 HTTP/1.1" 200 None 2026-03-27 10:17:04,591 - lava_callback - DEBUG - Processing node 69c3b613bbc022e792644f84: baseline-arm64-clabbe 2026-03-27 10:17:04,591 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b613bbc022e792644f84 2026-03-27 10:17:04,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:17:04,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b613bbc022e792644f84 HTTP/1.1" 200 None 2026-03-27 10:17:04,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:17:04,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-27 10:17:04,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:17:04,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b613bbc022e792644f84 HTTP/1.1" 200 None 2026-03-27 10:17:04,654 - lava_callback - ERROR - Error processing callback for node 69c3b613bbc022e792644f84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40590 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:18:13,893 - lava_callback - INFO - Received callback for node 69c65922b98d4d3c02068ebe INFO: 172.20.0.1:56718 - "POST /node/69c65922b98d4d3c02068ebe HTTP/1.0" 202 Accepted 2026-03-27 10:18:13,899 - lava_callback - INFO - Starting async job submit for node 69c65922b98d4d3c02068ebe 2026-03-27 10:18:13,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:18:13,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65922b98d4d3c02068ebe HTTP/1.1" 200 None 2026-03-27 10:18:13,961 - lava_callback - DEBUG - Processing node 69c65922b98d4d3c02068ebe: baseline-arm64 2026-03-27 10:18:14,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:18:14,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:18:14,573 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65922b98d4d3c02068ebe/log.txt.gz 2026-03-27 10:18:14,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:18:14,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:18:14,979 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65922b98d4d3c02068ebe/lava-logs.yaml 2026-03-27 10:18:14,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:18:15,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:18:15,141 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65922b98d4d3c02068ebe/lava_callback.json.gz INFO: 172.20.0.1:56732 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 10:18:15,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:18:15,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65922b98d4d3c02068ebe HTTP/1.1" 200 None 2026-03-27 10:18:15,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:18:15,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:18:15,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:18:15,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65922b98d4d3c02068ebe HTTP/1.1" 200 None 2026-03-27 10:18:15,308 - lava_callback - INFO - Completed processing callback for node 69c65922b98d4d3c02068ebe INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33008 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:18:24,609 - lava_callback - INFO - Received callback for node 69c3b65ebbc022e792644f90 INFO: 172.20.0.1:33012 - "POST /node/69c3b65ebbc022e792644f90 HTTP/1.0" 202 Accepted 2026-03-27 10:18:24,614 - lava_callback - INFO - Starting async job submit for node 69c3b65ebbc022e792644f90 2026-03-27 10:18:24,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:18:24,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b65ebbc022e792644f90 HTTP/1.1" 200 None 2026-03-27 10:18:24,630 - lava_callback - DEBUG - Processing node 69c3b65ebbc022e792644f90: baseline-arm64-clabbe 2026-03-27 10:18:24,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b65ebbc022e792644f90 2026-03-27 10:18:24,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:18:24,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b65ebbc022e792644f90 HTTP/1.1" 200 None 2026-03-27 10:18:24,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:18:24,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-27 10:18:24,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:18:24,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b65ebbc022e792644f90 HTTP/1.1" 200 None 2026-03-27 10:18:24,712 - lava_callback - ERROR - Error processing callback for node 69c3b65ebbc022e792644f90: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33028 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:18:33,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:18:33,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55788 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:19:34,645 - lava_callback - INFO - Received callback for node 69c6596cb98d4d3c02068ece INFO: 172.20.0.1:57772 - "POST /node/69c6596cb98d4d3c02068ece HTTP/1.0" 202 Accepted 2026-03-27 10:19:34,652 - lava_callback - INFO - Starting async job submit for node 69c6596cb98d4d3c02068ece 2026-03-27 10:19:34,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:19:34,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6596cb98d4d3c02068ece HTTP/1.1" 200 None 2026-03-27 10:19:34,707 - lava_callback - DEBUG - Processing node 69c6596cb98d4d3c02068ece: baseline-arm64 2026-03-27 10:19:34,845 - lava_callback - WARNING - Failed to upload log 2026-03-27 10:19:34,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:19:35,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:19:35,144 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6596cb98d4d3c02068ece/lava-logs.yaml 2026-03-27 10:19:35,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:19:35,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:19:35,289 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6596cb98d4d3c02068ece/lava_callback.json.gz 2026-03-27 10:19:35,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:19:35,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6596cb98d4d3c02068ece HTTP/1.1" 200 None 2026-03-27 10:19:35,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:19:35,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:19:35,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:19:35,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6596cb98d4d3c02068ece HTTP/1.1" 200 None 2026-03-27 10:19:35,449 - lava_callback - INFO - Completed processing callback for node 69c6596cb98d4d3c02068ece INFO: 172.20.0.1:57780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35176 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:20:03,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:20:03,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43552 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:20:44,724 - lava_callback - INFO - Received callback for node 69c3b6f6bbc022e792644fa3 INFO: 172.20.0.1:51384 - "POST /node/69c3b6f6bbc022e792644fa3 HTTP/1.0" 202 Accepted 2026-03-27 10:20:44,730 - lava_callback - INFO - Starting async job submit for node 69c3b6f6bbc022e792644fa3 2026-03-27 10:20:44,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:20:44,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b6f6bbc022e792644fa3 HTTP/1.1" 200 None 2026-03-27 10:20:44,746 - lava_callback - DEBUG - Processing node 69c3b6f6bbc022e792644fa3: baseline-arm64-clabbe 2026-03-27 10:20:44,746 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b6f6bbc022e792644fa3 2026-03-27 10:20:44,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:20:44,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b6f6bbc022e792644fa3 HTTP/1.1" 200 None 2026-03-27 10:20:44,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:20:44,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-27 10:20:44,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:20:44,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b6f6bbc022e792644fa3 HTTP/1.1" 200 None 2026-03-27 10:20:44,806 - lava_callback - ERROR - Error processing callback for node 69c3b6f6bbc022e792644fa3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47898 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:22:33,215 - lava_callback - INFO - Received callback for node 69c5f6b82cb62322ad969230 INFO: 172.20.0.1:39300 - "POST /node/69c5f6b82cb62322ad969230 HTTP/1.0" 202 Accepted 2026-03-27 10:22:33,222 - lava_callback - INFO - Starting async job submit for node 69c5f6b82cb62322ad969230 2026-03-27 10:22:33,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:22:33,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f6b82cb62322ad969230 HTTP/1.1" 200 None 2026-03-27 10:22:33,263 - lava_callback - DEBUG - Processing node 69c5f6b82cb62322ad969230: baseline-nfs-arm64 2026-03-27 10:22:33,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:22:33,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:22:33,922 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5f6b82cb62322ad969230/log.txt.gz 2026-03-27 10:22:34,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:22:34,401 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:22:34,403 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5f6b82cb62322ad969230/lava-logs.yaml 2026-03-27 10:22:34,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:22:34,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:22:34,569 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5f6b82cb62322ad969230/lava_callback.json.gz 2026-03-27 10:22:34,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:22:34,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5f6b82cb62322ad969230 HTTP/1.1" 200 None 2026-03-27 10:22:34,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:22:34,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 10:22:34,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:22:34,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5f6b82cb62322ad969230 HTTP/1.1" 200 None 2026-03-27 10:22:34,734 - lava_callback - INFO - Completed processing callback for node 69c5f6b82cb62322ad969230 INFO: 172.20.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42950 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:23:03,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:23:03,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32994 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 10:23:16,163 - lava_callback - INFO - Received callback for node 69c6567ab98d4d3c02068e37 INFO: 172.20.0.1:33010 - "POST /node/69c6567ab98d4d3c02068e37 HTTP/1.0" 202 Accepted 2026-03-27 10:23:16,170 - lava_callback - INFO - Starting async job submit for node 69c6567ab98d4d3c02068e37 2026-03-27 10:23:16,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:23:16,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6567ab98d4d3c02068e37 HTTP/1.1" 200 None 2026-03-27 10:23:16,268 - lava_callback - DEBUG - Processing node 69c6567ab98d4d3c02068e37: ltp-smoke 2026-03-27 10:23:16,602 - lava_callback - INFO - Received callback for node 69c5fa5f2cb62322ad9692a2 INFO: 172.20.0.1:33018 - "POST /node/69c5fa5f2cb62322ad9692a2 HTTP/1.0" 202 Accepted 2026-03-27 10:23:16,624 - lava_callback - INFO - Starting async job submit for node 69c5fa5f2cb62322ad9692a2 2026-03-27 10:23:16,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:33020 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:23:16,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fa5f2cb62322ad9692a2 HTTP/1.1" 200 None 2026-03-27 10:23:16,792 - lava_callback - DEBUG - Processing node 69c5fa5f2cb62322ad9692a2: baseline-arm64 2026-03-27 10:23:16,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:23:17,103 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:23:17,147 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c6567ab98d4d3c02068e37/log.txt.gz 2026-03-27 10:23:17,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:23:17,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:23:17,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:23:17,844 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c6567ab98d4d3c02068e37/lava-logs.yaml 2026-03-27 10:23:17,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:23:17,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:23:17,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5fa5f2cb62322ad9692a2/log.txt.gz 2026-03-27 10:23:18,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:23:18,116 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c6567ab98d4d3c02068e37/lava_callback.json.gz 2026-03-27 10:23:18,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:23:18,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:23:18,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6567ab98d4d3c02068e37 HTTP/1.1" 200 None 2026-03-27 10:23:18,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:23:18,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce8b98d4d3c02067dcb HTTP/1.1" 200 None 2026-03-27 10:23:18,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:23:18,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6567ab98d4d3c02068e37 HTTP/1.1" 200 None 2026-03-27 10:23:18,459 - lava_callback - INFO - Completed processing callback for node 69c6567ab98d4d3c02068e37 2026-03-27 10:23:18,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:23:18,518 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5fa5f2cb62322ad9692a2/lava-logs.yaml 2026-03-27 10:23:18,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:23:18,686 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:23:18,688 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c5fa5f2cb62322ad9692a2/lava_callback.json.gz 2026-03-27 10:23:18,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:23:18,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fa5f2cb62322ad9692a2 HTTP/1.1" 200 None 2026-03-27 10:23:18,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:23:18,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 10:23:18,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:23:18,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5fa5f2cb62322ad9692a2 HTTP/1.1" 200 None 2026-03-27 10:23:18,858 - lava_callback - INFO - Completed processing callback for node 69c5fa5f2cb62322ad9692a2 INFO: 172.20.0.1:48150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48164 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:23:33,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:23:33,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34214 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:25:04,961 - lava_callback - INFO - Received callback for node 69c3b7f9bbc022e792644fcb INFO: 172.20.0.1:41372 - "POST /node/69c3b7f9bbc022e792644fcb HTTP/1.0" 202 Accepted 2026-03-27 10:25:04,967 - lava_callback - INFO - Starting async job submit for node 69c3b7f9bbc022e792644fcb 2026-03-27 10:25:04,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:04,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b7f9bbc022e792644fcb HTTP/1.1" 200 None 2026-03-27 10:25:04,982 - lava_callback - DEBUG - Processing node 69c3b7f9bbc022e792644fcb: baseline-arm64-clabbe 2026-03-27 10:25:04,983 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b7f9bbc022e792644fcb 2026-03-27 10:25:04,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:04,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b7f9bbc022e792644fcb HTTP/1.1" 200 None 2026-03-27 10:25:04,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:05,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-27 10:25:05,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:05,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b7f9bbc022e792644fcb HTTP/1.1" 200 None 2026-03-27 10:25:05,039 - lava_callback - ERROR - Error processing callback for node 69c3b7f9bbc022e792644fcb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36140 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:25:28,065 - lava_callback - INFO - Received callback for node 69c659bdb98d4d3c02068edd INFO: 172.20.0.1:36146 - "POST /node/69c659bdb98d4d3c02068edd HTTP/1.0" 202 Accepted 2026-03-27 10:25:28,072 - lava_callback - INFO - Starting async job submit for node 69c659bdb98d4d3c02068edd 2026-03-27 10:25:28,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:28,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c659bdb98d4d3c02068edd HTTP/1.1" 200 None 2026-03-27 10:25:28,123 - lava_callback - DEBUG - Processing node 69c659bdb98d4d3c02068edd: baseline-arm64 2026-03-27 10:25:28,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:25:28,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:25:28,697 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c659bdb98d4d3c02068edd/log.txt.gz 2026-03-27 10:25:28,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:25:29,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:25:29,101 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c659bdb98d4d3c02068edd/lava-logs.yaml 2026-03-27 10:25:29,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:25:29,280 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:25:29,281 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c659bdb98d4d3c02068edd/lava_callback.json.gz DEBUG: Setup failed for 69c659bdb98d4d3c02068edd. Transitting job node result: incomplete -> fail 2026-03-27 10:25:29,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:29,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c659bdb98d4d3c02068edd HTTP/1.1" 200 None 2026-03-27 10:25:29,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:29,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:25:29,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:29,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c659bdb98d4d3c02068edd HTTP/1.1" 200 None 2026-03-27 10:25:29,482 - lava_callback - INFO - Completed processing callback for node 69c659bdb98d4d3c02068edd 2026-03-27 10:25:33,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:33,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 10:25:36,481 - lava_callback - INFO - Received callback for node 69c6570eb98d4d3c02068e57 INFO: 172.20.0.1:38432 - "POST /node/69c6570eb98d4d3c02068e57 HTTP/1.0" 202 Accepted 2026-03-27 10:25:36,488 - lava_callback - INFO - Starting async job submit for node 69c6570eb98d4d3c02068e57 INFO: 172.20.0.1:38446 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:25:36,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:36,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6570eb98d4d3c02068e57 HTTP/1.1" 200 None 2026-03-27 10:25:36,569 - lava_callback - DEBUG - Processing node 69c6570eb98d4d3c02068e57: baseline-arm 2026-03-27 10:25:37,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:25:37,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:25:37,329 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c6570eb98d4d3c02068e57/log.txt.gz 2026-03-27 10:25:37,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:25:37,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:25:37,933 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c6570eb98d4d3c02068e57/lava-logs.yaml 2026-03-27 10:25:37,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:25:38,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:25:38,099 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c6570eb98d4d3c02068e57/lava_callback.json.gz DEBUG: Setup failed for 69c6570eb98d4d3c02068e57. Transitting job node result: incomplete -> fail 2026-03-27 10:25:38,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:38,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6570eb98d4d3c02068e57 HTTP/1.1" 200 None 2026-03-27 10:25:38,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:38,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 10:25:38,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:25:38,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6570eb98d4d3c02068e57 HTTP/1.1" 200 None 2026-03-27 10:25:38,385 - lava_callback - INFO - Completed processing callback for node 69c6570eb98d4d3c02068e57 INFO: 172.20.0.1:49454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53778 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:26:01,608 - lava_callback - INFO - Received callback for node 69c5fb522cb62322ad9692c1 INFO: 172.20.0.1:52348 - "POST /node/69c5fb522cb62322ad9692c1 HTTP/1.0" 202 Accepted 2026-03-27 10:26:01,615 - lava_callback - INFO - Starting async job submit for node 69c5fb522cb62322ad9692c1 2026-03-27 10:26:01,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:01,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fb522cb62322ad9692c1 HTTP/1.1" 200 None 2026-03-27 10:26:01,656 - lava_callback - DEBUG - Processing node 69c5fb522cb62322ad9692c1: baseline-nfs-arm64 2026-03-27 10:26:02,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:26:02,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:26:02,337 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5fb522cb62322ad9692c1/log.txt.gz 2026-03-27 10:26:02,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:26:02,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:26:02,822 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5fb522cb62322ad9692c1/lava-logs.yaml 2026-03-27 10:26:02,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:26:03,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:26:03,043 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c5fb522cb62322ad9692c1/lava_callback.json.gz 2026-03-27 10:26:03,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:03,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5fb522cb62322ad9692c1 HTTP/1.1" 200 None 2026-03-27 10:26:03,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:03,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 10:26:03,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:03,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5fb522cb62322ad9692c1 HTTP/1.1" 200 None 2026-03-27 10:26:03,221 - lava_callback - INFO - Completed processing callback for node 69c5fb522cb62322ad9692c1 2026-03-27 10:26:03,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:03,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-27 10:26:03,832 - lava_callback - INFO - Received callback for node 69c65b38b98d4d3c02068f2b INFO: 172.20.0.1:52364 - "POST /node/69c65b38b98d4d3c02068f2b HTTP/1.0" 202 Accepted 2026-03-27 10:26:03,839 - lava_callback - INFO - Starting async job submit for node 69c65b38b98d4d3c02068f2b 2026-03-27 10:26:03,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:03,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65b38b98d4d3c02068f2b HTTP/1.1" 200 None 2026-03-27 10:26:03,857 - lava_callback - DEBUG - Processing node 69c65b38b98d4d3c02068f2b: baseline-nfs-arm64 2026-03-27 10:26:03,859 - lava_callback - WARNING - Failed to upload log 2026-03-27 10:26:03,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:26:03,996 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:26:03,998 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65b38b98d4d3c02068f2b/lava-logs.yaml 2026-03-27 10:26:04,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:26:04,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:26:04,121 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65b38b98d4d3c02068f2b/lava_callback.json.gz 2026-03-27 10:26:04,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:04,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65b38b98d4d3c02068f2b HTTP/1.1" 200 None 2026-03-27 10:26:04,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:04,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:26:04,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:04,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65b38b98d4d3c02068f2b HTTP/1.1" 200 None 2026-03-27 10:26:04,193 - lava_callback - INFO - Completed processing callback for node 69c65b38b98d4d3c02068f2b INFO: 172.20.0.1:52378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32812 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:26:24,959 - lava_callback - INFO - Received callback for node 69c3b844bbc022e792644fd7 INFO: 172.20.0.1:32814 - "POST /node/69c3b844bbc022e792644fd7 HTTP/1.0" 202 Accepted 2026-03-27 10:26:24,965 - lava_callback - INFO - Starting async job submit for node 69c3b844bbc022e792644fd7 2026-03-27 10:26:24,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:24,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b844bbc022e792644fd7 HTTP/1.1" 200 None 2026-03-27 10:26:24,981 - lava_callback - DEBUG - Processing node 69c3b844bbc022e792644fd7: baseline-arm64-clabbe 2026-03-27 10:26:24,981 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b844bbc022e792644fd7 2026-03-27 10:26:24,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:24,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b844bbc022e792644fd7 HTTP/1.1" 200 None 2026-03-27 10:26:24,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:25,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bbb13dec3f435d20d51aa7 HTTP/1.1" 200 None 2026-03-27 10:26:25,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:25,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b844bbc022e792644fd7 HTTP/1.1" 200 None 2026-03-27 10:26:25,042 - lava_callback - ERROR - Error processing callback for node 69c3b844bbc022e792644fd7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32828 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:26:33,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:26:33,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46434 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:27:04,545 - lava_callback - INFO - Received callback for node 69c656cab98d4d3c02068e4d INFO: 172.20.0.1:50670 - "POST /node/69c656cab98d4d3c02068e4d HTTP/1.0" 202 Accepted 2026-03-27 10:27:04,554 - lava_callback - INFO - Starting async job submit for node 69c656cab98d4d3c02068e4d 2026-03-27 10:27:04,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:04,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c656cab98d4d3c02068e4d HTTP/1.1" 200 None 2026-03-27 10:27:04,617 - lava_callback - DEBUG - Processing node 69c656cab98d4d3c02068e4d: baseline-nfs-arm64 2026-03-27 10:27:05,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:27:05,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:27:05,828 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c656cab98d4d3c02068e4d/log.txt.gz 2026-03-27 10:27:06,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:50678 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:27:06,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:27:06,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c656cab98d4d3c02068e4d/lava-logs.yaml 2026-03-27 10:27:06,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:27:06,883 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:27:06,885 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c656cab98d4d3c02068e4d/lava_callback.json.gz DEBUG: Setup failed for 69c656cab98d4d3c02068e4d. Transitting job node result: incomplete -> fail 2026-03-27 10:27:06,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:07,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c656cab98d4d3c02068e4d HTTP/1.1" 200 None 2026-03-27 10:27:07,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:07,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:27:07,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:07,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c656cab98d4d3c02068e4d HTTP/1.1" 200 None 2026-03-27 10:27:07,132 - lava_callback - INFO - Completed processing callback for node 69c656cab98d4d3c02068e4d INFO: 172.20.0.1:38704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37694 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:27:25,567 - lava_callback - INFO - Received callback for node 69c60f5b2cb62322ad9694d5 INFO: 172.20.0.1:37708 - "POST /node/69c60f5b2cb62322ad9694d5 HTTP/1.0" 202 Accepted 2026-03-27 10:27:25,573 - lava_callback - INFO - Starting async job submit for node 69c60f5b2cb62322ad9694d5 2026-03-27 10:27:25,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:25,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60f5b2cb62322ad9694d5 HTTP/1.1" 200 None 2026-03-27 10:27:25,618 - lava_callback - DEBUG - Processing node 69c60f5b2cb62322ad9694d5: baseline-arm64 2026-03-27 10:27:26,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:27:26,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:27:26,253 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c60f5b2cb62322ad9694d5/log.txt.gz 2026-03-27 10:27:26,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:37710 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:27:26,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:27:26,697 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c60f5b2cb62322ad9694d5/lava-logs.yaml 2026-03-27 10:27:26,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:27:27,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:27:27,026 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c60f5b2cb62322ad9694d5/lava_callback.json.gz 2026-03-27 10:27:27,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:27,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60f5b2cb62322ad9694d5 HTTP/1.1" 200 None 2026-03-27 10:27:27,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:27,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 10:27:27,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:27,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60f5b2cb62322ad9694d5 HTTP/1.1" 200 None 2026-03-27 10:27:27,162 - lava_callback - INFO - Completed processing callback for node 69c60f5b2cb62322ad9694d5 2026-03-27 10:27:33,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:33,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58756 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:27:57,928 - lava_callback - INFO - Received callback for node 69c65b7cb98d4d3c02068f40 INFO: 172.20.0.1:58770 - "POST /node/69c65b7cb98d4d3c02068f40 HTTP/1.0" 202 Accepted 2026-03-27 10:27:57,935 - lava_callback - INFO - Starting async job submit for node 69c65b7cb98d4d3c02068f40 2026-03-27 10:27:57,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:57,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65b7cb98d4d3c02068f40 HTTP/1.1" 200 None 2026-03-27 10:27:57,979 - lava_callback - DEBUG - Processing node 69c65b7cb98d4d3c02068f40: baseline-arm64 2026-03-27 10:27:58,112 - lava_callback - WARNING - Failed to upload log 2026-03-27 10:27:58,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:27:58,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:27:58,332 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65b7cb98d4d3c02068f40/lava-logs.yaml 2026-03-27 10:27:58,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:27:58,446 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:27:58,448 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65b7cb98d4d3c02068f40/lava_callback.json.gz 2026-03-27 10:27:58,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:58,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65b7cb98d4d3c02068f40 HTTP/1.1" 200 None 2026-03-27 10:27:58,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:58,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:27:58,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:27:58,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65b7cb98d4d3c02068f40 HTTP/1.1" 200 None 2026-03-27 10:27:58,614 - lava_callback - INFO - Completed processing callback for node 69c65b7cb98d4d3c02068f40 2026-03-27 10:28:03,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:28:03,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38544 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:28:39,281 - lava_callback - INFO - Received callback for node 69c3b8d1bbc022e792644fe6 INFO: 172.20.0.1:38548 - "POST /node/69c3b8d1bbc022e792644fe6 HTTP/1.0" 202 Accepted 2026-03-27 10:28:39,287 - lava_callback - INFO - Starting async job submit for node 69c3b8d1bbc022e792644fe6 2026-03-27 10:28:39,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:28:39,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b8d1bbc022e792644fe6 HTTP/1.1" 200 None 2026-03-27 10:28:39,313 - lava_callback - DEBUG - Processing node 69c3b8d1bbc022e792644fe6: baseline-arm64-baylibre 2026-03-27 10:28:39,313 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b8d1bbc022e792644fe6 2026-03-27 10:28:39,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:28:39,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b8d1bbc022e792644fe6 HTTP/1.1" 200 None 2026-03-27 10:28:39,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:28:39,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:28:39,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:28:39,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b8d1bbc022e792644fe6 HTTP/1.1" 200 None 2026-03-27 10:28:39,371 - lava_callback - ERROR - Error processing callback for node 69c3b8d1bbc022e792644fe6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35202 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:29:31,284 - lava_callback - INFO - Received callback for node 69c65bcdb98d4d3c02068f51 INFO: 172.20.0.1:60320 - "POST /node/69c65bcdb98d4d3c02068f51 HTTP/1.0" 202 Accepted 2026-03-27 10:29:31,293 - lava_callback - INFO - Starting async job submit for node 69c65bcdb98d4d3c02068f51 2026-03-27 10:29:31,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:29:31,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65bcdb98d4d3c02068f51 HTTP/1.1" 200 None 2026-03-27 10:29:31,331 - lava_callback - DEBUG - Processing node 69c65bcdb98d4d3c02068f51: baseline-arm64 2026-03-27 10:29:31,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:29:31,822 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:29:31,824 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65bcdb98d4d3c02068f51/log.txt.gz 2026-03-27 10:29:32,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:29:32,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:29:32,255 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65bcdb98d4d3c02068f51/lava-logs.yaml 2026-03-27 10:29:32,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:29:32,420 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:29:32,422 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65bcdb98d4d3c02068f51/lava_callback.json.gz 2026-03-27 10:29:32,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:29:32,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65bcdb98d4d3c02068f51 HTTP/1.1" 200 None 2026-03-27 10:29:32,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:29:32,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:29:32,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:29:32,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65bcdb98d4d3c02068f51 HTTP/1.1" 200 None 2026-03-27 10:29:32,509 - lava_callback - INFO - Completed processing callback for node 69c65bcdb98d4d3c02068f51 2026-03-27 10:29:33,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:29:33,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51294 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:30:50,230 - lava_callback - INFO - Received callback for node 69c65c1bb98d4d3c02068f65 INFO: 172.20.0.1:59312 - "POST /node/69c65c1bb98d4d3c02068f65 HTTP/1.0" 202 Accepted 2026-03-27 10:30:50,237 - lava_callback - INFO - Starting async job submit for node 69c65c1bb98d4d3c02068f65 2026-03-27 10:30:50,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:30:50,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65c1bb98d4d3c02068f65 HTTP/1.1" 200 None 2026-03-27 10:30:50,284 - lava_callback - DEBUG - Processing node 69c65c1bb98d4d3c02068f65: baseline-arm64 2026-03-27 10:30:50,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:30:50,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:30:50,593 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65c1bb98d4d3c02068f65/log.txt.gz 2026-03-27 10:30:50,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:30:50,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:30:50,846 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65c1bb98d4d3c02068f65/lava-logs.yaml 2026-03-27 10:30:50,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:30:50,992 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:30:50,994 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65c1bb98d4d3c02068f65/lava_callback.json.gz 2026-03-27 10:30:51,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:30:51,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65c1bb98d4d3c02068f65 HTTP/1.1" 200 None 2026-03-27 10:30:51,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:30:51,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:30:51,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:30:51,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65c1bb98d4d3c02068f65 HTTP/1.1" 200 None 2026-03-27 10:30:51,166 - lava_callback - INFO - Completed processing callback for node 69c65c1bb98d4d3c02068f65 INFO: 172.20.0.1:59314 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:31:03,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:31:03,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39080 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:31:19,267 - lava_callback - INFO - Received callback for node 69c3b965bbc022e792644fff INFO: 172.20.0.1:39086 - "POST /node/69c3b965bbc022e792644fff HTTP/1.0" 202 Accepted 2026-03-27 10:31:19,273 - lava_callback - INFO - Starting async job submit for node 69c3b965bbc022e792644fff 2026-03-27 10:31:19,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:31:19,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b965bbc022e792644fff HTTP/1.1" 200 None 2026-03-27 10:31:19,294 - lava_callback - DEBUG - Processing node 69c3b965bbc022e792644fff: baseline-arm64-baylibre 2026-03-27 10:31:19,294 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b965bbc022e792644fff 2026-03-27 10:31:19,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:31:19,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b965bbc022e792644fff HTTP/1.1" 200 None 2026-03-27 10:31:19,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:31:19,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:31:19,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:31:19,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b965bbc022e792644fff HTTP/1.1" 200 None 2026-03-27 10:31:19,347 - lava_callback - ERROR - Error processing callback for node 69c3b965bbc022e792644fff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35922 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:32:39,345 - lava_callback - INFO - Received callback for node 69c3b9b2bbc022e79264500a INFO: 172.20.0.1:35930 - "POST /node/69c3b9b2bbc022e79264500a HTTP/1.0" 202 Accepted 2026-03-27 10:32:39,351 - lava_callback - INFO - Starting async job submit for node 69c3b9b2bbc022e79264500a 2026-03-27 10:32:39,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:32:39,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b9b2bbc022e79264500a HTTP/1.1" 200 None 2026-03-27 10:32:39,368 - lava_callback - DEBUG - Processing node 69c3b9b2bbc022e79264500a: baseline-arm64-baylibre 2026-03-27 10:32:39,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3b9b2bbc022e79264500a 2026-03-27 10:32:39,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:32:39,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3b9b2bbc022e79264500a HTTP/1.1" 200 None 2026-03-27 10:32:39,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:32:39,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:32:39,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:32:39,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3b9b2bbc022e79264500a HTTP/1.1" 200 None 2026-03-27 10:32:39,430 - lava_callback - ERROR - Error processing callback for node 69c3b9b2bbc022e79264500a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56652 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:34:59,675 - lava_callback - INFO - Received callback for node 69c3ba45bbc022e792645020 INFO: 172.20.0.1:56658 - "POST /node/69c3ba45bbc022e792645020 HTTP/1.0" 202 Accepted 2026-03-27 10:34:59,681 - lava_callback - INFO - Starting async job submit for node 69c3ba45bbc022e792645020 2026-03-27 10:34:59,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:34:59,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ba45bbc022e792645020 HTTP/1.1" 200 None 2026-03-27 10:34:59,708 - lava_callback - DEBUG - Processing node 69c3ba45bbc022e792645020: baseline-arm64-baylibre 2026-03-27 10:34:59,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ba45bbc022e792645020 2026-03-27 10:34:59,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:34:59,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ba45bbc022e792645020 HTTP/1.1" 200 None 2026-03-27 10:34:59,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:34:59,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:34:59,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:34:59,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ba45bbc022e792645020 HTTP/1.1" 200 None 2026-03-27 10:34:59,774 - lava_callback - ERROR - Error processing callback for node 69c3ba45bbc022e792645020: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38300 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 10:35:15,698 - lava_callback - INFO - Received callback for node 69c60fb32cb62322ad9694ea INFO: 172.20.0.1:38306 - "POST /node/69c60fb32cb62322ad9694ea HTTP/1.0" 202 Accepted 2026-03-27 10:35:15,704 - lava_callback - INFO - Starting async job submit for node 69c60fb32cb62322ad9694ea 2026-03-27 10:35:15,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:35:15,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60fb32cb62322ad9694ea HTTP/1.1" 200 None 2026-03-27 10:35:15,745 - lava_callback - DEBUG - Processing node 69c60fb32cb62322ad9694ea: baseline-nfs-arm64 2026-03-27 10:35:16,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:35:16,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:35:16,419 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c60fb32cb62322ad9694ea/log.txt.gz INFO: 172.20.0.1:38312 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:35:16,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:35:16,878 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:35:16,880 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c60fb32cb62322ad9694ea/lava-logs.yaml 2026-03-27 10:35:16,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:35:17,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:35:17,057 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c60fb32cb62322ad9694ea/lava_callback.json.gz 2026-03-27 10:35:17,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:35:17,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c60fb32cb62322ad9694ea HTTP/1.1" 200 None 2026-03-27 10:35:17,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:35:17,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 10:35:17,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:35:17,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c60fb32cb62322ad9694ea HTTP/1.1" 200 None 2026-03-27 10:35:17,210 - lava_callback - INFO - Completed processing callback for node 69c60fb32cb62322ad9694ea INFO: 172.20.0.1:46904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46912 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:35:33,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:35:33,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53896 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:36:19,556 - lava_callback - INFO - Received callback for node 69c3ba92bbc022e792645027 INFO: 172.20.0.1:53898 - "POST /node/69c3ba92bbc022e792645027 HTTP/1.0" 202 Accepted 2026-03-27 10:36:19,561 - lava_callback - INFO - Starting async job submit for node 69c3ba92bbc022e792645027 2026-03-27 10:36:19,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:36:19,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ba92bbc022e792645027 HTTP/1.1" 200 None 2026-03-27 10:36:19,577 - lava_callback - DEBUG - Processing node 69c3ba92bbc022e792645027: baseline-arm64-baylibre 2026-03-27 10:36:19,577 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ba92bbc022e792645027 2026-03-27 10:36:19,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:36:19,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ba92bbc022e792645027 HTTP/1.1" 200 None 2026-03-27 10:36:19,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:36:19,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:36:19,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:36:19,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ba92bbc022e792645027 HTTP/1.1" 200 None 2026-03-27 10:36:19,634 - lava_callback - ERROR - Error processing callback for node 69c3ba92bbc022e792645027: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33732 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:37:19,635 - lava_callback - INFO - Received callback for node 69c3badebbc022e792645038 INFO: 172.20.0.1:33744 - "POST /node/69c3badebbc022e792645038 HTTP/1.0" 202 Accepted 2026-03-27 10:37:19,640 - lava_callback - INFO - Starting async job submit for node 69c3badebbc022e792645038 2026-03-27 10:37:19,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:37:19,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3badebbc022e792645038 HTTP/1.1" 200 None 2026-03-27 10:37:19,660 - lava_callback - DEBUG - Processing node 69c3badebbc022e792645038: baseline-arm64-baylibre 2026-03-27 10:37:19,661 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3badebbc022e792645038 2026-03-27 10:37:19,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:37:19,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3badebbc022e792645038 HTTP/1.1" 200 None 2026-03-27 10:37:19,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:37:19,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:37:19,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:37:19,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3badebbc022e792645038 HTTP/1.1" 200 None 2026-03-27 10:37:19,718 - lava_callback - ERROR - Error processing callback for node 69c3badebbc022e792645038: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46700 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:38:38,261 - lava_callback - INFO - Received callback for node 69c65c6fb98d4d3c02068f71 INFO: 172.20.0.1:46716 - "POST /node/69c65c6fb98d4d3c02068f71 HTTP/1.0" 202 Accepted 2026-03-27 10:38:38,269 - lava_callback - INFO - Starting async job submit for node 69c65c6fb98d4d3c02068f71 2026-03-27 10:38:38,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:38:38,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65c6fb98d4d3c02068f71 HTTP/1.1" 200 None 2026-03-27 10:38:38,325 - lava_callback - DEBUG - Processing node 69c65c6fb98d4d3c02068f71: baseline-arm64 2026-03-27 10:38:38,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:38:38,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:38:38,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65c6fb98d4d3c02068f71/log.txt.gz 2026-03-27 10:38:39,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:38:39,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:38:39,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65c6fb98d4d3c02068f71/lava-logs.yaml 2026-03-27 10:38:39,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:38:39,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:38:39,517 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65c6fb98d4d3c02068f71/lava_callback.json.gz DEBUG: Setup failed for 69c65c6fb98d4d3c02068f71. Transitting job node result: incomplete -> fail 2026-03-27 10:38:39,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:38:39,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65c6fb98d4d3c02068f71 HTTP/1.1" 200 None 2026-03-27 10:38:39,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:38:39,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:38:39,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:38:39,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65c6fb98d4d3c02068f71 HTTP/1.1" 200 None 2026-03-27 10:38:39,728 - lava_callback - INFO - Completed processing callback for node 69c65c6fb98d4d3c02068f71 2026-03-27 10:38:45,562 - lava_callback - INFO - Received callback for node 69c3bb2fbbc022e792645043 INFO: 172.20.0.1:41920 - "POST /node/69c3bb2fbbc022e792645043 HTTP/1.0" 202 Accepted 2026-03-27 10:38:45,567 - lava_callback - INFO - Starting async job submit for node 69c3bb2fbbc022e792645043 2026-03-27 10:38:45,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:38:45,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bb2fbbc022e792645043 HTTP/1.1" 200 None 2026-03-27 10:38:45,584 - lava_callback - DEBUG - Processing node 69c3bb2fbbc022e792645043: baseline-arm64-clabbe 2026-03-27 10:38:45,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bb2fbbc022e792645043 2026-03-27 10:38:45,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:38:45,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bb2fbbc022e792645043 HTTP/1.1" 200 None 2026-03-27 10:38:45,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:38:45,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:38:45,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:38:45,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bb2fbbc022e792645043 HTTP/1.1" 200 None 2026-03-27 10:38:45,643 - lava_callback - ERROR - Error processing callback for node 69c3bb2fbbc022e792645043: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44216 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:39:03,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:39:03,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47034 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 10:39:15,707 - lava_callback - INFO - Received callback for node 69c6148f2cb62322ad9695c3 INFO: 172.20.0.1:47046 - "POST /node/69c6148f2cb62322ad9695c3 HTTP/1.0" 202 Accepted 2026-03-27 10:39:15,713 - lava_callback - INFO - Starting async job submit for node 69c6148f2cb62322ad9695c3 2026-03-27 10:39:15,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:39:15,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6148f2cb62322ad9695c3 HTTP/1.1" 200 None 2026-03-27 10:39:15,752 - lava_callback - DEBUG - Processing node 69c6148f2cb62322ad9695c3: baseline-arm64 2026-03-27 10:39:16,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:39:16,270 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:39:16,271 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6148f2cb62322ad9695c3/log.txt.gz 2026-03-27 10:39:16,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:47050 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:39:16,668 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:39:16,670 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6148f2cb62322ad9695c3/lava-logs.yaml 2026-03-27 10:39:16,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:39:16,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:39:16,835 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6148f2cb62322ad9695c3/lava_callback.json.gz 2026-03-27 10:39:16,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:39:16,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6148f2cb62322ad9695c3 HTTP/1.1" 200 None 2026-03-27 10:39:16,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:39:16,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 10:39:16,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:39:16,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6148f2cb62322ad9695c3 HTTP/1.1" 200 None 2026-03-27 10:39:16,981 - lava_callback - INFO - Completed processing callback for node 69c6148f2cb62322ad9695c3 2026-03-27 10:39:18,254 - lava_callback - INFO - Received callback for node 69c6152d2cb62322ad9695e5 INFO: 172.20.0.1:47064 - "POST /node/69c6152d2cb62322ad9695e5 HTTP/1.0" 202 Accepted 2026-03-27 10:39:18,261 - lava_callback - INFO - Starting async job submit for node 69c6152d2cb62322ad9695e5 2026-03-27 10:39:18,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:39:18,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6152d2cb62322ad9695e5 HTTP/1.1" 200 None 2026-03-27 10:39:18,317 - lava_callback - DEBUG - Processing node 69c6152d2cb62322ad9695e5: baseline-nfs-arm64 2026-03-27 10:39:18,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:39:18,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:39:18,939 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6152d2cb62322ad9695e5/log.txt.gz 2026-03-27 10:39:19,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:39:19,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:39:19,422 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6152d2cb62322ad9695e5/lava-logs.yaml 2026-03-27 10:39:19,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:39:19,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:39:19,589 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6152d2cb62322ad9695e5/lava_callback.json.gz 2026-03-27 10:39:19,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:39:19,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6152d2cb62322ad9695e5 HTTP/1.1" 200 None 2026-03-27 10:39:19,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:39:19,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 10:39:19,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:39:19,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6152d2cb62322ad9695e5 HTTP/1.1" 200 None 2026-03-27 10:39:19,746 - lava_callback - INFO - Completed processing callback for node 69c6152d2cb62322ad9695e5 INFO: 172.20.0.1:52302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52308 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:39:33,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:39:33,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59342 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:40:05,641 - lava_callback - INFO - Received callback for node 69c3bb7bbbc022e79264504a INFO: 172.20.0.1:38310 - "POST /node/69c3bb7bbbc022e79264504a HTTP/1.0" 202 Accepted 2026-03-27 10:40:05,647 - lava_callback - INFO - Starting async job submit for node 69c3bb7bbbc022e79264504a 2026-03-27 10:40:05,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:40:05,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bb7bbbc022e79264504a HTTP/1.1" 200 None 2026-03-27 10:40:05,663 - lava_callback - DEBUG - Processing node 69c3bb7bbbc022e79264504a: baseline-arm64-clabbe 2026-03-27 10:40:05,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bb7bbbc022e79264504a 2026-03-27 10:40:05,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:40:05,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bb7bbbc022e79264504a HTTP/1.1" 200 None 2026-03-27 10:40:05,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:40:05,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:40:05,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:40:05,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bb7bbbc022e79264504a HTTP/1.1" 200 None 2026-03-27 10:40:05,715 - lava_callback - ERROR - Error processing callback for node 69c3bb7bbbc022e79264504a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32958 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:40:26,261 - lava_callback - INFO - Received callback for node 69c619032cb62322ad9696ee INFO: 172.20.0.1:32970 - "POST /node/69c619032cb62322ad9696ee HTTP/1.0" 202 Accepted 2026-03-27 10:40:26,268 - lava_callback - INFO - Starting async job submit for node 69c619032cb62322ad9696ee 2026-03-27 10:40:26,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:40:26,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c619032cb62322ad9696ee HTTP/1.1" 200 None 2026-03-27 10:40:26,309 - lava_callback - DEBUG - Processing node 69c619032cb62322ad9696ee: baseline-nfs-arm64 INFO: 172.20.0.1:32980 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:40:26,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:40:27,161 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:40:27,162 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c619032cb62322ad9696ee/log.txt.gz 2026-03-27 10:40:27,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:40:27,697 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:40:27,700 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c619032cb62322ad9696ee/lava-logs.yaml 2026-03-27 10:40:27,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:40:27,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:40:27,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c619032cb62322ad9696ee/lava_callback.json.gz 2026-03-27 10:40:27,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:40:27,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c619032cb62322ad9696ee HTTP/1.1" 200 None 2026-03-27 10:40:27,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:40:27,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 10:40:27,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:40:27,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c619032cb62322ad9696ee HTTP/1.1" 200 None 2026-03-27 10:40:28,050 - lava_callback - INFO - Completed processing callback for node 69c619032cb62322ad9696ee 2026-03-27 10:40:33,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:40:33,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46238 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:41:25,653 - lava_callback - INFO - Received callback for node 69c3bbc9bbc022e792645055 INFO: 172.20.0.1:46252 - "POST /node/69c3bbc9bbc022e792645055 HTTP/1.0" 202 Accepted 2026-03-27 10:41:25,658 - lava_callback - INFO - Starting async job submit for node 69c3bbc9bbc022e792645055 2026-03-27 10:41:25,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:41:25,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bbc9bbc022e792645055 HTTP/1.1" 200 None 2026-03-27 10:41:25,675 - lava_callback - DEBUG - Processing node 69c3bbc9bbc022e792645055: baseline-arm64-clabbe 2026-03-27 10:41:25,675 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bbc9bbc022e792645055 2026-03-27 10:41:25,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:41:25,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bbc9bbc022e792645055 HTTP/1.1" 200 None 2026-03-27 10:41:25,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:41:25,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:41:25,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:41:25,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bbc9bbc022e792645055 HTTP/1.1" 200 None 2026-03-27 10:41:25,787 - lava_callback - ERROR - Error processing callback for node 69c3bbc9bbc022e792645055: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52862 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:44:05,770 - lava_callback - INFO - Received callback for node 69c3bc64bbc022e792645071 INFO: 172.20.0.1:46476 - "POST /node/69c3bc64bbc022e792645071 HTTP/1.0" 202 Accepted 2026-03-27 10:44:05,776 - lava_callback - INFO - Starting async job submit for node 69c3bc64bbc022e792645071 2026-03-27 10:44:05,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:44:05,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bc64bbc022e792645071 HTTP/1.1" 200 None 2026-03-27 10:44:05,819 - lava_callback - DEBUG - Processing node 69c3bc64bbc022e792645071: baseline-arm64-clabbe 2026-03-27 10:44:05,819 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bc64bbc022e792645071 2026-03-27 10:44:05,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:44:05,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bc64bbc022e792645071 HTTP/1.1" 200 None 2026-03-27 10:44:05,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:44:05,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:44:05,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:44:05,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bc64bbc022e792645071 HTTP/1.1" 200 None 2026-03-27 10:44:05,872 - lava_callback - ERROR - Error processing callback for node 69c3bc64bbc022e792645071: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54478 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:45:25,752 - lava_callback - INFO - Received callback for node 69c3bcb0bbc022e79264507c INFO: 172.20.0.1:54494 - "POST /node/69c3bcb0bbc022e79264507c HTTP/1.0" 202 Accepted 2026-03-27 10:45:25,758 - lava_callback - INFO - Starting async job submit for node 69c3bcb0bbc022e79264507c 2026-03-27 10:45:25,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:45:25,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bcb0bbc022e79264507c HTTP/1.1" 200 None 2026-03-27 10:45:25,776 - lava_callback - DEBUG - Processing node 69c3bcb0bbc022e79264507c: baseline-arm64-clabbe 2026-03-27 10:45:25,776 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bcb0bbc022e79264507c 2026-03-27 10:45:25,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:45:25,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bcb0bbc022e79264507c HTTP/1.1" 200 None 2026-03-27 10:45:25,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:45:25,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:45:25,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:45:25,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bcb0bbc022e79264507c HTTP/1.1" 200 None 2026-03-27 10:45:25,843 - lava_callback - ERROR - Error processing callback for node 69c3bcb0bbc022e79264507c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45658 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:47:09,353 - lava_callback - INFO - Received callback for node 69c658bab98d4d3c02068eaa INFO: 172.20.0.1:45670 - "POST /node/69c658bab98d4d3c02068eaa HTTP/1.0" 202 Accepted 2026-03-27 10:47:09,401 - lava_callback - INFO - Starting async job submit for node 69c658bab98d4d3c02068eaa 2026-03-27 10:47:09,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:09,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c658bab98d4d3c02068eaa HTTP/1.1" 200 None 2026-03-27 10:47:09,463 - lava_callback - DEBUG - Processing node 69c658bab98d4d3c02068eaa: baseline-arm-pengutronix 2026-03-27 10:47:09,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:47:09,771 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:47:09,773 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c658bab98d4d3c02068eaa/log.txt.gz 2026-03-27 10:47:09,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:47:10,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:47:10,021 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c658bab98d4d3c02068eaa/lava-logs.yaml 2026-03-27 10:47:10,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:47:10,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:47:10,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c658bab98d4d3c02068eaa/lava_callback.json.gz DEBUG: Setup failed for 69c658bab98d4d3c02068eaa. Transitting job node result: incomplete -> fail 2026-03-27 10:47:10,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:10,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c658bab98d4d3c02068eaa HTTP/1.1" 200 None 2026-03-27 10:47:10,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:10,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 10:47:10,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:10,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c658bab98d4d3c02068eaa HTTP/1.1" 200 None 2026-03-27 10:47:10,353 - lava_callback - INFO - Completed processing callback for node 69c658bab98d4d3c02068eaa INFO: 172.20.0.1:34736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57000 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:47:23,359 - lava_callback - INFO - Received callback for node 69c629cd2cb62322ad9699fe INFO: 172.20.0.1:57014 - "POST /node/69c629cd2cb62322ad9699fe HTTP/1.0" 202 Accepted 2026-03-27 10:47:23,366 - lava_callback - INFO - Starting async job submit for node 69c629cd2cb62322ad9699fe 2026-03-27 10:47:23,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:23,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c629cd2cb62322ad9699fe HTTP/1.1" 200 None 2026-03-27 10:47:23,406 - lava_callback - DEBUG - Processing node 69c629cd2cb62322ad9699fe: baseline-arm64 2026-03-27 10:47:23,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:47:24,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:47:24,083 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c629cd2cb62322ad9699fe/log.txt.gz 2026-03-27 10:47:24,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:47:24,568 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:47:24,571 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c629cd2cb62322ad9699fe/lava-logs.yaml 2026-03-27 10:47:24,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:47:24,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:47:24,756 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c629cd2cb62322ad9699fe/lava_callback.json.gz 2026-03-27 10:47:24,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:24,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c629cd2cb62322ad9699fe HTTP/1.1" 200 None 2026-03-27 10:47:24,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:24,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 10:47:24,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:24,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c629cd2cb62322ad9699fe HTTP/1.1" 200 None 2026-03-27 10:47:24,908 - lava_callback - INFO - Completed processing callback for node 69c629cd2cb62322ad9699fe INFO: 172.20.0.1:57024 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:47:33,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:33,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46088 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:47:46,125 - lava_callback - INFO - Received callback for node 69c3bd4bbbc022e7926450a6 INFO: 172.20.0.1:41706 - "POST /node/69c3bd4bbbc022e7926450a6 HTTP/1.0" 202 Accepted 2026-03-27 10:47:46,131 - lava_callback - INFO - Starting async job submit for node 69c3bd4bbbc022e7926450a6 2026-03-27 10:47:46,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:46,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bd4bbbc022e7926450a6 HTTP/1.1" 200 None 2026-03-27 10:47:46,147 - lava_callback - DEBUG - Processing node 69c3bd4bbbc022e7926450a6: baseline-arm64-clabbe 2026-03-27 10:47:46,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bd4bbbc022e7926450a6 2026-03-27 10:47:46,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:46,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bd4bbbc022e7926450a6 HTTP/1.1" 200 None 2026-03-27 10:47:46,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:46,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be5461448e074b3ffe163c HTTP/1.1" 200 None 2026-03-27 10:47:46,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:47:46,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bd4bbbc022e7926450a6 HTTP/1.1" 200 None 2026-03-27 10:47:46,228 - lava_callback - ERROR - Error processing callback for node 69c3bd4bbbc022e7926450a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35494 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:48:56,834 - lava_callback - INFO - Received callback for node 69c65c67b98d4d3c02068f6d INFO: 172.20.0.1:35506 - "POST /node/69c65c67b98d4d3c02068f6d HTTP/1.0" 202 Accepted 2026-03-27 10:48:56,840 - lava_callback - INFO - Starting async job submit for node 69c65c67b98d4d3c02068f6d 2026-03-27 10:48:56,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:48:56,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65c67b98d4d3c02068f6d HTTP/1.1" 200 None 2026-03-27 10:48:56,920 - lava_callback - DEBUG - Processing node 69c65c67b98d4d3c02068f6d: ltp-smoke 2026-03-27 10:48:57,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:48:57,562 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:48:57,564 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c65c67b98d4d3c02068f6d/log.txt.gz 2026-03-27 10:48:57,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:48:58,084 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:48:58,086 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c65c67b98d4d3c02068f6d/lava-logs.yaml 2026-03-27 10:48:58,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:48:58,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:48:58,280 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c65c67b98d4d3c02068f6d/lava_callback.json.gz 2026-03-27 10:48:58,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:48:58,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65c67b98d4d3c02068f6d HTTP/1.1" 200 None 2026-03-27 10:48:58,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:48:58,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce8b98d4d3c02067dcb HTTP/1.1" 200 None 2026-03-27 10:48:58,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:48:58,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65c67b98d4d3c02068f6d HTTP/1.1" 200 None 2026-03-27 10:48:58,616 - lava_callback - INFO - Completed processing callback for node 69c65c67b98d4d3c02068f6d 2026-03-27 10:49:03,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:49:04,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53162 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:50:02,171 - lava_callback - INFO - Received callback for node 69c65cb7b98d4d3c02068f7e INFO: 172.20.0.1:54344 - "POST /node/69c65cb7b98d4d3c02068f7e HTTP/1.0" 202 Accepted 2026-03-27 10:50:02,178 - lava_callback - INFO - Starting async job submit for node 69c65cb7b98d4d3c02068f7e 2026-03-27 10:50:02,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:50:02,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65cb7b98d4d3c02068f7e HTTP/1.1" 200 None 2026-03-27 10:50:02,248 - lava_callback - DEBUG - Processing node 69c65cb7b98d4d3c02068f7e: baseline-arm 2026-03-27 10:50:02,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:50:03,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:50:03,015 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c65cb7b98d4d3c02068f7e/log.txt.gz 2026-03-27 10:50:03,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:50:03,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:50:03,622 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c65cb7b98d4d3c02068f7e/lava-logs.yaml 2026-03-27 10:50:03,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:50:03,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:50:03,826 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c65cb7b98d4d3c02068f7e/lava_callback.json.gz DEBUG: Setup failed for 69c65cb7b98d4d3c02068f7e. Transitting job node result: incomplete -> fail 2026-03-27 10:50:03,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:50:03,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65cb7b98d4d3c02068f7e HTTP/1.1" 200 None 2026-03-27 10:50:03,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:50:03,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 10:50:03,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:50:04,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65cb7b98d4d3c02068f7e HTTP/1.1" 200 None 2026-03-27 10:50:04,104 - lava_callback - INFO - Completed processing callback for node 69c65cb7b98d4d3c02068f7e INFO: 172.20.0.1:54346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43192 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:50:34,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:50:34,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39422 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:51:42,595 - lava_callback - INFO - Received callback for node 69c6328a2cb62322ad969c14 INFO: 172.20.0.1:52878 - "POST /node/69c6328a2cb62322ad969c14 HTTP/1.0" 202 Accepted 2026-03-27 10:51:42,602 - lava_callback - INFO - Starting async job submit for node 69c6328a2cb62322ad969c14 2026-03-27 10:51:42,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:51:42,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6328a2cb62322ad969c14 HTTP/1.1" 200 None 2026-03-27 10:51:42,643 - lava_callback - DEBUG - Processing node 69c6328a2cb62322ad969c14: baseline-arm64 2026-03-27 10:51:43,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:51:43,332 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:51:43,334 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6328a2cb62322ad969c14/log.txt.gz 2026-03-27 10:51:43,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:51:43,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:51:43,820 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6328a2cb62322ad969c14/lava-logs.yaml 2026-03-27 10:51:43,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:51:44,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:51:44,042 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6328a2cb62322ad969c14/lava_callback.json.gz 2026-03-27 10:51:44,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:51:44,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6328a2cb62322ad969c14 HTTP/1.1" 200 None 2026-03-27 10:51:44,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:51:44,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-27 10:51:44,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:51:44,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6328a2cb62322ad969c14 HTTP/1.1" 200 None 2026-03-27 10:51:44,204 - lava_callback - INFO - Completed processing callback for node 69c6328a2cb62322ad969c14 INFO: 172.20.0.1:52888 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:51:47,000 - lava_callback - INFO - Received callback for node 69c62c222cb62322ad969abd INFO: 172.20.0.1:52900 - "POST /node/69c62c222cb62322ad969abd HTTP/1.0" 202 Accepted 2026-03-27 10:51:47,007 - lava_callback - INFO - Starting async job submit for node 69c62c222cb62322ad969abd 2026-03-27 10:51:47,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:51:47,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62c222cb62322ad969abd HTTP/1.1" 200 None 2026-03-27 10:51:47,053 - lava_callback - DEBUG - Processing node 69c62c222cb62322ad969abd: baseline-nfs-arm64 2026-03-27 10:51:47,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:51:47,716 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:51:47,718 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c62c222cb62322ad969abd/log.txt.gz 2026-03-27 10:51:47,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:51:48,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:51:48,206 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c62c222cb62322ad969abd/lava-logs.yaml 2026-03-27 10:51:48,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:51:48,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:51:48,381 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c62c222cb62322ad969abd/lava_callback.json.gz 2026-03-27 10:51:48,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:51:48,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c62c222cb62322ad969abd HTTP/1.1" 200 None 2026-03-27 10:51:48,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:51:48,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 10:51:48,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:51:48,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c62c222cb62322ad969abd HTTP/1.1" 200 None 2026-03-27 10:51:48,534 - lava_callback - INFO - Completed processing callback for node 69c62c222cb62322ad969abd INFO: 172.20.0.1:53270 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:52:04,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:52:04,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46628 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:53:04,902 - lava_callback - INFO - Received callback for node 69c65ce2b98d4d3c02068f89 INFO: 172.20.0.1:55696 - "POST /node/69c65ce2b98d4d3c02068f89 HTTP/1.0" 202 Accepted 2026-03-27 10:53:04,910 - lava_callback - INFO - Starting async job submit for node 69c65ce2b98d4d3c02068f89 2026-03-27 10:53:04,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:53:04,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65ce2b98d4d3c02068f89 HTTP/1.1" 200 None 2026-03-27 10:53:04,978 - lava_callback - DEBUG - Processing node 69c65ce2b98d4d3c02068f89: baseline-nfs-arm64 2026-03-27 10:53:05,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:53:06,178 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:53:06,179 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65ce2b98d4d3c02068f89/log.txt.gz INFO: 172.20.0.1:55712 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:53:06,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:53:07,033 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:53:07,036 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65ce2b98d4d3c02068f89/lava-logs.yaml 2026-03-27 10:53:07,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 10:53:07,252 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 10:53:07,254 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65ce2b98d4d3c02068f89/lava_callback.json.gz DEBUG: Setup failed for 69c65ce2b98d4d3c02068f89. Transitting job node result: incomplete -> fail 2026-03-27 10:53:07,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:53:07,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65ce2b98d4d3c02068f89 HTTP/1.1" 200 None 2026-03-27 10:53:07,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:53:07,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 10:53:07,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:53:07,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65ce2b98d4d3c02068f89 HTTP/1.1" 200 None 2026-03-27 10:53:07,521 - lava_callback - INFO - Completed processing callback for node 69c65ce2b98d4d3c02068f89 INFO: 172.20.0.1:54702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52586 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:53:34,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:53:34,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58792 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:56:20,679 - lava_callback - INFO - Received callback for node 69c3bf4abbc022e792645132 INFO: 172.20.0.1:44390 - "POST /node/69c3bf4abbc022e792645132 HTTP/1.0" 202 Accepted 2026-03-27 10:56:20,684 - lava_callback - INFO - Starting async job submit for node 69c3bf4abbc022e792645132 2026-03-27 10:56:20,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:56:20,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bf4abbc022e792645132 HTTP/1.1" 200 None 2026-03-27 10:56:20,700 - lava_callback - DEBUG - Processing node 69c3bf4abbc022e792645132: baseline-arm-baylibre 2026-03-27 10:56:20,700 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bf4abbc022e792645132 2026-03-27 10:56:20,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:56:20,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bf4abbc022e792645132 HTTP/1.1" 200 None 2026-03-27 10:56:20,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:56:20,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 10:56:20,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:56:20,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bf4abbc022e792645132 HTTP/1.1" 200 None 2026-03-27 10:56:20,784 - lava_callback - ERROR - Error processing callback for node 69c3bf4abbc022e792645132: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38930 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:57:00,651 - lava_callback - INFO - Received callback for node 69c3bf72bbc022e79264513d INFO: 172.20.0.1:39768 - "POST /node/69c3bf72bbc022e79264513d HTTP/1.0" 202 Accepted 2026-03-27 10:57:00,657 - lava_callback - INFO - Starting async job submit for node 69c3bf72bbc022e79264513d 2026-03-27 10:57:00,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:57:00,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bf72bbc022e79264513d HTTP/1.1" 200 None 2026-03-27 10:57:00,672 - lava_callback - DEBUG - Processing node 69c3bf72bbc022e79264513d: baseline-arm-baylibre 2026-03-27 10:57:00,672 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bf72bbc022e79264513d 2026-03-27 10:57:00,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:57:00,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bf72bbc022e79264513d HTTP/1.1" 200 None 2026-03-27 10:57:00,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:57:00,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 10:57:00,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:57:00,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bf72bbc022e79264513d HTTP/1.1" 200 None 2026-03-27 10:57:00,732 - lava_callback - ERROR - Error processing callback for node 69c3bf72bbc022e79264513d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42210 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:57:40,731 - lava_callback - INFO - Received callback for node 69c3bf99bbc022e79264514b INFO: 172.20.0.1:58656 - "POST /node/69c3bf99bbc022e79264514b HTTP/1.0" 202 Accepted 2026-03-27 10:57:40,736 - lava_callback - INFO - Starting async job submit for node 69c3bf99bbc022e79264514b 2026-03-27 10:57:40,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:57:40,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bf99bbc022e79264514b HTTP/1.1" 200 None 2026-03-27 10:57:40,752 - lava_callback - DEBUG - Processing node 69c3bf99bbc022e79264514b: baseline-arm-baylibre 2026-03-27 10:57:40,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bf99bbc022e79264514b 2026-03-27 10:57:40,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:57:40,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bf99bbc022e79264514b HTTP/1.1" 200 None 2026-03-27 10:57:40,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:57:40,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 10:57:40,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:57:40,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bf99bbc022e79264514b HTTP/1.1" 200 None 2026-03-27 10:57:40,828 - lava_callback - ERROR - Error processing callback for node 69c3bf99bbc022e79264514b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51490 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:58:20,751 - lava_callback - INFO - Received callback for node 69c3bfc1bbc022e792645157 INFO: 172.20.0.1:53896 - "POST /node/69c3bfc1bbc022e792645157 HTTP/1.0" 202 Accepted 2026-03-27 10:58:20,757 - lava_callback - INFO - Starting async job submit for node 69c3bfc1bbc022e792645157 2026-03-27 10:58:20,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:58:20,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bfc1bbc022e792645157 HTTP/1.1" 200 None 2026-03-27 10:58:20,773 - lava_callback - DEBUG - Processing node 69c3bfc1bbc022e792645157: baseline-arm-baylibre 2026-03-27 10:58:20,773 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bfc1bbc022e792645157 2026-03-27 10:58:20,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:58:20,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bfc1bbc022e792645157 HTTP/1.1" 200 None 2026-03-27 10:58:20,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:58:20,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 10:58:20,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:58:20,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bfc1bbc022e792645157 HTTP/1.1" 200 None 2026-03-27 10:58:20,838 - lava_callback - ERROR - Error processing callback for node 69c3bfc1bbc022e792645157: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60598 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:59:00,836 - lava_callback - INFO - Received callback for node 69c3bfe9bbc022e792645163 INFO: 172.20.0.1:40108 - "POST /node/69c3bfe9bbc022e792645163 HTTP/1.0" 202 Accepted 2026-03-27 10:59:00,841 - lava_callback - INFO - Starting async job submit for node 69c3bfe9bbc022e792645163 2026-03-27 10:59:00,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:59:00,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bfe9bbc022e792645163 HTTP/1.1" 200 None 2026-03-27 10:59:00,857 - lava_callback - DEBUG - Processing node 69c3bfe9bbc022e792645163: baseline-arm-baylibre 2026-03-27 10:59:00,857 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3bfe9bbc022e792645163 2026-03-27 10:59:00,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:59:00,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3bfe9bbc022e792645163 HTTP/1.1" 200 None 2026-03-27 10:59:00,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:59:00,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 10:59:00,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:59:00,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3bfe9bbc022e792645163 HTTP/1.1" 200 None 2026-03-27 10:59:00,919 - lava_callback - ERROR - Error processing callback for node 69c3bfe9bbc022e792645163: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35504 - "GET / HTTP/1.0" 200 OK 2026-03-27 10:59:40,793 - lava_callback - INFO - Received callback for node 69c3c011bbc022e79264516d INFO: 172.20.0.1:46368 - "POST /node/69c3c011bbc022e79264516d HTTP/1.0" 202 Accepted 2026-03-27 10:59:40,799 - lava_callback - INFO - Starting async job submit for node 69c3c011bbc022e79264516d 2026-03-27 10:59:40,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:59:40,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c011bbc022e79264516d HTTP/1.1" 200 None 2026-03-27 10:59:40,815 - lava_callback - DEBUG - Processing node 69c3c011bbc022e79264516d: baseline-arm-baylibre 2026-03-27 10:59:40,815 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c011bbc022e79264516d 2026-03-27 10:59:40,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:59:40,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c011bbc022e79264516d HTTP/1.1" 200 None 2026-03-27 10:59:40,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:59:40,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 10:59:40,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 10:59:40,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c011bbc022e79264516d HTTP/1.1" 200 None 2026-03-27 10:59:40,872 - lava_callback - ERROR - Error processing callback for node 69c3c011bbc022e79264516d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60652 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:03:06,554 - lava_callback - INFO - Received callback for node 69c3c0dcbbc022e79264519b INFO: 172.20.0.1:60654 - "POST /node/69c3c0dcbbc022e79264519b HTTP/1.0" 202 Accepted 2026-03-27 11:03:06,560 - lava_callback - INFO - Starting async job submit for node 69c3c0dcbbc022e79264519b 2026-03-27 11:03:06,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:03:06,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c0dcbbc022e79264519b HTTP/1.1" 200 None 2026-03-27 11:03:06,580 - lava_callback - DEBUG - Processing node 69c3c0dcbbc022e79264519b: baseline-arm-clabbe 2026-03-27 11:03:06,580 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c0dcbbc022e79264519b 2026-03-27 11:03:06,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:03:06,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c0dcbbc022e79264519b HTTP/1.1" 200 None 2026-03-27 11:03:06,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:03:06,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 11:03:06,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:03:06,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c0dcbbc022e79264519b HTTP/1.1" 200 None 2026-03-27 11:03:06,632 - lava_callback - ERROR - Error processing callback for node 69c3c0dcbbc022e79264519b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34432 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:03:46,537 - lava_callback - INFO - Received callback for node 69c3c106bbc022e7926451a8 INFO: 172.20.0.1:34440 - "POST /node/69c3c106bbc022e7926451a8 HTTP/1.0" 202 Accepted 2026-03-27 11:03:46,543 - lava_callback - INFO - Starting async job submit for node 69c3c106bbc022e7926451a8 2026-03-27 11:03:46,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:03:46,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c106bbc022e7926451a8 HTTP/1.1" 200 None 2026-03-27 11:03:46,559 - lava_callback - DEBUG - Processing node 69c3c106bbc022e7926451a8: baseline-arm-clabbe 2026-03-27 11:03:46,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c106bbc022e7926451a8 2026-03-27 11:03:46,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:03:46,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c106bbc022e7926451a8 HTTP/1.1" 200 None 2026-03-27 11:03:46,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:03:46,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 11:03:46,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:03:46,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c106bbc022e7926451a8 HTTP/1.1" 200 None 2026-03-27 11:03:46,623 - lava_callback - ERROR - Error processing callback for node 69c3c106bbc022e7926451a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33262 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:04:26,897 - lava_callback - INFO - Received callback for node 69c3c130bbc022e7926451b2 INFO: 172.20.0.1:33276 - "POST /node/69c3c130bbc022e7926451b2 HTTP/1.0" 202 Accepted 2026-03-27 11:04:26,902 - lava_callback - INFO - Starting async job submit for node 69c3c130bbc022e7926451b2 2026-03-27 11:04:26,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:04:26,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c130bbc022e7926451b2 HTTP/1.1" 200 None 2026-03-27 11:04:26,918 - lava_callback - DEBUG - Processing node 69c3c130bbc022e7926451b2: baseline-arm-clabbe 2026-03-27 11:04:26,919 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c130bbc022e7926451b2 2026-03-27 11:04:26,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:04:26,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c130bbc022e7926451b2 HTTP/1.1" 200 None 2026-03-27 11:04:26,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:04:26,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 11:04:26,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:04:26,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c130bbc022e7926451b2 HTTP/1.1" 200 None 2026-03-27 11:04:26,975 - lava_callback - ERROR - Error processing callback for node 69c3c130bbc022e7926451b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52214 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:05:06,725 - lava_callback - INFO - Received callback for node 69c3c159bbc022e7926451bc INFO: 172.20.0.1:52218 - "POST /node/69c3c159bbc022e7926451bc HTTP/1.0" 202 Accepted 2026-03-27 11:05:06,730 - lava_callback - INFO - Starting async job submit for node 69c3c159bbc022e7926451bc 2026-03-27 11:05:06,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:05:06,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c159bbc022e7926451bc HTTP/1.1" 200 None 2026-03-27 11:05:06,750 - lava_callback - DEBUG - Processing node 69c3c159bbc022e7926451bc: baseline-arm-clabbe 2026-03-27 11:05:06,750 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c159bbc022e7926451bc 2026-03-27 11:05:06,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:05:06,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c159bbc022e7926451bc HTTP/1.1" 200 None 2026-03-27 11:05:06,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:05:06,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 11:05:06,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:05:06,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c159bbc022e7926451bc HTTP/1.1" 200 None 2026-03-27 11:05:06,831 - lava_callback - ERROR - Error processing callback for node 69c3c159bbc022e7926451bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52084 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:05:46,671 - lava_callback - INFO - Received callback for node 69c3c182bbc022e7926451cb INFO: 172.20.0.1:52094 - "POST /node/69c3c182bbc022e7926451cb HTTP/1.0" 202 Accepted 2026-03-27 11:05:46,677 - lava_callback - INFO - Starting async job submit for node 69c3c182bbc022e7926451cb 2026-03-27 11:05:46,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:05:46,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c182bbc022e7926451cb HTTP/1.1" 200 None 2026-03-27 11:05:46,692 - lava_callback - DEBUG - Processing node 69c3c182bbc022e7926451cb: baseline-arm-clabbe 2026-03-27 11:05:46,693 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c182bbc022e7926451cb 2026-03-27 11:05:46,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:05:46,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c182bbc022e7926451cb HTTP/1.1" 200 None 2026-03-27 11:05:46,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:05:46,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b94b6778148209fd336c HTTP/1.1" 200 None 2026-03-27 11:05:46,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:05:46,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c182bbc022e7926451cb HTTP/1.1" 200 None 2026-03-27 11:05:46,760 - lava_callback - ERROR - Error processing callback for node 69c3c182bbc022e7926451cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42020 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:12:57,897 - lava_callback - INFO - Received callback for node 69c633342cb62322ad969c2c INFO: 172.20.0.1:42024 - "POST /node/69c633342cb62322ad969c2c HTTP/1.0" 202 Accepted 2026-03-27 11:12:57,904 - lava_callback - INFO - Starting async job submit for node 69c633342cb62322ad969c2c 2026-03-27 11:12:57,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:12:57,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c633342cb62322ad969c2c HTTP/1.1" 200 None 2026-03-27 11:12:57,969 - lava_callback - DEBUG - Processing node 69c633342cb62322ad969c2c: baseline-arm64 2026-03-27 11:12:58,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:12:58,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:12:58,676 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c633342cb62322ad969c2c/log.txt.gz 2026-03-27 11:12:58,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:12:59,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:12:59,176 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c633342cb62322ad969c2c/lava-logs.yaml 2026-03-27 11:12:59,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:12:59,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:12:59,378 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c633342cb62322ad969c2c/lava_callback.json.gz 2026-03-27 11:12:59,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:12:59,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c633342cb62322ad969c2c HTTP/1.1" 200 None 2026-03-27 11:12:59,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:12:59,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 11:12:59,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:12:59,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c633342cb62322ad969c2c HTTP/1.1" 200 None 2026-03-27 11:12:59,529 - lava_callback - INFO - Completed processing callback for node 69c633342cb62322ad969c2c 2026-03-27 11:13:04,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:13:04,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34178 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:13:30,240 - lava_callback - INFO - Received callback for node 69c661d9b98d4d3c02069096 INFO: 172.20.0.1:60106 - "POST /node/69c661d9b98d4d3c02069096 HTTP/1.0" 202 Accepted 2026-03-27 11:13:30,285 - lava_callback - INFO - Starting async job submit for node 69c661d9b98d4d3c02069096 2026-03-27 11:13:30,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:13:30,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c661d9b98d4d3c02069096 HTTP/1.1" 200 None 2026-03-27 11:13:30,325 - lava_callback - DEBUG - Processing node 69c661d9b98d4d3c02069096: baseline-arm-pengutronix 2026-03-27 11:13:30,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:13:30,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:13:30,553 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c661d9b98d4d3c02069096/log.txt.gz 2026-03-27 11:13:30,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:13:30,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:13:30,791 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c661d9b98d4d3c02069096/lava-logs.yaml 2026-03-27 11:13:30,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:13:30,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:13:30,914 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c661d9b98d4d3c02069096/lava_callback.json.gz DEBUG: Setup failed for 69c661d9b98d4d3c02069096. Transitting job node result: incomplete -> fail 2026-03-27 11:13:30,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:13:30,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c661d9b98d4d3c02069096 HTTP/1.1" 200 None 2026-03-27 11:13:30,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:13:31,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63ce4b98d4d3c02067dbf HTTP/1.1" 200 None 2026-03-27 11:13:31,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:13:31,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c661d9b98d4d3c02069096 HTTP/1.1" 200 None 2026-03-27 11:13:31,098 - lava_callback - INFO - Completed processing callback for node 69c661d9b98d4d3c02069096 2026-03-27 11:13:34,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:13:34,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39450 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:15:54,962 - lava_callback - INFO - Received callback for node 69c63fcbb98d4d3c0206843c INFO: 172.20.0.1:44686 - "POST /node/69c63fcbb98d4d3c0206843c HTTP/1.0" 202 Accepted 2026-03-27 11:15:54,969 - lava_callback - INFO - Starting async job submit for node 69c63fcbb98d4d3c0206843c 2026-03-27 11:15:54,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:15:55,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63fcbb98d4d3c0206843c HTTP/1.1" 200 None 2026-03-27 11:15:55,017 - lava_callback - DEBUG - Processing node 69c63fcbb98d4d3c0206843c: baseline-nfs-arm64 2026-03-27 11:15:55,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:15:55,717 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:15:55,719 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c63fcbb98d4d3c0206843c/log.txt.gz 2026-03-27 11:15:55,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:44688 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:15:57,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:15:57,186 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c63fcbb98d4d3c0206843c/lava-logs.yaml 2026-03-27 11:15:57,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:15:57,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:15:57,370 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c63fcbb98d4d3c0206843c/lava_callback.json.gz 2026-03-27 11:15:57,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:15:57,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63fcbb98d4d3c0206843c HTTP/1.1" 200 None 2026-03-27 11:15:57,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:15:57,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 11:15:57,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:15:57,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c63fcbb98d4d3c0206843c HTTP/1.1" 200 None 2026-03-27 11:15:57,641 - lava_callback - INFO - Completed processing callback for node 69c63fcbb98d4d3c0206843c 2026-03-27 11:16:04,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:16:04,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53284 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:17:52,719 - lava_callback - INFO - Received callback for node 69c646a4b98d4d3c02068870 INFO: 172.20.0.1:48208 - "POST /node/69c646a4b98d4d3c02068870 HTTP/1.0" 202 Accepted 2026-03-27 11:17:52,726 - lava_callback - INFO - Starting async job submit for node 69c646a4b98d4d3c02068870 2026-03-27 11:17:52,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:17:52,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c646a4b98d4d3c02068870 HTTP/1.1" 200 None 2026-03-27 11:17:52,776 - lava_callback - DEBUG - Processing node 69c646a4b98d4d3c02068870: baseline-arm64 2026-03-27 11:17:53,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:17:53,488 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:17:53,490 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c646a4b98d4d3c02068870/log.txt.gz 2026-03-27 11:17:53,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:17:53,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:17:53,971 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c646a4b98d4d3c02068870/lava-logs.yaml 2026-03-27 11:17:53,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:17:54,164 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:17:54,166 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c646a4b98d4d3c02068870/lava_callback.json.gz 2026-03-27 11:17:54,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:17:54,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c646a4b98d4d3c02068870 HTTP/1.1" 200 None 2026-03-27 11:17:54,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:17:54,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 11:17:54,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:17:54,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c646a4b98d4d3c02068870 HTTP/1.1" 200 None 2026-03-27 11:17:54,305 - lava_callback - INFO - Completed processing callback for node 69c646a4b98d4d3c02068870 INFO: 172.20.0.1:48210 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:18:04,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:18:04,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57060 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:19:01,832 - lava_callback - INFO - Received callback for node 69c3c498bbc022e792645237 INFO: 172.20.0.1:36838 - "POST /node/69c3c498bbc022e792645237 HTTP/1.0" 202 Accepted 2026-03-27 11:19:01,838 - lava_callback - INFO - Starting async job submit for node 69c3c498bbc022e792645237 2026-03-27 11:19:01,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:19:01,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c498bbc022e792645237 HTTP/1.1" 200 None 2026-03-27 11:19:01,854 - lava_callback - DEBUG - Processing node 69c3c498bbc022e792645237: baseline-arm64-baylibre 2026-03-27 11:19:01,854 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c498bbc022e792645237 2026-03-27 11:19:01,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:19:01,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c498bbc022e792645237 HTTP/1.1" 200 None 2026-03-27 11:19:01,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:19:01,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:19:01,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:19:01,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c498bbc022e792645237 HTTP/1.1" 200 None 2026-03-27 11:19:01,917 - lava_callback - ERROR - Error processing callback for node 69c3c498bbc022e792645237: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33144 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:19:38,971 - lava_callback - INFO - Received callback for node 69c6632bb98d4d3c020690d6 INFO: 172.20.0.1:33156 - "POST /node/69c6632bb98d4d3c020690d6 HTTP/1.0" 202 Accepted 2026-03-27 11:19:38,979 - lava_callback - INFO - Starting async job submit for node 69c6632bb98d4d3c020690d6 2026-03-27 11:19:39,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:19:39,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6632bb98d4d3c020690d6 HTTP/1.1" 200 None 2026-03-27 11:19:39,038 - lava_callback - DEBUG - Processing node 69c6632bb98d4d3c020690d6: baseline-nfs-arm64 2026-03-27 11:19:40,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:19:40,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:19:40,208 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6632bb98d4d3c020690d6/log.txt.gz 2026-03-27 11:19:40,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:19:41,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:19:41,049 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6632bb98d4d3c020690d6/lava-logs.yaml 2026-03-27 11:19:41,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:19:41,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:19:41,242 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6632bb98d4d3c020690d6/lava_callback.json.gz DEBUG: Setup failed for 69c6632bb98d4d3c020690d6. Transitting job node result: incomplete -> fail 2026-03-27 11:19:41,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:19:41,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6632bb98d4d3c020690d6 HTTP/1.1" 200 None 2026-03-27 11:19:41,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:19:41,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 11:19:41,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:19:41,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6632bb98d4d3c020690d6 HTTP/1.1" 200 None 2026-03-27 11:19:41,494 - lava_callback - INFO - Completed processing callback for node 69c6632bb98d4d3c020690d6 INFO: 172.20.0.1:58970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45294 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:20:04,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:20:04,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39698 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:20:17,719 - lava_callback - INFO - Received callback for node 69c64762b98d4d3c020688a1 INFO: 172.20.0.1:39712 - "POST /node/69c64762b98d4d3c020688a1 HTTP/1.0" 202 Accepted 2026-03-27 11:20:17,726 - lava_callback - INFO - Starting async job submit for node 69c64762b98d4d3c020688a1 2026-03-27 11:20:17,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:20:17,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64762b98d4d3c020688a1 HTTP/1.1" 200 None 2026-03-27 11:20:17,770 - lava_callback - DEBUG - Processing node 69c64762b98d4d3c020688a1: baseline-nfs-arm64 2026-03-27 11:20:18,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:20:18,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:20:18,424 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64762b98d4d3c020688a1/log.txt.gz 2026-03-27 11:20:18,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:20:19,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:20:19,082 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64762b98d4d3c020688a1/lava-logs.yaml 2026-03-27 11:20:19,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:20:19,249 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:20:19,251 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64762b98d4d3c020688a1/lava_callback.json.gz 2026-03-27 11:20:19,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:20:19,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64762b98d4d3c020688a1 HTTP/1.1" 200 None 2026-03-27 11:20:19,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:20:19,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 11:20:19,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:20:19,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64762b98d4d3c020688a1 HTTP/1.1" 200 None 2026-03-27 11:20:19,413 - lava_callback - INFO - Completed processing callback for node 69c64762b98d4d3c020688a1 2026-03-27 11:20:21,954 - lava_callback - INFO - Received callback for node 69c3c4e9bbc022e792645244 INFO: 172.20.0.1:49438 - "POST /node/69c3c4e9bbc022e792645244 HTTP/1.0" 202 Accepted 2026-03-27 11:20:21,960 - lava_callback - INFO - Starting async job submit for node 69c3c4e9bbc022e792645244 2026-03-27 11:20:21,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:20:21,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c4e9bbc022e792645244 HTTP/1.1" 200 None 2026-03-27 11:20:21,980 - lava_callback - DEBUG - Processing node 69c3c4e9bbc022e792645244: baseline-arm64-baylibre 2026-03-27 11:20:21,981 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c4e9bbc022e792645244 2026-03-27 11:20:21,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:20:22,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c4e9bbc022e792645244 HTTP/1.1" 200 None 2026-03-27 11:20:22,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:20:22,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:20:22,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:20:22,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c4e9bbc022e792645244 HTTP/1.1" 200 None 2026-03-27 11:20:22,048 - lava_callback - ERROR - Error processing callback for node 69c3c4e9bbc022e792645244: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49442 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:20:34,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:20:34,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:48326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58208 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:21:41,957 - lava_callback - INFO - Received callback for node 69c3c539bbc022e792645252 INFO: 172.20.0.1:36820 - "POST /node/69c3c539bbc022e792645252 HTTP/1.0" 202 Accepted 2026-03-27 11:21:41,963 - lava_callback - INFO - Starting async job submit for node 69c3c539bbc022e792645252 2026-03-27 11:21:41,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:21:41,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c539bbc022e792645252 HTTP/1.1" 200 None 2026-03-27 11:21:41,979 - lava_callback - DEBUG - Processing node 69c3c539bbc022e792645252: baseline-arm64-baylibre 2026-03-27 11:21:41,979 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c539bbc022e792645252 2026-03-27 11:21:41,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:21:42,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c539bbc022e792645252 HTTP/1.1" 200 None 2026-03-27 11:21:42,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:21:42,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:21:42,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:21:42,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c539bbc022e792645252 HTTP/1.1" 200 None 2026-03-27 11:21:42,039 - lava_callback - ERROR - Error processing callback for node 69c3c539bbc022e792645252: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44678 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:23:02,173 - lava_callback - INFO - Received callback for node 69c3c587bbc022e792645261 INFO: 172.20.0.1:35958 - "POST /node/69c3c587bbc022e792645261 HTTP/1.0" 202 Accepted 2026-03-27 11:23:02,179 - lava_callback - INFO - Starting async job submit for node 69c3c587bbc022e792645261 2026-03-27 11:23:02,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:23:02,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c587bbc022e792645261 HTTP/1.1" 200 None 2026-03-27 11:23:02,195 - lava_callback - DEBUG - Processing node 69c3c587bbc022e792645261: baseline-arm64-baylibre 2026-03-27 11:23:02,195 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c587bbc022e792645261 2026-03-27 11:23:02,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:23:02,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c587bbc022e792645261 HTTP/1.1" 200 None 2026-03-27 11:23:02,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:23:02,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:23:02,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:23:02,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c587bbc022e792645261 HTTP/1.1" 200 None 2026-03-27 11:23:02,265 - lava_callback - ERROR - Error processing callback for node 69c3c587bbc022e792645261: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41366 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:24:02,043 - lava_callback - INFO - Received callback for node 69c3c5d0bbc022e792645270 INFO: 172.20.0.1:41582 - "POST /node/69c3c5d0bbc022e792645270 HTTP/1.0" 202 Accepted 2026-03-27 11:24:02,050 - lava_callback - INFO - Starting async job submit for node 69c3c5d0bbc022e792645270 2026-03-27 11:24:02,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:24:02,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c5d0bbc022e792645270 HTTP/1.1" 200 None 2026-03-27 11:24:02,067 - lava_callback - DEBUG - Processing node 69c3c5d0bbc022e792645270: baseline-arm64-baylibre 2026-03-27 11:24:02,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c5d0bbc022e792645270 2026-03-27 11:24:02,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:24:02,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c5d0bbc022e792645270 HTTP/1.1" 200 None 2026-03-27 11:24:02,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:24:02,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:24:02,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:24:02,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c5d0bbc022e792645270 HTTP/1.1" 200 None 2026-03-27 11:24:02,120 - lava_callback - ERROR - Error processing callback for node 69c3c5d0bbc022e792645270: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37294 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45034 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:27:07,652 - lava_callback - INFO - Received callback for node 69c3c67abbc022e79264528e INFO: 172.20.0.1:45046 - "POST /node/69c3c67abbc022e79264528e HTTP/1.0" 202 Accepted 2026-03-27 11:27:07,658 - lava_callback - INFO - Starting async job submit for node 69c3c67abbc022e79264528e 2026-03-27 11:27:07,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:27:07,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c67abbc022e79264528e HTTP/1.1" 200 None 2026-03-27 11:27:07,673 - lava_callback - DEBUG - Processing node 69c3c67abbc022e79264528e: baseline-arm64-clabbe 2026-03-27 11:27:07,674 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c67abbc022e79264528e 2026-03-27 11:27:07,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:27:07,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c67abbc022e79264528e HTTP/1.1" 200 None 2026-03-27 11:27:07,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:27:07,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:27:07,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:27:07,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c67abbc022e79264528e HTTP/1.1" 200 None 2026-03-27 11:27:07,744 - lava_callback - ERROR - Error processing callback for node 69c3c67abbc022e79264528e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55804 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:28:27,688 - lava_callback - INFO - Received callback for node 69c3c6cbbbc022e7926452a3 INFO: 172.20.0.1:55820 - "POST /node/69c3c6cbbbc022e7926452a3 HTTP/1.0" 202 Accepted 2026-03-27 11:28:27,693 - lava_callback - INFO - Starting async job submit for node 69c3c6cbbbc022e7926452a3 2026-03-27 11:28:27,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:28:27,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c6cbbbc022e7926452a3 HTTP/1.1" 200 None 2026-03-27 11:28:27,716 - lava_callback - DEBUG - Processing node 69c3c6cbbbc022e7926452a3: baseline-arm64-clabbe 2026-03-27 11:28:27,716 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c6cbbbc022e7926452a3 2026-03-27 11:28:27,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:28:27,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c6cbbbc022e7926452a3 HTTP/1.1" 200 None 2026-03-27 11:28:27,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:28:27,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:28:27,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:28:27,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c6cbbbc022e7926452a3 HTTP/1.1" 200 None 2026-03-27 11:28:27,767 - lava_callback - ERROR - Error processing callback for node 69c3c6cbbbc022e7926452a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42650 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:28:49,635 - lava_callback - INFO - Received callback for node 69c6496bb98d4d3c02068955 INFO: 172.20.0.1:42656 - "POST /node/69c6496bb98d4d3c02068955 HTTP/1.0" 202 Accepted 2026-03-27 11:28:49,642 - lava_callback - INFO - Starting async job submit for node 69c6496bb98d4d3c02068955 2026-03-27 11:28:49,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:28:49,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6496bb98d4d3c02068955 HTTP/1.1" 200 None 2026-03-27 11:28:49,690 - lava_callback - DEBUG - Processing node 69c6496bb98d4d3c02068955: baseline-arm64 2026-03-27 11:28:50,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:28:50,330 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:28:50,332 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6496bb98d4d3c02068955/log.txt.gz 2026-03-27 11:28:50,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:28:50,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:28:50,835 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6496bb98d4d3c02068955/lava-logs.yaml 2026-03-27 11:28:50,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:28:51,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:28:51,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6496bb98d4d3c02068955/lava_callback.json.gz 2026-03-27 11:28:51,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:28:51,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6496bb98d4d3c02068955 HTTP/1.1" 200 None 2026-03-27 11:28:51,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:28:51,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 11:28:51,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:28:51,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6496bb98d4d3c02068955 HTTP/1.1" 200 None 2026-03-27 11:28:51,176 - lava_callback - INFO - Completed processing callback for node 69c6496bb98d4d3c02068955 INFO: 172.20.0.1:38980 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:29:04,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:29:04,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45778 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:29:47,731 - lava_callback - INFO - Received callback for node 69c3c722bbc022e7926452b2 INFO: 172.20.0.1:45782 - "POST /node/69c3c722bbc022e7926452b2 HTTP/1.0" 202 Accepted 2026-03-27 11:29:47,736 - lava_callback - INFO - Starting async job submit for node 69c3c722bbc022e7926452b2 2026-03-27 11:29:47,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:29:47,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c722bbc022e7926452b2 HTTP/1.1" 200 None 2026-03-27 11:29:47,753 - lava_callback - DEBUG - Processing node 69c3c722bbc022e7926452b2: baseline-arm64-clabbe 2026-03-27 11:29:47,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c722bbc022e7926452b2 2026-03-27 11:29:47,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:29:47,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c722bbc022e7926452b2 HTTP/1.1" 200 None 2026-03-27 11:29:47,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:29:47,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:29:47,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:29:47,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c722bbc022e7926452b2 HTTP/1.1" 200 None 2026-03-27 11:29:47,816 - lava_callback - ERROR - Error processing callback for node 69c3c722bbc022e7926452b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51102 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:31:07,833 - lava_callback - INFO - Received callback for node 69c3c772bbc022e7926452bb INFO: 172.20.0.1:51114 - "POST /node/69c3c772bbc022e7926452bb HTTP/1.0" 202 Accepted 2026-03-27 11:31:07,838 - lava_callback - INFO - Starting async job submit for node 69c3c772bbc022e7926452bb 2026-03-27 11:31:07,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:31:07,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c772bbc022e7926452bb HTTP/1.1" 200 None 2026-03-27 11:31:07,855 - lava_callback - DEBUG - Processing node 69c3c772bbc022e7926452bb: baseline-arm64-clabbe 2026-03-27 11:31:07,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c772bbc022e7926452bb 2026-03-27 11:31:07,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:31:07,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c772bbc022e7926452bb HTTP/1.1" 200 None 2026-03-27 11:31:07,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:31:07,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:31:07,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:31:07,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c772bbc022e7926452bb HTTP/1.1" 200 None 2026-03-27 11:31:07,912 - lava_callback - ERROR - Error processing callback for node 69c3c772bbc022e7926452bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54306 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:32:14,749 - lava_callback - INFO - Received callback for node 69c6497cb98d4d3c02068999 INFO: 172.20.0.1:55958 - "POST /node/69c6497cb98d4d3c02068999 HTTP/1.0" 202 Accepted 2026-03-27 11:32:14,756 - lava_callback - INFO - Starting async job submit for node 69c6497cb98d4d3c02068999 2026-03-27 11:32:14,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:14,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6497cb98d4d3c02068999 HTTP/1.1" 200 None 2026-03-27 11:32:14,806 - lava_callback - DEBUG - Processing node 69c6497cb98d4d3c02068999: baseline-nfs-arm64 INFO: 172.20.0.1:55968 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 11:32:15,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:32:15,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:32:15,476 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497cb98d4d3c02068999/log.txt.gz 2026-03-27 11:32:15,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:32:16,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:32:16,008 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497cb98d4d3c02068999/lava-logs.yaml 2026-03-27 11:32:16,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:32:16,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:32:16,169 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6497cb98d4d3c02068999/lava_callback.json.gz 2026-03-27 11:32:16,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:16,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6497cb98d4d3c02068999 HTTP/1.1" 200 None 2026-03-27 11:32:16,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:16,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 11:32:16,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:16,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6497cb98d4d3c02068999 HTTP/1.1" 200 None 2026-03-27 11:32:16,303 - lava_callback - INFO - Completed processing callback for node 69c6497cb98d4d3c02068999 INFO: 172.20.0.1:55974 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:32:18,441 - lava_callback - INFO - Received callback for node 69c650afb98d4d3c02068cfe INFO: 172.20.0.1:55978 - "POST /node/69c650afb98d4d3c02068cfe HTTP/1.0" 202 Accepted 2026-03-27 11:32:18,448 - lava_callback - INFO - Starting async job submit for node 69c650afb98d4d3c02068cfe 2026-03-27 11:32:18,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:18,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c650afb98d4d3c02068cfe HTTP/1.1" 200 None 2026-03-27 11:32:18,490 - lava_callback - DEBUG - Processing node 69c650afb98d4d3c02068cfe: baseline-nfs-arm64 2026-03-27 11:32:18,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:32:19,131 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:32:19,133 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c650afb98d4d3c02068cfe/log.txt.gz 2026-03-27 11:32:19,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:32:19,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:32:19,575 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c650afb98d4d3c02068cfe/lava-logs.yaml 2026-03-27 11:32:19,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 11:32:19,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 11:32:19,757 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c650afb98d4d3c02068cfe/lava_callback.json.gz 2026-03-27 11:32:19,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:19,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c650afb98d4d3c02068cfe HTTP/1.1" 200 None 2026-03-27 11:32:19,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:19,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 11:32:19,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:19,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c650afb98d4d3c02068cfe HTTP/1.1" 200 None 2026-03-27 11:32:19,936 - lava_callback - INFO - Completed processing callback for node 69c650afb98d4d3c02068cfe INFO: 172.20.0.1:40730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40746 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:32:27,850 - lava_callback - INFO - Received callback for node 69c3c7bcbbc022e7926452d0 INFO: 172.20.0.1:40752 - "POST /node/69c3c7bcbbc022e7926452d0 HTTP/1.0" 202 Accepted 2026-03-27 11:32:27,856 - lava_callback - INFO - Starting async job submit for node 69c3c7bcbbc022e7926452d0 2026-03-27 11:32:27,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:27,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c7bcbbc022e7926452d0 HTTP/1.1" 200 None 2026-03-27 11:32:27,877 - lava_callback - DEBUG - Processing node 69c3c7bcbbc022e7926452d0: baseline-arm64-clabbe 2026-03-27 11:32:27,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c7bcbbc022e7926452d0 2026-03-27 11:32:27,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:27,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c7bcbbc022e7926452d0 HTTP/1.1" 200 None 2026-03-27 11:32:27,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:27,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:32:27,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:27,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c7bcbbc022e7926452d0 HTTP/1.1" 200 None 2026-03-27 11:32:27,934 - lava_callback - ERROR - Error processing callback for node 69c3c7bcbbc022e7926452d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 11:32:34,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:32:34,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45832 - "GET / HTTP/1.0" 200 OK 2026-03-27 11:33:47,895 - lava_callback - INFO - Received callback for node 69c3c813bbc022e7926452df INFO: 172.20.0.1:45838 - "POST /node/69c3c813bbc022e7926452df HTTP/1.0" 202 Accepted 2026-03-27 11:33:47,901 - lava_callback - INFO - Starting async job submit for node 69c3c813bbc022e7926452df 2026-03-27 11:33:47,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:33:47,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c813bbc022e7926452df HTTP/1.1" 200 None 2026-03-27 11:33:47,916 - lava_callback - DEBUG - Processing node 69c3c813bbc022e7926452df: baseline-arm64-clabbe 2026-03-27 11:33:47,916 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3c813bbc022e7926452df 2026-03-27 11:33:47,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:33:47,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3c813bbc022e7926452df HTTP/1.1" 200 None 2026-03-27 11:33:47,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:33:47,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c2b9626778148209fd337b HTTP/1.1" 200 None 2026-03-27 11:33:47,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 11:33:47,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3c813bbc022e7926452df HTTP/1.1" 200 None 2026-03-27 11:33:47,978 - lava_callback - ERROR - Error processing callback for node 69c3c813bbc022e7926452df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57168 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56234 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:16:08,127 - lava_callback - INFO - Received callback for node 69c65736b98d4d3c02068e61 INFO: 172.20.0.1:56242 - "POST /node/69c65736b98d4d3c02068e61 HTTP/1.0" 202 Accepted 2026-03-27 12:16:08,134 - lava_callback - INFO - Starting async job submit for node 69c65736b98d4d3c02068e61 2026-03-27 12:16:08,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:16:08,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65736b98d4d3c02068e61 HTTP/1.1" 200 None 2026-03-27 12:16:08,183 - lava_callback - DEBUG - Processing node 69c65736b98d4d3c02068e61: baseline-arm64 2026-03-27 12:16:08,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:16:08,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:16:08,834 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65736b98d4d3c02068e61/log.txt.gz 2026-03-27 12:16:09,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:16:09,470 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:16:09,472 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65736b98d4d3c02068e61/lava-logs.yaml 2026-03-27 12:16:09,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:16:09,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:16:09,654 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65736b98d4d3c02068e61/lava_callback.json.gz 2026-03-27 12:16:09,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:16:09,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65736b98d4d3c02068e61 HTTP/1.1" 200 None 2026-03-27 12:16:09,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:16:09,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 12:16:09,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:16:09,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65736b98d4d3c02068e61 HTTP/1.1" 200 None 2026-03-27 12:16:09,817 - lava_callback - INFO - Completed processing callback for node 69c65736b98d4d3c02068e61 INFO: 172.20.0.1:40922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52198 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:16:34,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:16:34,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50104 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:18:12,249 - lava_callback - INFO - Received callback for node 69c65a58b98d4d3c02068ef6 INFO: 172.20.0.1:55748 - "POST /node/69c65a58b98d4d3c02068ef6 HTTP/1.0" 202 Accepted 2026-03-27 12:18:12,255 - lava_callback - INFO - Starting async job submit for node 69c65a58b98d4d3c02068ef6 2026-03-27 12:18:12,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:18:12,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65a58b98d4d3c02068ef6 HTTP/1.1" 200 None 2026-03-27 12:18:12,296 - lava_callback - DEBUG - Processing node 69c65a58b98d4d3c02068ef6: baseline-nfs-arm64 2026-03-27 12:18:12,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:18:13,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:18:13,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65a58b98d4d3c02068ef6/log.txt.gz 2026-03-27 12:18:13,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:18:13,580 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:18:13,582 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65a58b98d4d3c02068ef6/lava-logs.yaml 2026-03-27 12:18:13,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:18:13,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:18:13,776 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65a58b98d4d3c02068ef6/lava_callback.json.gz 2026-03-27 12:18:13,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:18:13,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65a58b98d4d3c02068ef6 HTTP/1.1" 200 None 2026-03-27 12:18:13,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:18:13,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ebd8f63ce05331e2a492 HTTP/1.1" 200 None 2026-03-27 12:18:13,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:18:13,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65a58b98d4d3c02068ef6 HTTP/1.1" 200 None 2026-03-27 12:18:13,911 - lava_callback - INFO - Completed processing callback for node 69c65a58b98d4d3c02068ef6 INFO: 172.20.0.1:55754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53278 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:18:34,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:18:34,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56172 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51406 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:25:06,740 - lava_callback - INFO - Received callback for node 69c65d2bb98d4d3c02068f93 INFO: 172.20.0.1:51420 - "POST /node/69c65d2bb98d4d3c02068f93 HTTP/1.0" 202 Accepted 2026-03-27 12:25:06,747 - lava_callback - INFO - Starting async job submit for node 69c65d2bb98d4d3c02068f93 2026-03-27 12:25:06,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:25:06,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65d2bb98d4d3c02068f93 HTTP/1.1" 200 None 2026-03-27 12:25:06,806 - lava_callback - DEBUG - Processing node 69c65d2bb98d4d3c02068f93: baseline-arm64 2026-03-27 12:25:07,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:25:07,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:25:07,483 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65d2bb98d4d3c02068f93/log.txt.gz 2026-03-27 12:25:07,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:25:07,998 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:25:08,000 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65d2bb98d4d3c02068f93/lava-logs.yaml 2026-03-27 12:25:08,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:25:08,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:25:08,193 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65d2bb98d4d3c02068f93/lava_callback.json.gz 2026-03-27 12:25:08,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:25:08,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65d2bb98d4d3c02068f93 HTTP/1.1" 200 None 2026-03-27 12:25:08,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:25:08,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 12:25:08,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:25:08,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65d2bb98d4d3c02068f93 HTTP/1.1" 200 None 2026-03-27 12:25:08,343 - lava_callback - INFO - Completed processing callback for node 69c65d2bb98d4d3c02068f93 INFO: 172.20.0.1:56250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56434 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:25:34,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:25:34,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55280 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:27:25,438 - lava_callback - INFO - Received callback for node 69c65eb3b98d4d3c02068fda INFO: 172.20.0.1:55282 - "POST /node/69c65eb3b98d4d3c02068fda HTTP/1.0" 202 Accepted 2026-03-27 12:27:25,445 - lava_callback - INFO - Starting async job submit for node 69c65eb3b98d4d3c02068fda 2026-03-27 12:27:25,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:27:25,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65eb3b98d4d3c02068fda HTTP/1.1" 200 None 2026-03-27 12:27:25,486 - lava_callback - DEBUG - Processing node 69c65eb3b98d4d3c02068fda: baseline-nfs-arm64 2026-03-27 12:27:26,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:27:26,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:27:26,188 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65eb3b98d4d3c02068fda/log.txt.gz 2026-03-27 12:27:26,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:55284 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:27:26,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:27:26,682 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65eb3b98d4d3c02068fda/lava-logs.yaml 2026-03-27 12:27:26,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:27:26,875 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:27:26,877 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c65eb3b98d4d3c02068fda/lava_callback.json.gz 2026-03-27 12:27:26,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:27:26,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65eb3b98d4d3c02068fda HTTP/1.1" 200 None 2026-03-27 12:27:26,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:27:26,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 12:27:26,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:27:26,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65eb3b98d4d3c02068fda HTTP/1.1" 200 None 2026-03-27 12:27:27,057 - lava_callback - INFO - Completed processing callback for node 69c65eb3b98d4d3c02068fda 2026-03-27 12:27:34,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:27:34,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43806 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:28:12,431 - lava_callback - INFO - Received callback for node 69c65feab98d4d3c0206900d INFO: 172.20.0.1:42442 - "POST /node/69c65feab98d4d3c0206900d HTTP/1.0" 202 Accepted 2026-03-27 12:28:12,438 - lava_callback - INFO - Starting async job submit for node 69c65feab98d4d3c0206900d 2026-03-27 12:28:12,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:28:12,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65feab98d4d3c0206900d HTTP/1.1" 200 None 2026-03-27 12:28:12,478 - lava_callback - DEBUG - Processing node 69c65feab98d4d3c0206900d: baseline-arm64 2026-03-27 12:28:13,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:28:13,154 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:28:13,155 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65feab98d4d3c0206900d/log.txt.gz 2026-03-27 12:28:13,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:28:13,634 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:28:13,636 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65feab98d4d3c0206900d/lava-logs.yaml 2026-03-27 12:28:13,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:28:13,810 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:28:13,812 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c65feab98d4d3c0206900d/lava_callback.json.gz 2026-03-27 12:28:13,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:28:13,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c65feab98d4d3c0206900d HTTP/1.1" 200 None 2026-03-27 12:28:13,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:28:13,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 12:28:13,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:28:13,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c65feab98d4d3c0206900d HTTP/1.1" 200 None 2026-03-27 12:28:13,951 - lava_callback - INFO - Completed processing callback for node 69c65feab98d4d3c0206900d INFO: 172.20.0.1:42458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37620 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:28:34,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:28:34,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:30:54,527 - lava_callback - INFO - Received callback for node 69c66040b98d4d3c02069022 INFO: 172.20.0.1:54462 - "POST /node/69c66040b98d4d3c02069022 HTTP/1.0" 202 Accepted 2026-03-27 12:30:54,533 - lava_callback - INFO - Starting async job submit for node 69c66040b98d4d3c02069022 2026-03-27 12:30:54,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:30:54,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c66040b98d4d3c02069022 HTTP/1.1" 200 None 2026-03-27 12:30:54,574 - lava_callback - DEBUG - Processing node 69c66040b98d4d3c02069022: baseline-nfs-arm64 2026-03-27 12:30:55,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:30:55,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:30:55,255 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c66040b98d4d3c02069022/log.txt.gz 2026-03-27 12:30:55,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:30:55,748 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:30:55,750 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c66040b98d4d3c02069022/lava-logs.yaml 2026-03-27 12:30:55,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:30:55,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:30:55,942 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c66040b98d4d3c02069022/lava_callback.json.gz 2026-03-27 12:30:55,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:30:55,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c66040b98d4d3c02069022 HTTP/1.1" 200 None 2026-03-27 12:30:55,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:30:56,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 12:30:56,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:30:56,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c66040b98d4d3c02069022 HTTP/1.1" 200 None 2026-03-27 12:30:56,094 - lava_callback - INFO - Completed processing callback for node 69c66040b98d4d3c02069022 INFO: 172.20.0.1:54468 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:31:04,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:31:04,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32850 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35748 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:36:46,079 - lava_callback - INFO - Received callback for node 69c3d6c9bbc022e792645380 INFO: 172.20.0.1:50778 - "POST /node/69c3d6c9bbc022e792645380 HTTP/1.0" 202 Accepted 2026-03-27 12:36:46,086 - lava_callback - INFO - Starting async job submit for node 69c3d6c9bbc022e792645380 2026-03-27 12:36:46,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:36:46,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d6c9bbc022e792645380 HTTP/1.1" 200 None 2026-03-27 12:36:46,102 - lava_callback - DEBUG - Processing node 69c3d6c9bbc022e792645380: baseline-arm-baylibre 2026-03-27 12:36:46,102 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d6c9bbc022e792645380 2026-03-27 12:36:46,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:36:46,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d6c9bbc022e792645380 HTTP/1.1" 200 None 2026-03-27 12:36:46,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:36:46,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 12:36:46,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:36:46,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d6c9bbc022e792645380 HTTP/1.1" 200 None 2026-03-27 12:36:46,154 - lava_callback - ERROR - Error processing callback for node 69c3d6c9bbc022e792645380: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52904 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:37:45,985 - lava_callback - INFO - Received callback for node 69c3d706bbc022e79264538c INFO: 172.20.0.1:47576 - "POST /node/69c3d706bbc022e79264538c HTTP/1.0" 202 Accepted 2026-03-27 12:37:45,990 - lava_callback - INFO - Starting async job submit for node 69c3d706bbc022e79264538c 2026-03-27 12:37:45,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:37:46,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d706bbc022e79264538c HTTP/1.1" 200 None 2026-03-27 12:37:46,007 - lava_callback - DEBUG - Processing node 69c3d706bbc022e79264538c: baseline-arm-baylibre 2026-03-27 12:37:46,007 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d706bbc022e79264538c 2026-03-27 12:37:46,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:37:46,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d706bbc022e79264538c HTTP/1.1" 200 None 2026-03-27 12:37:46,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:37:46,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 12:37:46,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:37:46,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d706bbc022e79264538c HTTP/1.1" 200 None 2026-03-27 12:37:46,081 - lava_callback - ERROR - Error processing callback for node 69c3d706bbc022e79264538c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47584 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:37:47,213 - lava_callback - INFO - Received callback for node 69c661f7b98d4d3c0206909c INFO: 172.20.0.1:47598 - "POST /node/69c661f7b98d4d3c0206909c HTTP/1.0" 202 Accepted 2026-03-27 12:37:47,220 - lava_callback - INFO - Starting async job submit for node 69c661f7b98d4d3c0206909c 2026-03-27 12:37:47,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:37:47,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c661f7b98d4d3c0206909c HTTP/1.1" 200 None 2026-03-27 12:37:47,260 - lava_callback - DEBUG - Processing node 69c661f7b98d4d3c0206909c: baseline-arm64 2026-03-27 12:37:47,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:37:47,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:37:47,963 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c661f7b98d4d3c0206909c/log.txt.gz 2026-03-27 12:37:48,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:37:48,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:37:48,476 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c661f7b98d4d3c0206909c/lava-logs.yaml 2026-03-27 12:37:48,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:37:48,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:37:48,666 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c661f7b98d4d3c0206909c/lava_callback.json.gz 2026-03-27 12:37:48,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:37:48,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c661f7b98d4d3c0206909c HTTP/1.1" 200 None 2026-03-27 12:37:48,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:37:48,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 12:37:48,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:37:48,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c661f7b98d4d3c0206909c HTTP/1.1" 200 None 2026-03-27 12:37:48,793 - lava_callback - INFO - Completed processing callback for node 69c661f7b98d4d3c0206909c INFO: 172.20.0.1:40464 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:38:04,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:38:04,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37536 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:40:23,477 - lava_callback - INFO - Received callback for node 69c662dcb98d4d3c020690c3 INFO: 172.20.0.1:41822 - "POST /node/69c662dcb98d4d3c020690c3 HTTP/1.0" 202 Accepted 2026-03-27 12:40:23,484 - lava_callback - INFO - Starting async job submit for node 69c662dcb98d4d3c020690c3 2026-03-27 12:40:23,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:40:23,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c662dcb98d4d3c020690c3 HTTP/1.1" 200 None 2026-03-27 12:40:23,525 - lava_callback - DEBUG - Processing node 69c662dcb98d4d3c020690c3: baseline-nfs-arm64 2026-03-27 12:40:24,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:41828 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:40:24,232 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:40:24,234 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c662dcb98d4d3c020690c3/log.txt.gz 2026-03-27 12:40:24,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:40:24,860 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:40:24,862 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c662dcb98d4d3c020690c3/lava-logs.yaml 2026-03-27 12:40:24,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:40:25,092 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:40:25,094 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c662dcb98d4d3c020690c3/lava_callback.json.gz 2026-03-27 12:40:25,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:40:25,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c662dcb98d4d3c020690c3 HTTP/1.1" 200 None 2026-03-27 12:40:25,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:40:25,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 12:40:25,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:40:25,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c662dcb98d4d3c020690c3 HTTP/1.1" 200 None 2026-03-27 12:40:25,252 - lava_callback - INFO - Completed processing callback for node 69c662dcb98d4d3c020690c3 INFO: 172.20.0.1:41842 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:40:34,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:40:34,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49696 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 12:41:15,358 - lava_callback - INFO - Received callback for node 69c669f8b98d4d3c020691e0 INFO: 172.20.0.1:49710 - "POST /node/69c669f8b98d4d3c020691e0 HTTP/1.0" 202 Accepted 2026-03-27 12:41:15,365 - lava_callback - INFO - Starting async job submit for node 69c669f8b98d4d3c020691e0 2026-03-27 12:41:15,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:41:15,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c669f8b98d4d3c020691e0 HTTP/1.1" 200 None 2026-03-27 12:41:15,406 - lava_callback - DEBUG - Processing node 69c669f8b98d4d3c020691e0: baseline-arm64 2026-03-27 12:41:15,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:49726 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:41:16,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:41:16,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c669f8b98d4d3c020691e0/log.txt.gz 2026-03-27 12:41:16,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:41:16,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:41:16,967 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c669f8b98d4d3c020691e0/lava-logs.yaml 2026-03-27 12:41:16,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:41:17,118 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:41:17,120 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c669f8b98d4d3c020691e0/lava_callback.json.gz 2026-03-27 12:41:17,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:41:17,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c669f8b98d4d3c020691e0 HTTP/1.1" 200 None 2026-03-27 12:41:17,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:41:17,180 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 12:41:17,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:41:17,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c669f8b98d4d3c020691e0 HTTP/1.1" 200 None 2026-03-27 12:41:17,256 - lava_callback - INFO - Completed processing callback for node 69c669f8b98d4d3c020691e0 INFO: 172.20.0.1:46298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46310 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:41:30,813 - lava_callback - INFO - Received callback for node 69c3d7e5bbc022e7926453b4 INFO: 172.20.0.1:42650 - "POST /node/69c3d7e5bbc022e7926453b4 HTTP/1.0" 202 Accepted 2026-03-27 12:41:30,819 - lava_callback - INFO - Starting async job submit for node 69c3d7e5bbc022e7926453b4 2026-03-27 12:41:30,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:41:30,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d7e5bbc022e7926453b4 HTTP/1.1" 200 None 2026-03-27 12:41:30,840 - lava_callback - DEBUG - Processing node 69c3d7e5bbc022e7926453b4: baseline-arm-clabbe 2026-03-27 12:41:30,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d7e5bbc022e7926453b4 2026-03-27 12:41:30,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:41:30,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d7e5bbc022e7926453b4 HTTP/1.1" 200 None 2026-03-27 12:41:30,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:41:30,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 12:41:30,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:41:30,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d7e5bbc022e7926453b4 HTTP/1.1" 200 None 2026-03-27 12:41:30,901 - lava_callback - ERROR - Error processing callback for node 69c3d7e5bbc022e7926453b4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 12:41:34,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:41:34,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48374 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:42:10,763 - lava_callback - INFO - Received callback for node 69c3d80ebbc022e7926453c2 INFO: 172.20.0.1:38670 - "POST /node/69c3d80ebbc022e7926453c2 HTTP/1.0" 202 Accepted 2026-03-27 12:42:10,768 - lava_callback - INFO - Starting async job submit for node 69c3d80ebbc022e7926453c2 2026-03-27 12:42:10,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:42:10,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d80ebbc022e7926453c2 HTTP/1.1" 200 None 2026-03-27 12:42:10,784 - lava_callback - DEBUG - Processing node 69c3d80ebbc022e7926453c2: baseline-arm-clabbe 2026-03-27 12:42:10,784 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d80ebbc022e7926453c2 2026-03-27 12:42:10,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:42:10,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d80ebbc022e7926453c2 HTTP/1.1" 200 None 2026-03-27 12:42:10,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:42:10,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 12:42:10,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:42:10,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d80ebbc022e7926453c2 HTTP/1.1" 200 None 2026-03-27 12:42:10,884 - lava_callback - ERROR - Error processing callback for node 69c3d80ebbc022e7926453c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60868 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:42:50,828 - lava_callback - INFO - Received callback for node 69c3d836bbc022e7926453d0 INFO: 172.20.0.1:56636 - "POST /node/69c3d836bbc022e7926453d0 HTTP/1.0" 202 Accepted 2026-03-27 12:42:50,834 - lava_callback - INFO - Starting async job submit for node 69c3d836bbc022e7926453d0 2026-03-27 12:42:50,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:42:50,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d836bbc022e7926453d0 HTTP/1.1" 200 None 2026-03-27 12:42:50,851 - lava_callback - DEBUG - Processing node 69c3d836bbc022e7926453d0: baseline-arm-clabbe 2026-03-27 12:42:50,851 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d836bbc022e7926453d0 2026-03-27 12:42:50,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:42:50,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d836bbc022e7926453d0 HTTP/1.1" 200 None 2026-03-27 12:42:50,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:42:50,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 12:42:50,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:42:50,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d836bbc022e7926453d0 HTTP/1.1" 200 None 2026-03-27 12:42:50,913 - lava_callback - ERROR - Error processing callback for node 69c3d836bbc022e7926453d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52186 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:43:30,786 - lava_callback - INFO - Received callback for node 69c3d85ebbc022e7926453dd INFO: 172.20.0.1:37914 - "POST /node/69c3d85ebbc022e7926453dd HTTP/1.0" 202 Accepted 2026-03-27 12:43:30,792 - lava_callback - INFO - Starting async job submit for node 69c3d85ebbc022e7926453dd 2026-03-27 12:43:30,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:43:30,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d85ebbc022e7926453dd HTTP/1.1" 200 None 2026-03-27 12:43:30,814 - lava_callback - DEBUG - Processing node 69c3d85ebbc022e7926453dd: baseline-arm-clabbe 2026-03-27 12:43:30,814 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d85ebbc022e7926453dd 2026-03-27 12:43:30,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:43:30,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d85ebbc022e7926453dd HTTP/1.1" 200 None 2026-03-27 12:43:30,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:43:30,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 12:43:30,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:43:30,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d85ebbc022e7926453dd HTTP/1.1" 200 None 2026-03-27 12:43:30,875 - lava_callback - ERROR - Error processing callback for node 69c3d85ebbc022e7926453dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56280 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53104 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:45:30,958 - lava_callback - INFO - Received callback for node 69c3d8d6bbc022e7926453f6 INFO: 172.20.0.1:44114 - "POST /node/69c3d8d6bbc022e7926453f6 HTTP/1.0" 202 Accepted 2026-03-27 12:45:30,964 - lava_callback - INFO - Starting async job submit for node 69c3d8d6bbc022e7926453f6 2026-03-27 12:45:30,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:45:30,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d8d6bbc022e7926453f6 HTTP/1.1" 200 None 2026-03-27 12:45:30,980 - lava_callback - DEBUG - Processing node 69c3d8d6bbc022e7926453f6: baseline-arm-clabbe 2026-03-27 12:45:30,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d8d6bbc022e7926453f6 2026-03-27 12:45:30,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:45:30,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d8d6bbc022e7926453f6 HTTP/1.1" 200 None 2026-03-27 12:45:30,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:45:31,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3293e18a3717aa65d01ad HTTP/1.1" 200 None 2026-03-27 12:45:31,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:45:31,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d8d6bbc022e7926453f6 HTTP/1.1" 200 None 2026-03-27 12:45:31,054 - lava_callback - ERROR - Error processing callback for node 69c3d8d6bbc022e7926453f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38934 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:45:46,630 - lava_callback - INFO - Received callback for node 69c3d8f7bbc022e792645403 INFO: 172.20.0.1:38942 - "POST /node/69c3d8f7bbc022e792645403 HTTP/1.0" 202 Accepted 2026-03-27 12:45:46,639 - lava_callback - INFO - Starting async job submit for node 69c3d8f7bbc022e792645403 2026-03-27 12:45:46,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:45:46,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d8f7bbc022e792645403 HTTP/1.1" 200 None 2026-03-27 12:45:46,656 - lava_callback - DEBUG - Processing node 69c3d8f7bbc022e792645403: baseline-arm-baylibre 2026-03-27 12:45:46,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d8f7bbc022e792645403 2026-03-27 12:45:46,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:45:46,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d8f7bbc022e792645403 HTTP/1.1" 200 None 2026-03-27 12:45:46,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:45:46,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:45:46,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:45:46,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d8f7bbc022e792645403 HTTP/1.1" 200 None 2026-03-27 12:45:46,730 - lava_callback - ERROR - Error processing callback for node 69c3d8f7bbc022e792645403: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59600 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:46:26,500 - lava_callback - INFO - Received callback for node 69c3d91ebbc022e79264540e INFO: 172.20.0.1:59612 - "POST /node/69c3d91ebbc022e79264540e HTTP/1.0" 202 Accepted 2026-03-27 12:46:26,506 - lava_callback - INFO - Starting async job submit for node 69c3d91ebbc022e79264540e 2026-03-27 12:46:26,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:59628 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:46:26,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d91ebbc022e79264540e HTTP/1.1" 200 None 2026-03-27 12:46:26,523 - lava_callback - DEBUG - Processing node 69c3d91ebbc022e79264540e: baseline-arm-baylibre 2026-03-27 12:46:26,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d91ebbc022e79264540e 2026-03-27 12:46:26,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:46:26,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d91ebbc022e79264540e HTTP/1.1" 200 None 2026-03-27 12:46:26,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:46:26,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:46:26,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:46:26,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d91ebbc022e79264540e HTTP/1.1" 200 None 2026-03-27 12:46:26,585 - lava_callback - ERROR - Error processing callback for node 69c3d91ebbc022e79264540e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:47:06,608 - lava_callback - INFO - Received callback for node 69c3d945bbc022e79264541f INFO: 172.20.0.1:60900 - "POST /node/69c3d945bbc022e79264541f HTTP/1.0" 202 Accepted 2026-03-27 12:47:06,613 - lava_callback - INFO - Starting async job submit for node 69c3d945bbc022e79264541f 2026-03-27 12:47:06,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:06,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d945bbc022e79264541f HTTP/1.1" 200 None 2026-03-27 12:47:06,630 - lava_callback - DEBUG - Processing node 69c3d945bbc022e79264541f: baseline-arm-baylibre 2026-03-27 12:47:06,630 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d945bbc022e79264541f 2026-03-27 12:47:06,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:06,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d945bbc022e79264541f HTTP/1.1" 200 None 2026-03-27 12:47:06,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:06,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:47:06,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:06,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d945bbc022e79264541f HTTP/1.1" 200 None 2026-03-27 12:47:06,719 - lava_callback - ERROR - Error processing callback for node 69c3d945bbc022e79264541f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 12:47:12,114 - lava_callback - INFO - Received callback for node 69c5cff72cb62322ad965e10 INFO: 172.20.0.1:38654 - "POST /node/69c5cff72cb62322ad965e10 HTTP/1.0" 202 Accepted 2026-03-27 12:47:12,120 - lava_callback - INFO - Starting async job submit for node 69c5cff72cb62322ad965e10 2026-03-27 12:47:12,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:12,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cff72cb62322ad965e10 HTTP/1.1" 200 None 2026-03-27 12:47:12,151 - lava_callback - DEBUG - Processing node 69c5cff72cb62322ad965e10: baseline-x86 2026-03-27 12:47:12,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:47:12,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:47:12,609 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff72cb62322ad965e10/log.txt.gz 2026-03-27 12:47:12,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:47:13,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:47:13,054 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff72cb62322ad965e10/lava-logs.yaml 2026-03-27 12:47:13,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:47:13,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:47:13,393 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c5cff72cb62322ad965e10/lava_callback.json.gz 2026-03-27 12:47:13,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:13,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cff72cb62322ad965e10 HTTP/1.1" 200 None 2026-03-27 12:47:13,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:13,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc5a2cb62322ad965dce HTTP/1.1" 200 None 2026-03-27 12:47:13,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:13,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c5cff72cb62322ad965e10 HTTP/1.1" 200 None 2026-03-27 12:47:13,468 - lava_callback - INFO - Completed processing callback for node 69c5cff72cb62322ad965e10 INFO: 172.20.0.1:38670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49860 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:47:34,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:34,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44958 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:47:46,583 - lava_callback - INFO - Received callback for node 69c3d96cbbc022e792645431 INFO: 172.20.0.1:44966 - "POST /node/69c3d96cbbc022e792645431 HTTP/1.0" 202 Accepted 2026-03-27 12:47:46,589 - lava_callback - INFO - Starting async job submit for node 69c3d96cbbc022e792645431 2026-03-27 12:47:46,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:46,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d96cbbc022e792645431 HTTP/1.1" 200 None 2026-03-27 12:47:46,625 - lava_callback - DEBUG - Processing node 69c3d96cbbc022e792645431: baseline-arm-baylibre 2026-03-27 12:47:46,626 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d96cbbc022e792645431 2026-03-27 12:47:46,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:46,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d96cbbc022e792645431 HTTP/1.1" 200 None 2026-03-27 12:47:46,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:46,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:47:46,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:47:46,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d96cbbc022e792645431 HTTP/1.1" 200 None 2026-03-27 12:47:46,678 - lava_callback - ERROR - Error processing callback for node 69c3d96cbbc022e792645431: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60628 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:48:26,653 - lava_callback - INFO - Received callback for node 69c3d993bbc022e79264543a INFO: 172.20.0.1:60644 - "POST /node/69c3d993bbc022e79264543a HTTP/1.0" 202 Accepted 2026-03-27 12:48:26,660 - lava_callback - INFO - Starting async job submit for node 69c3d993bbc022e79264543a 2026-03-27 12:48:26,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:48:26,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d993bbc022e79264543a HTTP/1.1" 200 None 2026-03-27 12:48:26,677 - lava_callback - DEBUG - Processing node 69c3d993bbc022e79264543a: baseline-arm-baylibre 2026-03-27 12:48:26,677 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d993bbc022e79264543a 2026-03-27 12:48:26,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:48:26,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d993bbc022e79264543a HTTP/1.1" 200 None 2026-03-27 12:48:26,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:48:26,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:48:26,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:48:26,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d993bbc022e79264543a HTTP/1.1" 200 None 2026-03-27 12:48:26,738 - lava_callback - ERROR - Error processing callback for node 69c3d993bbc022e79264543a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40276 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:48:57,298 - lava_callback - INFO - Received callback for node 69c64966b98d4d3c0206894c INFO: 172.20.0.1:40288 - "POST /node/69c64966b98d4d3c0206894c HTTP/1.0" 202 Accepted 2026-03-27 12:48:57,304 - lava_callback - INFO - Starting async job submit for node 69c64966b98d4d3c0206894c 2026-03-27 12:48:57,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:48:57,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64966b98d4d3c0206894c HTTP/1.1" 200 None 2026-03-27 12:48:57,363 - lava_callback - DEBUG - Processing node 69c64966b98d4d3c0206894c: baseline-arm64 2026-03-27 12:48:57,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:48:57,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:48:57,886 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64966b98d4d3c0206894c/log.txt.gz 2026-03-27 12:48:58,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:48:58,372 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:48:58,374 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64966b98d4d3c0206894c/lava-logs.yaml 2026-03-27 12:48:58,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:48:58,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:48:58,571 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c64966b98d4d3c0206894c/lava_callback.json.gz 2026-03-27 12:48:58,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:48:58,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64966b98d4d3c0206894c HTTP/1.1" 200 None 2026-03-27 12:48:58,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:48:58,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 12:48:58,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:48:58,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64966b98d4d3c0206894c HTTP/1.1" 200 None 2026-03-27 12:48:58,655 - lava_callback - INFO - Completed processing callback for node 69c64966b98d4d3c0206894c 2026-03-27 12:49:04,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:49:04,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:36866 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:49:06,632 - lava_callback - INFO - Received callback for node 69c3d9babbc022e792645448 INFO: 172.20.0.1:36878 - "POST /node/69c3d9babbc022e792645448 HTTP/1.0" 202 Accepted 2026-03-27 12:49:06,637 - lava_callback - INFO - Starting async job submit for node 69c3d9babbc022e792645448 2026-03-27 12:49:06,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:49:06,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d9babbc022e792645448 HTTP/1.1" 200 None 2026-03-27 12:49:06,654 - lava_callback - DEBUG - Processing node 69c3d9babbc022e792645448: baseline-arm-baylibre 2026-03-27 12:49:06,654 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d9babbc022e792645448 2026-03-27 12:49:06,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:49:06,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d9babbc022e792645448 HTTP/1.1" 200 None 2026-03-27 12:49:06,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:49:06,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:49:06,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:49:06,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d9babbc022e792645448 HTTP/1.1" 200 None 2026-03-27 12:49:06,719 - lava_callback - ERROR - Error processing callback for node 69c3d9babbc022e792645448: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39350 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:49:46,701 - lava_callback - INFO - Received callback for node 69c3d9e4bbc022e792645459 INFO: 172.20.0.1:39358 - "POST /node/69c3d9e4bbc022e792645459 HTTP/1.0" 202 Accepted 2026-03-27 12:49:46,707 - lava_callback - INFO - Starting async job submit for node 69c3d9e4bbc022e792645459 2026-03-27 12:49:46,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:49:46,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d9e4bbc022e792645459 HTTP/1.1" 200 None 2026-03-27 12:49:46,729 - lava_callback - DEBUG - Processing node 69c3d9e4bbc022e792645459: baseline-arm-baylibre 2026-03-27 12:49:46,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3d9e4bbc022e792645459 2026-03-27 12:49:46,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:49:46,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3d9e4bbc022e792645459 HTTP/1.1" 200 None 2026-03-27 12:49:46,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:49:46,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:49:46,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:49:46,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3d9e4bbc022e792645459 HTTP/1.1" 200 None 2026-03-27 12:49:46,788 - lava_callback - ERROR - Error processing callback for node 69c3d9e4bbc022e792645459: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40002 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:50:26,728 - lava_callback - INFO - Received callback for node 69c3da09bbc022e792645462 INFO: 172.20.0.1:40016 - "POST /node/69c3da09bbc022e792645462 HTTP/1.0" 202 Accepted 2026-03-27 12:50:26,734 - lava_callback - INFO - Starting async job submit for node 69c3da09bbc022e792645462 2026-03-27 12:50:26,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:50:26,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3da09bbc022e792645462 HTTP/1.1" 200 None 2026-03-27 12:50:26,751 - lava_callback - DEBUG - Processing node 69c3da09bbc022e792645462: baseline-arm-baylibre 2026-03-27 12:50:26,751 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3da09bbc022e792645462 2026-03-27 12:50:26,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:50:26,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3da09bbc022e792645462 HTTP/1.1" 200 None 2026-03-27 12:50:26,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:50:26,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:50:26,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:50:26,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3da09bbc022e792645462 HTTP/1.1" 200 None 2026-03-27 12:50:26,807 - lava_callback - ERROR - Error processing callback for node 69c3da09bbc022e792645462: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53772 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:50:38,639 - lava_callback - INFO - Received callback for node 69c66b30b98d4d3c02069244 INFO: 172.20.0.1:53786 - "POST /node/69c66b30b98d4d3c02069244 HTTP/1.0" 202 Accepted 2026-03-27 12:50:38,645 - lava_callback - INFO - Starting async job submit for node 69c66b30b98d4d3c02069244 2026-03-27 12:50:38,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:50:38,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c66b30b98d4d3c02069244 HTTP/1.1" 200 None 2026-03-27 12:50:38,710 - lava_callback - DEBUG - Processing node 69c66b30b98d4d3c02069244: baseline-nfs-arm64 2026-03-27 12:50:39,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:50:39,519 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:50:39,521 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c66b30b98d4d3c02069244/log.txt.gz 2026-03-27 12:50:39,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:50:40,017 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:50:40,019 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c66b30b98d4d3c02069244/lava-logs.yaml 2026-03-27 12:50:40,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:50:40,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:50:40,207 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c66b30b98d4d3c02069244/lava_callback.json.gz 2026-03-27 12:50:40,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:50:40,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c66b30b98d4d3c02069244 HTTP/1.1" 200 None 2026-03-27 12:50:40,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:50:40,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 12:50:40,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:50:40,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c66b30b98d4d3c02069244 HTTP/1.1" 200 None 2026-03-27 12:50:40,370 - lava_callback - INFO - Completed processing callback for node 69c66b30b98d4d3c02069244 INFO: 172.20.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41820 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:51:04,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:51:04,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54948 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:51:26,937 - lava_callback - INFO - Received callback for node 69c3da3bbbc022e79264546d INFO: 172.20.0.1:54952 - "POST /node/69c3da3bbbc022e79264546d HTTP/1.0" 202 Accepted 2026-03-27 12:51:26,942 - lava_callback - INFO - Starting async job submit for node 69c3da3bbbc022e79264546d 2026-03-27 12:51:26,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:51:26,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3da3bbbc022e79264546d HTTP/1.1" 200 None 2026-03-27 12:51:26,965 - lava_callback - DEBUG - Processing node 69c3da3bbbc022e79264546d: baseline-arm-baylibre 2026-03-27 12:51:26,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3da3bbbc022e79264546d 2026-03-27 12:51:26,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:51:26,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3da3bbbc022e79264546d HTTP/1.1" 200 None 2026-03-27 12:51:26,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:51:26,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:51:26,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:51:27,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3da3bbbc022e79264546d HTTP/1.1" 200 None 2026-03-27 12:51:27,021 - lava_callback - ERROR - Error processing callback for node 69c3da3bbbc022e79264546d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59162 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:51:46,803 - lava_callback - INFO - Received callback for node 69c3da5fbbc022e79264547b INFO: 172.20.0.1:59164 - "POST /node/69c3da5fbbc022e79264547b HTTP/1.0" 202 Accepted 2026-03-27 12:51:46,808 - lava_callback - INFO - Starting async job submit for node 69c3da5fbbc022e79264547b 2026-03-27 12:51:46,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:51:46,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3da5fbbc022e79264547b HTTP/1.1" 200 None 2026-03-27 12:51:46,824 - lava_callback - DEBUG - Processing node 69c3da5fbbc022e79264547b: baseline-arm-baylibre 2026-03-27 12:51:46,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3da5fbbc022e79264547b 2026-03-27 12:51:46,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:51:46,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3da5fbbc022e79264547b HTTP/1.1" 200 None 2026-03-27 12:51:46,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:51:46,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:51:46,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:51:46,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3da5fbbc022e79264547b HTTP/1.1" 200 None 2026-03-27 12:51:46,877 - lava_callback - ERROR - Error processing callback for node 69c3da5fbbc022e79264547b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56158 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:52:05,065 - lava_callback - INFO - Received callback for node 69c624ca2cb62322ad96981d INFO: 172.20.0.1:55872 - "POST /node/69c624ca2cb62322ad96981d HTTP/1.0" 202 Accepted 2026-03-27 12:52:05,071 - lava_callback - INFO - Starting async job submit for node 69c624ca2cb62322ad96981d 2026-03-27 12:52:05,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:05,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c624ca2cb62322ad96981d HTTP/1.1" 200 None 2026-03-27 12:52:05,133 - lava_callback - DEBUG - Processing node 69c624ca2cb62322ad96981d: baseline-x86 2026-03-27 12:52:05,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:52:05,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:52:05,579 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c624ca2cb62322ad96981d/log.txt.gz 2026-03-27 12:52:05,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:52:05,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:52:06,001 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c624ca2cb62322ad96981d/lava-logs.yaml 2026-03-27 12:52:06,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:52:06,153 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:52:06,154 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c624ca2cb62322ad96981d/lava_callback.json.gz 2026-03-27 12:52:06,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:06,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c624ca2cb62322ad96981d HTTP/1.1" 200 None 2026-03-27 12:52:06,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:06,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc5a2cb62322ad965dce HTTP/1.1" 200 None 2026-03-27 12:52:06,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:06,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c624ca2cb62322ad96981d HTTP/1.1" 200 None 2026-03-27 12:52:06,242 - lava_callback - INFO - Completed processing callback for node 69c624ca2cb62322ad96981d INFO: 172.20.0.1:55880 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:52:10,466 - lava_callback - INFO - Received callback for node 69c6772bb98d4d3c020692b6 INFO: 172.20.0.1:53966 - "POST /node/69c6772bb98d4d3c020692b6 HTTP/1.0" 202 Accepted 2026-03-27 12:52:10,476 - lava_callback - INFO - Starting async job submit for node 69c6772bb98d4d3c020692b6 2026-03-27 12:52:10,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:10,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6772bb98d4d3c020692b6 HTTP/1.1" 200 None 2026-03-27 12:52:10,537 - lava_callback - DEBUG - Processing node 69c6772bb98d4d3c020692b6: baseline-arm64 2026-03-27 12:52:11,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:52:11,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:52:11,169 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6772bb98d4d3c020692b6/log.txt.gz 2026-03-27 12:52:11,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:52:11,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:52:11,645 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6772bb98d4d3c020692b6/lava-logs.yaml 2026-03-27 12:52:11,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:52:11,808 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:52:11,810 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c6772bb98d4d3c020692b6/lava_callback.json.gz 2026-03-27 12:52:11,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:11,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6772bb98d4d3c020692b6 HTTP/1.1" 200 None 2026-03-27 12:52:11,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:11,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 12:52:11,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:11,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6772bb98d4d3c020692b6 HTTP/1.1" 200 None 2026-03-27 12:52:11,971 - lava_callback - INFO - Completed processing callback for node 69c6772bb98d4d3c020692b6 INFO: 172.20.0.1:53982 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48110 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:52:26,854 - lava_callback - INFO - Received callback for node 69c3da88bbc022e792645485 INFO: 172.20.0.1:48118 - "POST /node/69c3da88bbc022e792645485 HTTP/1.0" 202 Accepted 2026-03-27 12:52:26,860 - lava_callback - INFO - Starting async job submit for node 69c3da88bbc022e792645485 2026-03-27 12:52:26,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:26,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3da88bbc022e792645485 HTTP/1.1" 200 None 2026-03-27 12:52:26,876 - lava_callback - DEBUG - Processing node 69c3da88bbc022e792645485: baseline-arm-baylibre 2026-03-27 12:52:26,876 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3da88bbc022e792645485 2026-03-27 12:52:26,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:26,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3da88bbc022e792645485 HTTP/1.1" 200 None 2026-03-27 12:52:26,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:26,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:52:26,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:26,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3da88bbc022e792645485 HTTP/1.1" 200 None 2026-03-27 12:52:26,928 - lava_callback - ERROR - Error processing callback for node 69c3da88bbc022e792645485: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-27 12:52:34,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:52:34,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35960 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:53:01,384 - lava_callback - INFO - Received callback for node 69c640cbb98d4d3c0206848a INFO: 172.20.0.1:34754 - "POST /node/69c640cbb98d4d3c0206848a HTTP/1.0" 202 Accepted 2026-03-27 12:53:01,390 - lava_callback - INFO - Starting async job submit for node 69c640cbb98d4d3c0206848a 2026-03-27 12:53:01,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:01,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c640cbb98d4d3c0206848a HTTP/1.1" 200 None 2026-03-27 12:53:01,423 - lava_callback - DEBUG - Processing node 69c640cbb98d4d3c0206848a: baseline-x86 2026-03-27 12:53:01,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:53:01,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:53:01,854 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c640cbb98d4d3c0206848a/log.txt.gz 2026-03-27 12:53:02,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:53:02,213 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:53:02,215 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c640cbb98d4d3c0206848a/lava-logs.yaml 2026-03-27 12:53:02,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:53:02,387 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:53:02,389 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c640cbb98d4d3c0206848a/lava_callback.json.gz 2026-03-27 12:53:02,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:02,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c640cbb98d4d3c0206848a HTTP/1.1" 200 None 2026-03-27 12:53:02,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:02,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63d02b98d4d3c02067dea HTTP/1.1" 200 None 2026-03-27 12:53:02,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:02,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c640cbb98d4d3c0206848a HTTP/1.1" 200 None 2026-03-27 12:53:02,468 - lava_callback - INFO - Completed processing callback for node 69c640cbb98d4d3c0206848a 2026-03-27 12:53:04,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:04,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34764 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:53:11,261 - lava_callback - INFO - Received callback for node 69c3dab2bbc022e79264548f INFO: 172.20.0.1:45762 - "POST /node/69c3dab2bbc022e79264548f HTTP/1.0" 202 Accepted 2026-03-27 12:53:11,266 - lava_callback - INFO - Starting async job submit for node 69c3dab2bbc022e79264548f 2026-03-27 12:53:11,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:11,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dab2bbc022e79264548f HTTP/1.1" 200 None 2026-03-27 12:53:11,286 - lava_callback - DEBUG - Processing node 69c3dab2bbc022e79264548f: baseline-arm-clabbe 2026-03-27 12:53:11,287 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dab2bbc022e79264548f 2026-03-27 12:53:11,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:11,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dab2bbc022e79264548f HTTP/1.1" 200 None 2026-03-27 12:53:11,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:11,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:53:11,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:11,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dab2bbc022e79264548f HTTP/1.1" 200 None 2026-03-27 12:53:11,350 - lava_callback - ERROR - Error processing callback for node 69c3dab2bbc022e79264548f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54166 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:53:46,968 - lava_callback - INFO - Received callback for node 69c3dad8bbc022e7926454a1 INFO: 172.20.0.1:54182 - "POST /node/69c3dad8bbc022e7926454a1 HTTP/1.0" 202 Accepted 2026-03-27 12:53:46,974 - lava_callback - INFO - Starting async job submit for node 69c3dad8bbc022e7926454a1 2026-03-27 12:53:46,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:46,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dad8bbc022e7926454a1 HTTP/1.1" 200 None 2026-03-27 12:53:46,990 - lava_callback - DEBUG - Processing node 69c3dad8bbc022e7926454a1: baseline-arm-baylibre 2026-03-27 12:53:46,990 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dad8bbc022e7926454a1 2026-03-27 12:53:46,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:47,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dad8bbc022e7926454a1 HTTP/1.1" 200 None 2026-03-27 12:53:47,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:47,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:53:47,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:53:47,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dad8bbc022e7926454a1 HTTP/1.1" 200 None 2026-03-27 12:53:47,044 - lava_callback - ERROR - Error processing callback for node 69c3dad8bbc022e7926454a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55646 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:54:31,302 - lava_callback - INFO - Received callback for node 69c3db01bbc022e7926454ab INFO: 172.20.0.1:43700 - "POST /node/69c3db01bbc022e7926454ab HTTP/1.0" 202 Accepted 2026-03-27 12:54:31,308 - lava_callback - INFO - Starting async job submit for node 69c3db01bbc022e7926454ab 2026-03-27 12:54:31,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:54:31,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3db01bbc022e7926454ab HTTP/1.1" 200 None 2026-03-27 12:54:31,325 - lava_callback - DEBUG - Processing node 69c3db01bbc022e7926454ab: baseline-arm-clabbe 2026-03-27 12:54:31,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3db01bbc022e7926454ab 2026-03-27 12:54:31,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:54:31,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3db01bbc022e7926454ab HTTP/1.1" 200 None 2026-03-27 12:54:31,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:54:31,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:54:31,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:54:31,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3db01bbc022e7926454ab HTTP/1.1" 200 None 2026-03-27 12:54:31,377 - lava_callback - ERROR - Error processing callback for node 69c3db01bbc022e7926454ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43712 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:54:43,596 - lava_callback - INFO - Received callback for node 69c6797db98d4d3c020692f4 INFO: 172.20.0.1:52958 - "POST /node/69c6797db98d4d3c020692f4 HTTP/1.0" 202 Accepted 2026-03-27 12:54:43,603 - lava_callback - INFO - Starting async job submit for node 69c6797db98d4d3c020692f4 2026-03-27 12:54:43,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:54:43,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6797db98d4d3c020692f4 HTTP/1.1" 200 None 2026-03-27 12:54:43,637 - lava_callback - DEBUG - Processing node 69c6797db98d4d3c020692f4: baseline-x86 2026-03-27 12:54:43,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:54:44,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:54:44,122 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c6797db98d4d3c020692f4/log.txt.gz 2026-03-27 12:54:44,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:54:44,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:54:44,526 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c6797db98d4d3c020692f4/lava-logs.yaml 2026-03-27 12:54:44,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:54:44,671 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:54:44,672 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c6797db98d4d3c020692f4/lava_callback.json.gz 2026-03-27 12:54:44,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:54:44,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6797db98d4d3c020692f4 HTTP/1.1" 200 None 2026-03-27 12:54:44,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:54:44,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc5a2cb62322ad965dce HTTP/1.1" 200 None 2026-03-27 12:54:44,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:54:44,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6797db98d4d3c020692f4 HTTP/1.1" 200 None 2026-03-27 12:54:44,752 - lava_callback - INFO - Completed processing callback for node 69c6797db98d4d3c020692f4 INFO: 172.20.0.1:52966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56548 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:55:04,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:04,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38208 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:55:08,263 - lava_callback - INFO - Received callback for node 69c677b9b98d4d3c020692c2 INFO: 172.20.0.1:38216 - "POST /node/69c677b9b98d4d3c020692c2 HTTP/1.0" 202 Accepted 2026-03-27 12:55:08,270 - lava_callback - INFO - Starting async job submit for node 69c677b9b98d4d3c020692c2 2026-03-27 12:55:08,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:08,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c677b9b98d4d3c020692c2 HTTP/1.1" 200 None 2026-03-27 12:55:08,315 - lava_callback - DEBUG - Processing node 69c677b9b98d4d3c020692c2: baseline-nfs-arm64 2026-03-27 12:55:08,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:55:09,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:55:09,160 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c677b9b98d4d3c020692c2/log.txt.gz 2026-03-27 12:55:09,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:55:09,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:55:09,654 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c677b9b98d4d3c020692c2/lava-logs.yaml 2026-03-27 12:55:09,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 12:55:09,826 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 12:55:09,828 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c677b9b98d4d3c020692c2/lava_callback.json.gz 2026-03-27 12:55:09,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:09,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c677b9b98d4d3c020692c2 HTTP/1.1" 200 None 2026-03-27 12:55:09,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:09,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 12:55:09,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:09,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c677b9b98d4d3c020692c2 HTTP/1.1" 200 None 2026-03-27 12:55:09,968 - lava_callback - INFO - Completed processing callback for node 69c677b9b98d4d3c020692c2 2026-03-27 12:55:11,220 - lava_callback - INFO - Received callback for node 69c3db28bbc022e7926454b6 INFO: 172.20.0.1:54728 - "POST /node/69c3db28bbc022e7926454b6 HTTP/1.0" 202 Accepted 2026-03-27 12:55:11,226 - lava_callback - INFO - Starting async job submit for node 69c3db28bbc022e7926454b6 2026-03-27 12:55:11,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:11,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3db28bbc022e7926454b6 HTTP/1.1" 200 None 2026-03-27 12:55:11,244 - lava_callback - DEBUG - Processing node 69c3db28bbc022e7926454b6: baseline-arm-clabbe 2026-03-27 12:55:11,245 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3db28bbc022e7926454b6 2026-03-27 12:55:11,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:11,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3db28bbc022e7926454b6 HTTP/1.1" 200 None 2026-03-27 12:55:11,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:11,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:55:11,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:11,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3db28bbc022e7926454b6 HTTP/1.1" 200 None 2026-03-27 12:55:11,303 - lava_callback - ERROR - Error processing callback for node 69c3db28bbc022e7926454b6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55148 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:55:34,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:34,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36820 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:55:52,350 - lava_callback - INFO - Received callback for node 69c3db50bbc022e7926454c6 INFO: 172.20.0.1:47116 - "POST /node/69c3db50bbc022e7926454c6 HTTP/1.0" 202 Accepted 2026-03-27 12:55:52,356 - lava_callback - INFO - Starting async job submit for node 69c3db50bbc022e7926454c6 2026-03-27 12:55:52,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:52,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3db50bbc022e7926454c6 HTTP/1.1" 200 None 2026-03-27 12:55:52,382 - lava_callback - DEBUG - Processing node 69c3db50bbc022e7926454c6: baseline-arm-clabbe 2026-03-27 12:55:52,382 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3db50bbc022e7926454c6 2026-03-27 12:55:52,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:52,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3db50bbc022e7926454c6 HTTP/1.1" 200 None 2026-03-27 12:55:52,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:52,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:55:52,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:55:52,440 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3db50bbc022e7926454c6 HTTP/1.1" 200 None 2026-03-27 12:55:52,446 - lava_callback - ERROR - Error processing callback for node 69c3db50bbc022e7926454c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57542 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:56:31,321 - lava_callback - INFO - Received callback for node 69c3db78bbc022e7926454d1 INFO: 172.20.0.1:43714 - "POST /node/69c3db78bbc022e7926454d1 HTTP/1.0" 202 Accepted 2026-03-27 12:56:31,327 - lava_callback - INFO - Starting async job submit for node 69c3db78bbc022e7926454d1 2026-03-27 12:56:31,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:56:31,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3db78bbc022e7926454d1 HTTP/1.1" 200 None 2026-03-27 12:56:31,344 - lava_callback - DEBUG - Processing node 69c3db78bbc022e7926454d1: baseline-arm-clabbe 2026-03-27 12:56:31,344 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3db78bbc022e7926454d1 2026-03-27 12:56:31,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:56:31,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3db78bbc022e7926454d1 HTTP/1.1" 200 None 2026-03-27 12:56:31,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:56:31,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:56:31,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:56:31,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3db78bbc022e7926454d1 HTTP/1.1" 200 None 2026-03-27 12:56:31,402 - lava_callback - ERROR - Error processing callback for node 69c3db78bbc022e7926454d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41096 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:57:11,361 - lava_callback - INFO - Received callback for node 69c3dba1bbc022e7926454dc INFO: 172.20.0.1:34290 - "POST /node/69c3dba1bbc022e7926454dc HTTP/1.0" 202 Accepted 2026-03-27 12:57:11,366 - lava_callback - INFO - Starting async job submit for node 69c3dba1bbc022e7926454dc 2026-03-27 12:57:11,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:57:11,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dba1bbc022e7926454dc HTTP/1.1" 200 None 2026-03-27 12:57:11,386 - lava_callback - DEBUG - Processing node 69c3dba1bbc022e7926454dc: baseline-arm-clabbe 2026-03-27 12:57:11,386 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dba1bbc022e7926454dc 2026-03-27 12:57:11,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:57:11,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dba1bbc022e7926454dc HTTP/1.1" 200 None 2026-03-27 12:57:11,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:57:11,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:57:11,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:57:11,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dba1bbc022e7926454dc HTTP/1.1" 200 None 2026-03-27 12:57:11,441 - lava_callback - ERROR - Error processing callback for node 69c3dba1bbc022e7926454dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38256 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:57:51,382 - lava_callback - INFO - Received callback for node 69c3dbc8bbc022e7926454ed INFO: 172.20.0.1:40304 - "POST /node/69c3dbc8bbc022e7926454ed HTTP/1.0" 202 Accepted 2026-03-27 12:57:51,387 - lava_callback - INFO - Starting async job submit for node 69c3dbc8bbc022e7926454ed 2026-03-27 12:57:51,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:57:51,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dbc8bbc022e7926454ed HTTP/1.1" 200 None 2026-03-27 12:57:51,403 - lava_callback - DEBUG - Processing node 69c3dbc8bbc022e7926454ed: baseline-arm-clabbe 2026-03-27 12:57:51,403 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dbc8bbc022e7926454ed 2026-03-27 12:57:51,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:57:51,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dbc8bbc022e7926454ed HTTP/1.1" 200 None 2026-03-27 12:57:51,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:57:51,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:57:51,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:57:51,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dbc8bbc022e7926454ed HTTP/1.1" 200 None 2026-03-27 12:57:51,475 - lava_callback - ERROR - Error processing callback for node 69c3dbc8bbc022e7926454ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45602 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:58:31,453 - lava_callback - INFO - Received callback for node 69c3dbf0bbc022e7926454f6 INFO: 172.20.0.1:33538 - "POST /node/69c3dbf0bbc022e7926454f6 HTTP/1.0" 202 Accepted 2026-03-27 12:58:31,459 - lava_callback - INFO - Starting async job submit for node 69c3dbf0bbc022e7926454f6 2026-03-27 12:58:31,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:58:31,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dbf0bbc022e7926454f6 HTTP/1.1" 200 None 2026-03-27 12:58:31,475 - lava_callback - DEBUG - Processing node 69c3dbf0bbc022e7926454f6: baseline-arm-clabbe 2026-03-27 12:58:31,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dbf0bbc022e7926454f6 2026-03-27 12:58:31,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:58:31,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dbf0bbc022e7926454f6 HTTP/1.1" 200 None 2026-03-27 12:58:31,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:58:31,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:58:31,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:58:31,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dbf0bbc022e7926454f6 HTTP/1.1" 200 None 2026-03-27 12:58:31,534 - lava_callback - ERROR - Error processing callback for node 69c3dbf0bbc022e7926454f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33322 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:59:11,439 - lava_callback - INFO - Received callback for node 69c3dc18bbc022e792645501 INFO: 172.20.0.1:46938 - "POST /node/69c3dc18bbc022e792645501 HTTP/1.0" 202 Accepted 2026-03-27 12:59:11,445 - lava_callback - INFO - Starting async job submit for node 69c3dc18bbc022e792645501 2026-03-27 12:59:11,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:59:11,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dc18bbc022e792645501 HTTP/1.1" 200 None 2026-03-27 12:59:11,470 - lava_callback - DEBUG - Processing node 69c3dc18bbc022e792645501: baseline-arm-clabbe 2026-03-27 12:59:11,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dc18bbc022e792645501 2026-03-27 12:59:11,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:59:11,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dc18bbc022e792645501 HTTP/1.1" 200 None 2026-03-27 12:59:11,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:59:11,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:59:11,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:59:11,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dc18bbc022e792645501 HTTP/1.1" 200 None 2026-03-27 12:59:11,536 - lava_callback - ERROR - Error processing callback for node 69c3dc18bbc022e792645501: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36632 - "GET / HTTP/1.0" 200 OK 2026-03-27 12:59:51,556 - lava_callback - INFO - Received callback for node 69c3dc3fbbc022e792645513 INFO: 172.20.0.1:54124 - "POST /node/69c3dc3fbbc022e792645513 HTTP/1.0" 202 Accepted 2026-03-27 12:59:51,562 - lava_callback - INFO - Starting async job submit for node 69c3dc3fbbc022e792645513 2026-03-27 12:59:51,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:59:51,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dc3fbbc022e792645513 HTTP/1.1" 200 None 2026-03-27 12:59:51,578 - lava_callback - DEBUG - Processing node 69c3dc3fbbc022e792645513: baseline-arm-clabbe 2026-03-27 12:59:51,578 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dc3fbbc022e792645513 2026-03-27 12:59:51,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:59:51,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dc3fbbc022e792645513 HTTP/1.1" 200 None 2026-03-27 12:59:51,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:59:51,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 12:59:51,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 12:59:51,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dc3fbbc022e792645513 HTTP/1.1" 200 None 2026-03-27 12:59:51,637 - lava_callback - ERROR - Error processing callback for node 69c3dc3fbbc022e792645513: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37242 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:00:31,550 - lava_callback - INFO - Received callback for node 69c3dc68bbc022e79264551b INFO: 172.20.0.1:35028 - "POST /node/69c3dc68bbc022e79264551b HTTP/1.0" 202 Accepted 2026-03-27 13:00:31,555 - lava_callback - INFO - Starting async job submit for node 69c3dc68bbc022e79264551b 2026-03-27 13:00:31,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:00:31,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dc68bbc022e79264551b HTTP/1.1" 200 None 2026-03-27 13:00:31,589 - lava_callback - DEBUG - Processing node 69c3dc68bbc022e79264551b: baseline-arm-clabbe 2026-03-27 13:00:31,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dc68bbc022e79264551b 2026-03-27 13:00:31,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:00:31,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dc68bbc022e79264551b HTTP/1.1" 200 None 2026-03-27 13:00:31,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:00:31,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 13:00:31,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:00:31,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dc68bbc022e79264551b HTTP/1.1" 200 None 2026-03-27 13:00:31,658 - lava_callback - ERROR - Error processing callback for node 69c3dc68bbc022e79264551b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43062 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:01:11,622 - lava_callback - INFO - Received callback for node 69c3dc8fbbc022e792645529 INFO: 172.20.0.1:55116 - "POST /node/69c3dc8fbbc022e792645529 HTTP/1.0" 202 Accepted 2026-03-27 13:01:11,628 - lava_callback - INFO - Starting async job submit for node 69c3dc8fbbc022e792645529 2026-03-27 13:01:11,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:01:11,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dc8fbbc022e792645529 HTTP/1.1" 200 None 2026-03-27 13:01:11,643 - lava_callback - DEBUG - Processing node 69c3dc8fbbc022e792645529: baseline-arm-clabbe 2026-03-27 13:01:11,643 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dc8fbbc022e792645529 2026-03-27 13:01:11,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:01:11,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dc8fbbc022e792645529 HTTP/1.1" 200 None 2026-03-27 13:01:11,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:01:11,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 13:01:11,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:01:11,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dc8fbbc022e792645529 HTTP/1.1" 200 None 2026-03-27 13:01:11,709 - lava_callback - ERROR - Error processing callback for node 69c3dc8fbbc022e792645529: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49552 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:01:51,569 - lava_callback - INFO - Received callback for node 69c3dcb7bbc022e792645537 INFO: 172.20.0.1:38986 - "POST /node/69c3dcb7bbc022e792645537 HTTP/1.0" 202 Accepted 2026-03-27 13:01:51,575 - lava_callback - INFO - Starting async job submit for node 69c3dcb7bbc022e792645537 2026-03-27 13:01:51,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:01:51,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dcb7bbc022e792645537 HTTP/1.1" 200 None 2026-03-27 13:01:51,593 - lava_callback - DEBUG - Processing node 69c3dcb7bbc022e792645537: baseline-arm-clabbe 2026-03-27 13:01:51,593 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dcb7bbc022e792645537 2026-03-27 13:01:51,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:01:51,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dcb7bbc022e792645537 HTTP/1.1" 200 None 2026-03-27 13:01:51,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:01:51,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 13:01:51,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:01:51,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dcb7bbc022e792645537 HTTP/1.1" 200 None 2026-03-27 13:01:51,645 - lava_callback - ERROR - Error processing callback for node 69c3dcb7bbc022e792645537: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36918 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:02:31,637 - lava_callback - INFO - Received callback for node 69c3dcdfbbc022e79264553e INFO: 172.20.0.1:43206 - "POST /node/69c3dcdfbbc022e79264553e HTTP/1.0" 202 Accepted 2026-03-27 13:02:31,642 - lava_callback - INFO - Starting async job submit for node 69c3dcdfbbc022e79264553e 2026-03-27 13:02:31,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:02:31,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dcdfbbc022e79264553e HTTP/1.1" 200 None 2026-03-27 13:02:31,659 - lava_callback - DEBUG - Processing node 69c3dcdfbbc022e79264553e: baseline-arm-clabbe 2026-03-27 13:02:31,659 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dcdfbbc022e79264553e 2026-03-27 13:02:31,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:02:31,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dcdfbbc022e79264553e HTTP/1.1" 200 None 2026-03-27 13:02:31,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:02:31,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 13:02:31,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:02:31,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dcdfbbc022e79264553e HTTP/1.1" 200 None 2026-03-27 13:02:31,720 - lava_callback - ERROR - Error processing callback for node 69c3dcdfbbc022e79264553e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40018 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:03:08,298 - lava_callback - INFO - Received callback for node 69c67807b98d4d3c020692cd INFO: 172.20.0.1:40022 - "POST /node/69c67807b98d4d3c020692cd HTTP/1.0" 202 Accepted 2026-03-27 13:03:08,305 - lava_callback - INFO - Starting async job submit for node 69c67807b98d4d3c020692cd 2026-03-27 13:03:08,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:08,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c67807b98d4d3c020692cd HTTP/1.1" 200 None 2026-03-27 13:03:08,345 - lava_callback - DEBUG - Processing node 69c67807b98d4d3c020692cd: baseline-arm64 2026-03-27 13:03:08,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:03:09,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:03:09,099 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c67807b98d4d3c020692cd/log.txt.gz 2026-03-27 13:03:09,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:03:09,628 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:03:09,630 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c67807b98d4d3c020692cd/lava-logs.yaml 2026-03-27 13:03:09,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:03:09,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:03:09,821 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c67807b98d4d3c020692cd/lava_callback.json.gz 2026-03-27 13:03:09,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:09,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c67807b98d4d3c020692cd HTTP/1.1" 200 None 2026-03-27 13:03:09,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:09,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 13:03:09,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:09,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c67807b98d4d3c020692cd HTTP/1.1" 200 None 2026-03-27 13:03:09,967 - lava_callback - INFO - Completed processing callback for node 69c67807b98d4d3c020692cd INFO: 172.20.0.1:43148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43164 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:03:21,000 - lava_callback - INFO - Received callback for node 69c64978b98d4d3c02068990 INFO: 172.20.0.1:56240 - "POST /node/69c64978b98d4d3c02068990 HTTP/1.0" 202 Accepted 2026-03-27 13:03:21,009 - lava_callback - INFO - Starting async job submit for node 69c64978b98d4d3c02068990 2026-03-27 13:03:21,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:21,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64978b98d4d3c02068990 HTTP/1.1" 200 None 2026-03-27 13:03:21,065 - lava_callback - DEBUG - Processing node 69c64978b98d4d3c02068990: baseline-nfs-arm64 2026-03-27 13:03:22,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:03:22,378 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:03:22,379 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64978b98d4d3c02068990/log.txt.gz 2026-03-27 13:03:23,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:03:23,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:03:23,318 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64978b98d4d3c02068990/lava-logs.yaml 2026-03-27 13:03:23,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:03:23,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:03:23,534 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c64978b98d4d3c02068990/lava_callback.json.gz 2026-03-27 13:03:23,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:23,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c64978b98d4d3c02068990 HTTP/1.1" 200 None 2026-03-27 13:03:23,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:23,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 13:03:23,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:23,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c64978b98d4d3c02068990 HTTP/1.1" 200 None 2026-03-27 13:03:23,639 - lava_callback - INFO - Completed processing callback for node 69c64978b98d4d3c02068990 INFO: 172.20.0.1:56248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56258 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:03:34,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:34,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49204 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:03:51,725 - lava_callback - INFO - Received callback for node 69c3dd25bbc022e792645551 INFO: 172.20.0.1:57810 - "POST /node/69c3dd25bbc022e792645551 HTTP/1.0" 202 Accepted 2026-03-27 13:03:51,731 - lava_callback - INFO - Starting async job submit for node 69c3dd25bbc022e792645551 2026-03-27 13:03:51,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:51,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dd25bbc022e792645551 HTTP/1.1" 200 None 2026-03-27 13:03:51,768 - lava_callback - DEBUG - Processing node 69c3dd25bbc022e792645551: baseline-arm-clabbe 2026-03-27 13:03:51,768 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3dd25bbc022e792645551 2026-03-27 13:03:51,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:51,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3dd25bbc022e792645551 HTTP/1.1" 200 None 2026-03-27 13:03:51,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:51,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086a5ce0c9bc43bccb62b HTTP/1.1" 200 None 2026-03-27 13:03:51,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:03:51,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3dd25bbc022e792645551 HTTP/1.1" 200 None 2026-03-27 13:03:51,829 - lava_callback - ERROR - Error processing callback for node 69c3dd25bbc022e792645551: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43392 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:04:22,656 - lava_callback - INFO - Received callback for node 69c6788bb98d4d3c020692d8 INFO: 172.20.0.1:44748 - "POST /node/69c6788bb98d4d3c020692d8 HTTP/1.0" 202 Accepted 2026-03-27 13:04:22,663 - lava_callback - INFO - Starting async job submit for node 69c6788bb98d4d3c020692d8 2026-03-27 13:04:22,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:04:22,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6788bb98d4d3c020692d8 HTTP/1.1" 200 None 2026-03-27 13:04:22,707 - lava_callback - DEBUG - Processing node 69c6788bb98d4d3c020692d8: baseline-nfs-arm64 2026-03-27 13:04:23,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:04:23,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:04:23,424 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6788bb98d4d3c020692d8/log.txt.gz 2026-03-27 13:04:23,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:04:23,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:04:23,943 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6788bb98d4d3c020692d8/lava-logs.yaml 2026-03-27 13:04:23,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:04:24,152 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:04:24,154 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6788bb98d4d3c020692d8/lava_callback.json.gz 2026-03-27 13:04:24,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:04:24,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6788bb98d4d3c020692d8 HTTP/1.1" 200 None 2026-03-27 13:04:24,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:04:24,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5b114abf66a8f4d9bcb37 HTTP/1.1" 200 None 2026-03-27 13:04:24,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:04:24,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6788bb98d4d3c020692d8 HTTP/1.1" 200 None 2026-03-27 13:04:24,306 - lava_callback - INFO - Completed processing callback for node 69c6788bb98d4d3c020692d8 INFO: 172.20.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44762 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:04:34,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:04:34,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57608 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:09:11,023 - lava_callback - INFO - Received callback for node 69c67a81b98d4d3c0206935a INFO: 172.20.0.1:45320 - "POST /node/69c67a81b98d4d3c0206935a HTTP/1.0" 202 Accepted 2026-03-27 13:09:11,030 - lava_callback - INFO - Starting async job submit for node 69c67a81b98d4d3c0206935a 2026-03-27 13:09:11,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:09:11,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c67a81b98d4d3c0206935a HTTP/1.1" 200 None 2026-03-27 13:09:11,242 - lava_callback - DEBUG - Processing node 69c67a81b98d4d3c0206935a: baseline-arm64 2026-03-27 13:09:11,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:09:11,918 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:09:11,920 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c67a81b98d4d3c0206935a/log.txt.gz 2026-03-27 13:09:12,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:09:12,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:09:12,414 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c67a81b98d4d3c0206935a/lava-logs.yaml 2026-03-27 13:09:12,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:09:12,583 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:09:12,585 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c67a81b98d4d3c0206935a/lava_callback.json.gz 2026-03-27 13:09:12,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:09:12,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c67a81b98d4d3c0206935a HTTP/1.1" 200 None 2026-03-27 13:09:12,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:09:12,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 13:09:12,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:09:12,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c67a81b98d4d3c0206935a HTTP/1.1" 200 None 2026-03-27 13:09:12,720 - lava_callback - INFO - Completed processing callback for node 69c67a81b98d4d3c0206935a INFO: 172.20.0.1:45326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50068 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:09:34,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:09:34,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42006 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:16:02,242 - lava_callback - INFO - Received callback for node 69c67b8bb98d4d3c020693c9 INFO: 172.20.0.1:55200 - "POST /node/69c67b8bb98d4d3c020693c9 HTTP/1.0" 202 Accepted 2026-03-27 13:16:02,249 - lava_callback - INFO - Starting async job submit for node 69c67b8bb98d4d3c020693c9 2026-03-27 13:16:02,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:16:02,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c67b8bb98d4d3c020693c9 HTTP/1.1" 200 None 2026-03-27 13:16:02,290 - lava_callback - DEBUG - Processing node 69c67b8bb98d4d3c020693c9: baseline-nfs-arm64 2026-03-27 13:16:02,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:16:03,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:16:03,004 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c67b8bb98d4d3c020693c9/log.txt.gz 2026-03-27 13:16:03,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:16:03,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:16:03,506 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c67b8bb98d4d3c020693c9/lava-logs.yaml 2026-03-27 13:16:03,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:16:03,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:16:03,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c67b8bb98d4d3c020693c9/lava_callback.json.gz 2026-03-27 13:16:03,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:16:03,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c67b8bb98d4d3c020693c9 HTTP/1.1" 200 None 2026-03-27 13:16:03,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:16:03,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-03-27 13:16:03,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:16:03,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c67b8bb98d4d3c020693c9 HTTP/1.1" 200 None 2026-03-27 13:16:03,832 - lava_callback - INFO - Completed processing callback for node 69c67b8bb98d4d3c020693c9 2026-03-27 13:16:04,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:16:04,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55216 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:16:10,519 - lava_callback - INFO - Received callback for node 69c67bfeb98d4d3c020693e8 INFO: 172.20.0.1:59198 - "POST /node/69c67bfeb98d4d3c020693e8 HTTP/1.0" 202 Accepted 2026-03-27 13:16:10,530 - lava_callback - INFO - Starting async job submit for node 69c67bfeb98d4d3c020693e8 2026-03-27 13:16:10,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:16:10,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c67bfeb98d4d3c020693e8 HTTP/1.1" 200 None 2026-03-27 13:16:10,573 - lava_callback - DEBUG - Processing node 69c67bfeb98d4d3c020693e8: baseline-arm64 2026-03-27 13:16:11,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:16:11,421 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:16:11,423 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c67bfeb98d4d3c020693e8/log.txt.gz 2026-03-27 13:16:11,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:16:12,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:16:12,056 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c67bfeb98d4d3c020693e8/lava-logs.yaml 2026-03-27 13:16:12,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:16:12,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:16:12,239 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c67bfeb98d4d3c020693e8/lava_callback.json.gz 2026-03-27 13:16:12,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:16:12,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c67bfeb98d4d3c020693e8 HTTP/1.1" 200 None 2026-03-27 13:16:12,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:16:12,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 13:16:12,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:16:12,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c67bfeb98d4d3c020693e8 HTTP/1.1" 200 None 2026-03-27 13:16:12,450 - lava_callback - INFO - Completed processing callback for node 69c67bfeb98d4d3c020693e8 INFO: 172.20.0.1:59208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46738 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:16:34,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:16:34,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44534 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49096 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59674 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:26:39,297 - lava_callback - INFO - Received callback for node 69c67f0db98d4d3c02069517 INFO: 172.20.0.1:59684 - "POST /node/69c67f0db98d4d3c02069517 HTTP/1.0" 202 Accepted 2026-03-27 13:26:39,304 - lava_callback - INFO - Starting async job submit for node 69c67f0db98d4d3c02069517 2026-03-27 13:26:39,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:26:39,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c67f0db98d4d3c02069517 HTTP/1.1" 200 None 2026-03-27 13:26:39,346 - lava_callback - DEBUG - Processing node 69c67f0db98d4d3c02069517: baseline-nfs-arm64 2026-03-27 13:26:39,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:26:39,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:26:39,865 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c67f0db98d4d3c02069517/log.txt.gz 2026-03-27 13:26:40,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:26:40,316 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:26:40,319 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c67f0db98d4d3c02069517/lava-logs.yaml 2026-03-27 13:26:40,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:26:40,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:26:40,510 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c67f0db98d4d3c02069517/lava_callback.json.gz 2026-03-27 13:26:40,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:26:40,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c67f0db98d4d3c02069517 HTTP/1.1" 200 None 2026-03-27 13:26:40,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:26:40,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 13:26:40,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:26:40,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c67f0db98d4d3c02069517 HTTP/1.1" 200 None 2026-03-27 13:26:40,680 - lava_callback - INFO - Completed processing callback for node 69c67f0db98d4d3c02069517 INFO: 172.20.0.1:52494 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:26:48,755 - lava_callback - INFO - Received callback for node 69c3e290bbc022e7926455af INFO: 172.20.0.1:52500 - "POST /node/69c3e290bbc022e7926455af HTTP/1.0" 202 Accepted 2026-03-27 13:26:48,760 - lava_callback - INFO - Starting async job submit for node 69c3e290bbc022e7926455af 2026-03-27 13:26:48,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:26:48,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e290bbc022e7926455af HTTP/1.1" 200 None 2026-03-27 13:26:48,777 - lava_callback - DEBUG - Processing node 69c3e290bbc022e7926455af: baseline-arm64-baylibre 2026-03-27 13:26:48,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e290bbc022e7926455af 2026-03-27 13:26:48,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:26:48,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e290bbc022e7926455af HTTP/1.1" 200 None 2026-03-27 13:26:48,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:26:48,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:26:48,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:26:48,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e290bbc022e7926455af HTTP/1.1" 200 None 2026-03-27 13:26:48,833 - lava_callback - ERROR - Error processing callback for node 69c3e290bbc022e7926455af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33510 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:27:04,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:27:04,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49184 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:27:51,364 - lava_callback - INFO - Received callback for node 69c68387b98d4d3c020695ba INFO: 172.20.0.1:49448 - "POST /node/69c68387b98d4d3c020695ba HTTP/1.0" 202 Accepted 2026-03-27 13:27:51,371 - lava_callback - INFO - Starting async job submit for node 69c68387b98d4d3c020695ba 2026-03-27 13:27:51,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:27:51,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c68387b98d4d3c020695ba HTTP/1.1" 200 None 2026-03-27 13:27:51,409 - lava_callback - DEBUG - Processing node 69c68387b98d4d3c020695ba: baseline-arm64 2026-03-27 13:27:51,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:27:52,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:27:52,062 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c68387b98d4d3c020695ba/log.txt.gz 2026-03-27 13:27:52,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:27:52,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:27:52,557 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c68387b98d4d3c020695ba/lava-logs.yaml 2026-03-27 13:27:52,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:27:52,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:27:52,737 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c68387b98d4d3c020695ba/lava_callback.json.gz 2026-03-27 13:27:52,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:27:52,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c68387b98d4d3c020695ba HTTP/1.1" 200 None 2026-03-27 13:27:52,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:27:52,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 13:27:52,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:27:52,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c68387b98d4d3c020695ba HTTP/1.1" 200 None 2026-03-27 13:27:52,884 - lava_callback - INFO - Completed processing callback for node 69c68387b98d4d3c020695ba INFO: 172.20.0.1:49464 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:28:04,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:28:04,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51248 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:28:49,007 - lava_callback - INFO - Received callback for node 69c3e308bbc022e7926455ba INFO: 172.20.0.1:51250 - "POST /node/69c3e308bbc022e7926455ba HTTP/1.0" 202 Accepted 2026-03-27 13:28:49,012 - lava_callback - INFO - Starting async job submit for node 69c3e308bbc022e7926455ba 2026-03-27 13:28:49,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:28:49,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e308bbc022e7926455ba HTTP/1.1" 200 None 2026-03-27 13:28:49,037 - lava_callback - DEBUG - Processing node 69c3e308bbc022e7926455ba: baseline-arm64-baylibre 2026-03-27 13:28:49,037 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e308bbc022e7926455ba 2026-03-27 13:28:49,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:28:49,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e308bbc022e7926455ba HTTP/1.1" 200 None 2026-03-27 13:28:49,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:28:49,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:28:49,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:28:49,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e308bbc022e7926455ba HTTP/1.1" 200 None 2026-03-27 13:28:49,113 - lava_callback - ERROR - Error processing callback for node 69c3e308bbc022e7926455ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38496 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:30:08,949 - lava_callback - INFO - Received callback for node 69c3e35dbbc022e7926455c2 INFO: 172.20.0.1:38508 - "POST /node/69c3e35dbbc022e7926455c2 HTTP/1.0" 202 Accepted 2026-03-27 13:30:08,955 - lava_callback - INFO - Starting async job submit for node 69c3e35dbbc022e7926455c2 2026-03-27 13:30:08,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:30:08,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e35dbbc022e7926455c2 HTTP/1.1" 200 None 2026-03-27 13:30:08,970 - lava_callback - DEBUG - Processing node 69c3e35dbbc022e7926455c2: baseline-arm64-baylibre 2026-03-27 13:30:08,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e35dbbc022e7926455c2 2026-03-27 13:30:08,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:30:08,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e35dbbc022e7926455c2 HTTP/1.1" 200 None 2026-03-27 13:30:08,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:30:09,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:30:09,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:30:09,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e35dbbc022e7926455c2 HTTP/1.1" 200 None 2026-03-27 13:30:09,031 - lava_callback - ERROR - Error processing callback for node 69c3e35dbbc022e7926455c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56786 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:31:29,075 - lava_callback - INFO - Received callback for node 69c3e3afbbc022e7926455d0 INFO: 172.20.0.1:56788 - "POST /node/69c3e3afbbc022e7926455d0 HTTP/1.0" 202 Accepted 2026-03-27 13:31:29,081 - lava_callback - INFO - Starting async job submit for node 69c3e3afbbc022e7926455d0 2026-03-27 13:31:29,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:31:29,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e3afbbc022e7926455d0 HTTP/1.1" 200 None 2026-03-27 13:31:29,097 - lava_callback - DEBUG - Processing node 69c3e3afbbc022e7926455d0: baseline-arm64-baylibre 2026-03-27 13:31:29,097 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e3afbbc022e7926455d0 2026-03-27 13:31:29,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:31:29,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e3afbbc022e7926455d0 HTTP/1.1" 200 None 2026-03-27 13:31:29,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:31:29,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:31:29,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:31:29,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e3afbbc022e7926455d0 HTTP/1.1" 200 None 2026-03-27 13:31:29,157 - lava_callback - ERROR - Error processing callback for node 69c3e3afbbc022e7926455d0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58714 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:33:09,097 - lava_callback - INFO - Received callback for node 69c3e404bbc022e7926455df INFO: 172.20.0.1:58722 - "POST /node/69c3e404bbc022e7926455df HTTP/1.0" 202 Accepted 2026-03-27 13:33:09,103 - lava_callback - INFO - Starting async job submit for node 69c3e404bbc022e7926455df 2026-03-27 13:33:09,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:33:09,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e404bbc022e7926455df HTTP/1.1" 200 None 2026-03-27 13:33:09,119 - lava_callback - DEBUG - Processing node 69c3e404bbc022e7926455df: baseline-arm64-baylibre 2026-03-27 13:33:09,119 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e404bbc022e7926455df 2026-03-27 13:33:09,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:33:09,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e404bbc022e7926455df HTTP/1.1" 200 None 2026-03-27 13:33:09,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:33:09,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:33:09,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:33:09,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e404bbc022e7926455df HTTP/1.1" 200 None 2026-03-27 13:33:09,176 - lava_callback - ERROR - Error processing callback for node 69c3e404bbc022e7926455df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59046 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49408 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:34:29,188 - lava_callback - INFO - Received callback for node 69c3e455bbc022e7926455ef INFO: 172.20.0.1:49422 - "POST /node/69c3e455bbc022e7926455ef HTTP/1.0" 202 Accepted 2026-03-27 13:34:29,194 - lava_callback - INFO - Starting async job submit for node 69c3e455bbc022e7926455ef 2026-03-27 13:34:29,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:34:29,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e455bbc022e7926455ef HTTP/1.1" 200 None 2026-03-27 13:34:29,211 - lava_callback - DEBUG - Processing node 69c3e455bbc022e7926455ef: baseline-arm64-baylibre 2026-03-27 13:34:29,211 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e455bbc022e7926455ef 2026-03-27 13:34:29,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:34:29,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e455bbc022e7926455ef HTTP/1.1" 200 None 2026-03-27 13:34:29,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:34:29,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:34:29,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:34:29,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e455bbc022e7926455ef HTTP/1.1" 200 None 2026-03-27 13:34:29,270 - lava_callback - ERROR - Error processing callback for node 69c3e455bbc022e7926455ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44134 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:36:13,214 - lava_callback - INFO - Received callback for node 69c3e4b7bbc022e7926455fe INFO: 172.20.0.1:60876 - "POST /node/69c3e4b7bbc022e7926455fe HTTP/1.0" 202 Accepted 2026-03-27 13:36:13,220 - lava_callback - INFO - Starting async job submit for node 69c3e4b7bbc022e7926455fe 2026-03-27 13:36:13,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:36:13,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e4b7bbc022e7926455fe HTTP/1.1" 200 None 2026-03-27 13:36:13,236 - lava_callback - DEBUG - Processing node 69c3e4b7bbc022e7926455fe: baseline-arm64-clabbe 2026-03-27 13:36:13,237 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e4b7bbc022e7926455fe 2026-03-27 13:36:13,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:36:13,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e4b7bbc022e7926455fe HTTP/1.1" 200 None 2026-03-27 13:36:13,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:36:13,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:36:13,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:36:13,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e4b7bbc022e7926455fe HTTP/1.1" 200 None 2026-03-27 13:36:13,289 - lava_callback - ERROR - Error processing callback for node 69c3e4b7bbc022e7926455fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58426 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:37:40,027 - lava_callback - INFO - Received callback for node 69c6859ab98d4d3c0206960d INFO: 172.20.0.1:34226 - "POST /node/69c6859ab98d4d3c0206960d HTTP/1.0" 202 Accepted 2026-03-27 13:37:40,033 - lava_callback - INFO - Starting async job submit for node 69c6859ab98d4d3c0206960d 2026-03-27 13:37:40,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:37:40,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6859ab98d4d3c0206960d HTTP/1.1" 200 None 2026-03-27 13:37:40,097 - lava_callback - DEBUG - Processing node 69c6859ab98d4d3c0206960d: baseline-nfs-arm64 2026-03-27 13:37:40,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:37:40,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:37:40,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6859ab98d4d3c0206960d/log.txt.gz 2026-03-27 13:37:41,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:37:41,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:37:41,432 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6859ab98d4d3c0206960d/lava-logs.yaml 2026-03-27 13:37:41,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-27 13:37:41,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-27 13:37:41,630 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c6859ab98d4d3c0206960d/lava_callback.json.gz 2026-03-27 13:37:41,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:37:41,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6859ab98d4d3c0206960d HTTP/1.1" 200 None 2026-03-27 13:37:41,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:37:41,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c63cf0b98d4d3c02067dd7 HTTP/1.1" 200 None 2026-03-27 13:37:41,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:37:41,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c6859ab98d4d3c0206960d HTTP/1.1" 200 None 2026-03-27 13:37:41,770 - lava_callback - INFO - Completed processing callback for node 69c6859ab98d4d3c0206960d INFO: 172.20.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48778 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:38:04,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:38:04,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:52962 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:38:13,139 - lava_callback - INFO - Received callback for node 69c3e52ebbc022e79264560d INFO: 172.20.0.1:36108 - "POST /node/69c3e52ebbc022e79264560d HTTP/1.0" 202 Accepted 2026-03-27 13:38:13,144 - lava_callback - INFO - Starting async job submit for node 69c3e52ebbc022e79264560d 2026-03-27 13:38:13,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:38:13,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e52ebbc022e79264560d HTTP/1.1" 200 None 2026-03-27 13:38:13,160 - lava_callback - DEBUG - Processing node 69c3e52ebbc022e79264560d: baseline-arm64-clabbe 2026-03-27 13:38:13,161 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e52ebbc022e79264560d 2026-03-27 13:38:13,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:38:13,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e52ebbc022e79264560d HTTP/1.1" 200 None 2026-03-27 13:38:13,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:38:13,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:38:13,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:38:13,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e52ebbc022e79264560d HTTP/1.1" 200 None 2026-03-27 13:38:13,232 - lava_callback - ERROR - Error processing callback for node 69c3e52ebbc022e79264560d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51560 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:39:33,338 - lava_callback - INFO - Received callback for node 69c3e580bbc022e792645620 INFO: 172.20.0.1:58952 - "POST /node/69c3e580bbc022e792645620 HTTP/1.0" 202 Accepted 2026-03-27 13:39:33,344 - lava_callback - INFO - Starting async job submit for node 69c3e580bbc022e792645620 2026-03-27 13:39:33,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:39:33,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e580bbc022e792645620 HTTP/1.1" 200 None 2026-03-27 13:39:33,373 - lava_callback - DEBUG - Processing node 69c3e580bbc022e792645620: baseline-arm64-clabbe 2026-03-27 13:39:33,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e580bbc022e792645620 2026-03-27 13:39:33,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:39:33,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e580bbc022e792645620 HTTP/1.1" 200 None 2026-03-27 13:39:33,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:39:33,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:39:33,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:39:33,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e580bbc022e792645620 HTTP/1.1" 200 None 2026-03-27 13:39:33,425 - lava_callback - ERROR - Error processing callback for node 69c3e580bbc022e792645620: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54402 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:40:53,340 - lava_callback - INFO - Received callback for node 69c3e5d2bbc022e792645628 INFO: 172.20.0.1:43542 - "POST /node/69c3e5d2bbc022e792645628 HTTP/1.0" 202 Accepted 2026-03-27 13:40:53,345 - lava_callback - INFO - Starting async job submit for node 69c3e5d2bbc022e792645628 2026-03-27 13:40:53,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:40:53,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e5d2bbc022e792645628 HTTP/1.1" 200 None 2026-03-27 13:40:53,361 - lava_callback - DEBUG - Processing node 69c3e5d2bbc022e792645628: baseline-arm64-clabbe 2026-03-27 13:40:53,362 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e5d2bbc022e792645628 2026-03-27 13:40:53,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:40:53,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e5d2bbc022e792645628 HTTP/1.1" 200 None 2026-03-27 13:40:53,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:40:53,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:40:53,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:40:53,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e5d2bbc022e792645628 HTTP/1.1" 200 None 2026-03-27 13:40:53,421 - lava_callback - ERROR - Error processing callback for node 69c3e5d2bbc022e792645628: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48928 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:42:13,385 - lava_callback - INFO - Received callback for node 69c3e626bbc022e792645639 INFO: 172.20.0.1:43386 - "POST /node/69c3e626bbc022e792645639 HTTP/1.0" 202 Accepted 2026-03-27 13:42:13,391 - lava_callback - INFO - Starting async job submit for node 69c3e626bbc022e792645639 2026-03-27 13:42:13,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:42:13,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e626bbc022e792645639 HTTP/1.1" 200 None 2026-03-27 13:42:13,408 - lava_callback - DEBUG - Processing node 69c3e626bbc022e792645639: baseline-arm64-clabbe 2026-03-27 13:42:13,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e626bbc022e792645639 2026-03-27 13:42:13,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:42:13,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e626bbc022e792645639 HTTP/1.1" 200 None 2026-03-27 13:42:13,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:42:13,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:42:13,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:42:13,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e626bbc022e792645639 HTTP/1.1" 200 None 2026-03-27 13:42:13,499 - lava_callback - ERROR - Error processing callback for node 69c3e626bbc022e792645639: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53972 - "GET / HTTP/1.0" 200 OK 2026-03-27 13:43:33,448 - lava_callback - INFO - Received callback for node 69c3e67bbbc022e792645649 INFO: 172.20.0.1:49852 - "POST /node/69c3e67bbbc022e792645649 HTTP/1.0" 202 Accepted 2026-03-27 13:43:33,453 - lava_callback - INFO - Starting async job submit for node 69c3e67bbbc022e792645649 2026-03-27 13:43:33,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:43:33,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e67bbbc022e792645649 HTTP/1.1" 200 None 2026-03-27 13:43:33,470 - lava_callback - DEBUG - Processing node 69c3e67bbbc022e792645649: baseline-arm64-clabbe 2026-03-27 13:43:33,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3e67bbbc022e792645649 2026-03-27 13:43:33,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:43:33,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3e67bbbc022e792645649 HTTP/1.1" 200 None 2026-03-27 13:43:33,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:43:33,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3295318a3717aa65d01c3 HTTP/1.1" 200 None 2026-03-27 13:43:33,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 13:43:33,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3e67bbbc022e792645649 HTTP/1.1" 200 None 2026-03-27 13:43:33,537 - lava_callback - ERROR - Error processing callback for node 69c3e67bbbc022e792645649: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40478 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48854 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58588 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:13:51,385 - lava_callback - INFO - Received callback for node 69c3ed9abbc022e792645691 INFO: 172.20.0.1:37916 - "POST /node/69c3ed9abbc022e792645691 HTTP/1.0" 202 Accepted 2026-03-27 14:13:51,391 - lava_callback - INFO - Starting async job submit for node 69c3ed9abbc022e792645691 2026-03-27 14:13:51,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:13:51,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ed9abbc022e792645691 HTTP/1.1" 200 None 2026-03-27 14:13:51,407 - lava_callback - DEBUG - Processing node 69c3ed9abbc022e792645691: baseline-arm64-baylibre 2026-03-27 14:13:51,407 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ed9abbc022e792645691 2026-03-27 14:13:51,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:13:51,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ed9abbc022e792645691 HTTP/1.1" 200 None 2026-03-27 14:13:51,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:13:51,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 14:13:51,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:13:51,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ed9abbc022e792645691 HTTP/1.1" 200 None 2026-03-27 14:13:51,481 - lava_callback - ERROR - Error processing callback for node 69c3ed9abbc022e792645691: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39346 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:15:51,396 - lava_callback - INFO - Received callback for node 69c3ee13bbc022e79264569d INFO: 172.20.0.1:43820 - "POST /node/69c3ee13bbc022e79264569d HTTP/1.0" 202 Accepted 2026-03-27 14:15:51,402 - lava_callback - INFO - Starting async job submit for node 69c3ee13bbc022e79264569d 2026-03-27 14:15:51,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:15:51,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ee13bbc022e79264569d HTTP/1.1" 200 None 2026-03-27 14:15:51,419 - lava_callback - DEBUG - Processing node 69c3ee13bbc022e79264569d: baseline-arm64-baylibre 2026-03-27 14:15:51,419 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ee13bbc022e79264569d 2026-03-27 14:15:51,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:15:51,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ee13bbc022e79264569d HTTP/1.1" 200 None 2026-03-27 14:15:51,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:15:51,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 14:15:51,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:15:51,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ee13bbc022e79264569d HTTP/1.1" 200 None 2026-03-27 14:15:51,474 - lava_callback - ERROR - Error processing callback for node 69c3ee13bbc022e79264569d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56748 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:17:11,469 - lava_callback - INFO - Received callback for node 69c3ee65bbc022e7926456ab INFO: 172.20.0.1:60140 - "POST /node/69c3ee65bbc022e7926456ab HTTP/1.0" 202 Accepted 2026-03-27 14:17:11,474 - lava_callback - INFO - Starting async job submit for node 69c3ee65bbc022e7926456ab 2026-03-27 14:17:11,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:17:11,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ee65bbc022e7926456ab HTTP/1.1" 200 None 2026-03-27 14:17:11,491 - lava_callback - DEBUG - Processing node 69c3ee65bbc022e7926456ab: baseline-arm64-baylibre 2026-03-27 14:17:11,491 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ee65bbc022e7926456ab 2026-03-27 14:17:11,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:17:11,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ee65bbc022e7926456ab HTTP/1.1" 200 None 2026-03-27 14:17:11,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:17:11,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 14:17:11,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:17:11,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ee65bbc022e7926456ab HTTP/1.1" 200 None 2026-03-27 14:17:11,561 - lava_callback - ERROR - Error processing callback for node 69c3ee65bbc022e7926456ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41748 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:18:51,902 - lava_callback - INFO - Received callback for node 69c3eec8bbc022e7926456bb INFO: 172.20.0.1:38154 - "POST /node/69c3eec8bbc022e7926456bb HTTP/1.0" 202 Accepted 2026-03-27 14:18:51,907 - lava_callback - INFO - Starting async job submit for node 69c3eec8bbc022e7926456bb 2026-03-27 14:18:51,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:18:51,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3eec8bbc022e7926456bb HTTP/1.1" 200 None 2026-03-27 14:18:51,945 - lava_callback - DEBUG - Processing node 69c3eec8bbc022e7926456bb: baseline-arm64-baylibre 2026-03-27 14:18:51,945 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3eec8bbc022e7926456bb 2026-03-27 14:18:51,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:18:51,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3eec8bbc022e7926456bb HTTP/1.1" 200 None 2026-03-27 14:18:51,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:18:51,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 14:18:51,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:18:51,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3eec8bbc022e7926456bb HTTP/1.1" 200 None 2026-03-27 14:18:52,008 - lava_callback - ERROR - Error processing callback for node 69c3eec8bbc022e7926456bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38222 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:20:35,163 - lava_callback - INFO - Received callback for node 69c3ef1fbbc022e7926456c8 INFO: 172.20.0.1:57182 - "POST /node/69c3ef1fbbc022e7926456c8 HTTP/1.0" 202 Accepted 2026-03-27 14:20:35,169 - lava_callback - INFO - Starting async job submit for node 69c3ef1fbbc022e7926456c8 2026-03-27 14:20:35,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:20:35,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ef1fbbc022e7926456c8 HTTP/1.1" 200 None 2026-03-27 14:20:35,204 - lava_callback - DEBUG - Processing node 69c3ef1fbbc022e7926456c8: baseline-arm64-clabbe 2026-03-27 14:20:35,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ef1fbbc022e7926456c8 2026-03-27 14:20:35,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:20:35,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ef1fbbc022e7926456c8 HTTP/1.1" 200 None 2026-03-27 14:20:35,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:20:35,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 14:20:35,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:20:35,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ef1fbbc022e7926456c8 HTTP/1.1" 200 None 2026-03-27 14:20:35,277 - lava_callback - ERROR - Error processing callback for node 69c3ef1fbbc022e7926456c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33810 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 14:22:15,227 - lava_callback - INFO - Received callback for node 69c3ef83bbc022e7926456d2 INFO: 172.20.0.1:33816 - "POST /node/69c3ef83bbc022e7926456d2 HTTP/1.0" 202 Accepted 2026-03-27 14:22:15,232 - lava_callback - INFO - Starting async job submit for node 69c3ef83bbc022e7926456d2 2026-03-27 14:22:15,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:22:15,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ef83bbc022e7926456d2 HTTP/1.1" 200 None 2026-03-27 14:22:15,280 - lava_callback - DEBUG - Processing node 69c3ef83bbc022e7926456d2: baseline-arm64-clabbe 2026-03-27 14:22:15,280 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ef83bbc022e7926456d2 2026-03-27 14:22:15,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:22:15,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ef83bbc022e7926456d2 HTTP/1.1" 200 None 2026-03-27 14:22:15,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:22:15,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 14:22:15,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:22:15,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ef83bbc022e7926456d2 HTTP/1.1" 200 None 2026-03-27 14:22:15,341 - lava_callback - ERROR - Error processing callback for node 69c3ef83bbc022e7926456d2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52034 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:23:35,210 - lava_callback - INFO - Received callback for node 69c3efd3bbc022e7926456e0 INFO: 172.20.0.1:55870 - "POST /node/69c3efd3bbc022e7926456e0 HTTP/1.0" 202 Accepted 2026-03-27 14:23:35,215 - lava_callback - INFO - Starting async job submit for node 69c3efd3bbc022e7926456e0 2026-03-27 14:23:35,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:23:35,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3efd3bbc022e7926456e0 HTTP/1.1" 200 None 2026-03-27 14:23:35,237 - lava_callback - DEBUG - Processing node 69c3efd3bbc022e7926456e0: baseline-arm64-clabbe 2026-03-27 14:23:35,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3efd3bbc022e7926456e0 2026-03-27 14:23:35,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:23:35,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3efd3bbc022e7926456e0 HTTP/1.1" 200 None 2026-03-27 14:23:35,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:23:35,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 14:23:35,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:23:35,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3efd3bbc022e7926456e0 HTTP/1.1" 200 None 2026-03-27 14:23:35,297 - lava_callback - ERROR - Error processing callback for node 69c3efd3bbc022e7926456e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55086 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:24:55,299 - lava_callback - INFO - Received callback for node 69c3f027bbc022e7926456e9 INFO: 172.20.0.1:54316 - "POST /node/69c3f027bbc022e7926456e9 HTTP/1.0" 202 Accepted 2026-03-27 14:24:55,305 - lava_callback - INFO - Starting async job submit for node 69c3f027bbc022e7926456e9 2026-03-27 14:24:55,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:24:55,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f027bbc022e7926456e9 HTTP/1.1" 200 None 2026-03-27 14:24:55,321 - lava_callback - DEBUG - Processing node 69c3f027bbc022e7926456e9: baseline-arm64-clabbe 2026-03-27 14:24:55,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f027bbc022e7926456e9 2026-03-27 14:24:55,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:24:55,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f027bbc022e7926456e9 HTTP/1.1" 200 None 2026-03-27 14:24:55,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:24:55,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 14:24:55,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:24:55,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f027bbc022e7926456e9 HTTP/1.1" 200 None 2026-03-27 14:24:55,382 - lava_callback - ERROR - Error processing callback for node 69c3f027bbc022e7926456e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44466 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 14:26:15,403 - lava_callback - INFO - Received callback for node 69c3f079bbc022e7926456f5 INFO: 172.20.0.1:44476 - "POST /node/69c3f079bbc022e7926456f5 HTTP/1.0" 202 Accepted 2026-03-27 14:26:15,409 - lava_callback - INFO - Starting async job submit for node 69c3f079bbc022e7926456f5 2026-03-27 14:26:15,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:26:15,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f079bbc022e7926456f5 HTTP/1.1" 200 None 2026-03-27 14:26:15,426 - lava_callback - DEBUG - Processing node 69c3f079bbc022e7926456f5: baseline-arm64-clabbe 2026-03-27 14:26:15,427 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f079bbc022e7926456f5 2026-03-27 14:26:15,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:26:15,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f079bbc022e7926456f5 HTTP/1.1" 200 None 2026-03-27 14:26:15,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:26:15,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 14:26:15,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:26:15,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f079bbc022e7926456f5 HTTP/1.1" 200 None 2026-03-27 14:26:15,509 - lava_callback - ERROR - Error processing callback for node 69c3f079bbc022e7926456f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35912 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:27:35,398 - lava_callback - INFO - Received callback for node 69c3f0c8bbc022e792645704 INFO: 172.20.0.1:39780 - "POST /node/69c3f0c8bbc022e792645704 HTTP/1.0" 202 Accepted 2026-03-27 14:27:35,403 - lava_callback - INFO - Starting async job submit for node 69c3f0c8bbc022e792645704 2026-03-27 14:27:35,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:27:35,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f0c8bbc022e792645704 HTTP/1.1" 200 None 2026-03-27 14:27:35,419 - lava_callback - DEBUG - Processing node 69c3f0c8bbc022e792645704: baseline-arm64-clabbe 2026-03-27 14:27:35,420 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f0c8bbc022e792645704 2026-03-27 14:27:35,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:27:35,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f0c8bbc022e792645704 HTTP/1.1" 200 None 2026-03-27 14:27:35,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:27:35,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c086b1ce0c9bc43bccb63a HTTP/1.1" 200 None 2026-03-27 14:27:35,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:27:35,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f0c8bbc022e792645704 HTTP/1.1" 200 None 2026-03-27 14:27:35,514 - lava_callback - ERROR - Error processing callback for node 69c3f0c8bbc022e792645704: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41558 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:38:12,688 - lava_callback - INFO - Received callback for node 69c3f346bbc022e792645746 INFO: 172.20.0.1:56764 - "POST /node/69c3f346bbc022e792645746 HTTP/1.0" 202 Accepted 2026-03-27 14:38:12,694 - lava_callback - INFO - Starting async job submit for node 69c3f346bbc022e792645746 2026-03-27 14:38:12,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:38:12,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f346bbc022e792645746 HTTP/1.1" 200 None 2026-03-27 14:38:12,711 - lava_callback - DEBUG - Processing node 69c3f346bbc022e792645746: baseline-arm-baylibre 2026-03-27 14:38:12,711 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f346bbc022e792645746 2026-03-27 14:38:12,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:38:12,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f346bbc022e792645746 HTTP/1.1" 200 None 2026-03-27 14:38:12,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:38:12,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:38:12,743 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:38:12,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f346bbc022e792645746 HTTP/1.1" 200 None 2026-03-27 14:38:12,772 - lava_callback - ERROR - Error processing callback for node 69c3f346bbc022e792645746: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56772 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:39:12,691 - lava_callback - INFO - Received callback for node 69c3f37dbbc022e792645753 INFO: 172.20.0.1:54834 - "POST /node/69c3f37dbbc022e792645753 HTTP/1.0" 202 Accepted 2026-03-27 14:39:12,697 - lava_callback - INFO - Starting async job submit for node 69c3f37dbbc022e792645753 2026-03-27 14:39:12,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:39:12,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f37dbbc022e792645753 HTTP/1.1" 200 None 2026-03-27 14:39:12,714 - lava_callback - DEBUG - Processing node 69c3f37dbbc022e792645753: baseline-arm-baylibre 2026-03-27 14:39:12,714 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f37dbbc022e792645753 2026-03-27 14:39:12,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:39:12,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f37dbbc022e792645753 HTTP/1.1" 200 None 2026-03-27 14:39:12,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:39:12,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:39:12,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:39:12,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f37dbbc022e792645753 HTTP/1.1" 200 None 2026-03-27 14:39:12,791 - lava_callback - ERROR - Error processing callback for node 69c3f37dbbc022e792645753: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52842 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:39:52,762 - lava_callback - INFO - Received callback for node 69c3f3a6bbc022e79264575c INFO: 172.20.0.1:48116 - "POST /node/69c3f3a6bbc022e79264575c HTTP/1.0" 202 Accepted 2026-03-27 14:39:52,768 - lava_callback - INFO - Starting async job submit for node 69c3f3a6bbc022e79264575c 2026-03-27 14:39:52,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:39:52,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f3a6bbc022e79264575c HTTP/1.1" 200 None 2026-03-27 14:39:52,784 - lava_callback - DEBUG - Processing node 69c3f3a6bbc022e79264575c: baseline-arm-baylibre 2026-03-27 14:39:52,785 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f3a6bbc022e79264575c 2026-03-27 14:39:52,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:39:52,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f3a6bbc022e79264575c HTTP/1.1" 200 None 2026-03-27 14:39:52,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:39:52,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:39:52,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:39:52,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f3a6bbc022e79264575c HTTP/1.1" 200 None 2026-03-27 14:39:52,838 - lava_callback - ERROR - Error processing callback for node 69c3f3a6bbc022e79264575c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55006 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:40:32,765 - lava_callback - INFO - Received callback for node 69c3f3d1bbc022e792645769 INFO: 172.20.0.1:44312 - "POST /node/69c3f3d1bbc022e792645769 HTTP/1.0" 202 Accepted 2026-03-27 14:40:32,770 - lava_callback - INFO - Starting async job submit for node 69c3f3d1bbc022e792645769 2026-03-27 14:40:32,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:40:32,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f3d1bbc022e792645769 HTTP/1.1" 200 None 2026-03-27 14:40:32,795 - lava_callback - DEBUG - Processing node 69c3f3d1bbc022e792645769: baseline-arm-baylibre 2026-03-27 14:40:32,795 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f3d1bbc022e792645769 2026-03-27 14:40:32,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:40:32,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f3d1bbc022e792645769 HTTP/1.1" 200 None 2026-03-27 14:40:32,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:40:32,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:40:32,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:40:32,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f3d1bbc022e792645769 HTTP/1.1" 200 None 2026-03-27 14:40:32,859 - lava_callback - ERROR - Error processing callback for node 69c3f3d1bbc022e792645769: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38190 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:41:12,848 - lava_callback - INFO - Received callback for node 69c3f3fcbbc022e792645776 INFO: 172.20.0.1:60110 - "POST /node/69c3f3fcbbc022e792645776 HTTP/1.0" 202 Accepted 2026-03-27 14:41:12,853 - lava_callback - INFO - Starting async job submit for node 69c3f3fcbbc022e792645776 2026-03-27 14:41:12,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:41:12,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f3fcbbc022e792645776 HTTP/1.1" 200 None 2026-03-27 14:41:12,874 - lava_callback - DEBUG - Processing node 69c3f3fcbbc022e792645776: baseline-arm-baylibre 2026-03-27 14:41:12,875 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f3fcbbc022e792645776 2026-03-27 14:41:12,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:41:12,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f3fcbbc022e792645776 HTTP/1.1" 200 None 2026-03-27 14:41:12,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:41:12,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:41:12,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:41:12,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f3fcbbc022e792645776 HTTP/1.1" 200 None 2026-03-27 14:41:12,941 - lava_callback - ERROR - Error processing callback for node 69c3f3fcbbc022e792645776: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57674 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:41:52,819 - lava_callback - INFO - Received callback for node 69c3f424bbc022e792645787 INFO: 172.20.0.1:42026 - "POST /node/69c3f424bbc022e792645787 HTTP/1.0" 202 Accepted 2026-03-27 14:41:52,825 - lava_callback - INFO - Starting async job submit for node 69c3f424bbc022e792645787 2026-03-27 14:41:52,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:41:52,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f424bbc022e792645787 HTTP/1.1" 200 None 2026-03-27 14:41:52,841 - lava_callback - DEBUG - Processing node 69c3f424bbc022e792645787: baseline-arm-baylibre 2026-03-27 14:41:52,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f424bbc022e792645787 2026-03-27 14:41:52,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:41:52,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f424bbc022e792645787 HTTP/1.1" 200 None 2026-03-27 14:41:52,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:41:52,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:41:52,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:41:52,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f424bbc022e792645787 HTTP/1.1" 200 None 2026-03-27 14:41:52,915 - lava_callback - ERROR - Error processing callback for node 69c3f424bbc022e792645787: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47564 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:43:56,165 - lava_callback - INFO - Received callback for node 69c3f49dbbc022e7926457b3 INFO: 172.20.0.1:38762 - "POST /node/69c3f49dbbc022e7926457b3 HTTP/1.0" 202 Accepted 2026-03-27 14:43:56,171 - lava_callback - INFO - Starting async job submit for node 69c3f49dbbc022e7926457b3 2026-03-27 14:43:56,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:43:56,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f49dbbc022e7926457b3 HTTP/1.1" 200 None 2026-03-27 14:43:56,187 - lava_callback - DEBUG - Processing node 69c3f49dbbc022e7926457b3: baseline-arm-clabbe 2026-03-27 14:43:56,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f49dbbc022e7926457b3 2026-03-27 14:43:56,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:43:56,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f49dbbc022e7926457b3 HTTP/1.1" 200 None 2026-03-27 14:43:56,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:43:56,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:43:56,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:43:56,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f49dbbc022e7926457b3 HTTP/1.1" 200 None 2026-03-27 14:43:56,241 - lava_callback - ERROR - Error processing callback for node 69c3f49dbbc022e7926457b3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54348 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:44:36,169 - lava_callback - INFO - Received callback for node 69c3f4c6bbc022e7926457bc INFO: 172.20.0.1:36916 - "POST /node/69c3f4c6bbc022e7926457bc HTTP/1.0" 202 Accepted 2026-03-27 14:44:36,175 - lava_callback - INFO - Starting async job submit for node 69c3f4c6bbc022e7926457bc 2026-03-27 14:44:36,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:44:36,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f4c6bbc022e7926457bc HTTP/1.1" 200 None 2026-03-27 14:44:36,191 - lava_callback - DEBUG - Processing node 69c3f4c6bbc022e7926457bc: baseline-arm-clabbe 2026-03-27 14:44:36,192 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f4c6bbc022e7926457bc 2026-03-27 14:44:36,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:44:36,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f4c6bbc022e7926457bc HTTP/1.1" 200 None 2026-03-27 14:44:36,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:44:36,220 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:44:36,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:44:36,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f4c6bbc022e7926457bc HTTP/1.1" 200 None 2026-03-27 14:44:36,244 - lava_callback - ERROR - Error processing callback for node 69c3f4c6bbc022e7926457bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60288 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 14:45:16,242 - lava_callback - INFO - Received callback for node 69c3f4efbbc022e7926457c8 INFO: 172.20.0.1:60290 - "POST /node/69c3f4efbbc022e7926457c8 HTTP/1.0" 202 Accepted 2026-03-27 14:45:16,248 - lava_callback - INFO - Starting async job submit for node 69c3f4efbbc022e7926457c8 2026-03-27 14:45:16,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:45:16,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f4efbbc022e7926457c8 HTTP/1.1" 200 None 2026-03-27 14:45:16,264 - lava_callback - DEBUG - Processing node 69c3f4efbbc022e7926457c8: baseline-arm-clabbe 2026-03-27 14:45:16,264 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f4efbbc022e7926457c8 2026-03-27 14:45:16,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:45:16,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f4efbbc022e7926457c8 HTTP/1.1" 200 None 2026-03-27 14:45:16,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:45:16,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:45:16,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:45:16,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f4efbbc022e7926457c8 HTTP/1.1" 200 None 2026-03-27 14:45:16,320 - lava_callback - ERROR - Error processing callback for node 69c3f4efbbc022e7926457c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43422 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:45:56,227 - lava_callback - INFO - Received callback for node 69c3f518bbc022e7926457d8 INFO: 172.20.0.1:45634 - "POST /node/69c3f518bbc022e7926457d8 HTTP/1.0" 202 Accepted 2026-03-27 14:45:56,232 - lava_callback - INFO - Starting async job submit for node 69c3f518bbc022e7926457d8 2026-03-27 14:45:56,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:45:56,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f518bbc022e7926457d8 HTTP/1.1" 200 None 2026-03-27 14:45:56,249 - lava_callback - DEBUG - Processing node 69c3f518bbc022e7926457d8: baseline-arm-clabbe 2026-03-27 14:45:56,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f518bbc022e7926457d8 2026-03-27 14:45:56,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:45:56,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f518bbc022e7926457d8 HTTP/1.1" 200 None 2026-03-27 14:45:56,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:45:56,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:45:56,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:45:56,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f518bbc022e7926457d8 HTTP/1.1" 200 None 2026-03-27 14:45:56,321 - lava_callback - ERROR - Error processing callback for node 69c3f518bbc022e7926457d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33988 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:46:36,297 - lava_callback - INFO - Received callback for node 69c3f541bbc022e7926457e3 INFO: 172.20.0.1:34642 - "POST /node/69c3f541bbc022e7926457e3 HTTP/1.0" 202 Accepted 2026-03-27 14:46:36,302 - lava_callback - INFO - Starting async job submit for node 69c3f541bbc022e7926457e3 2026-03-27 14:46:36,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:46:36,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f541bbc022e7926457e3 HTTP/1.1" 200 None 2026-03-27 14:46:36,318 - lava_callback - DEBUG - Processing node 69c3f541bbc022e7926457e3: baseline-arm-clabbe 2026-03-27 14:46:36,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f541bbc022e7926457e3 2026-03-27 14:46:36,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:46:36,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f541bbc022e7926457e3 HTTP/1.1" 200 None 2026-03-27 14:46:36,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:46:36,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-27 14:46:36,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:46:36,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f541bbc022e7926457e3 HTTP/1.1" 200 None 2026-03-27 14:46:36,376 - lava_callback - ERROR - Error processing callback for node 69c3f541bbc022e7926457e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42956 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:48:13,221 - lava_callback - INFO - Received callback for node 69c3f5a6bbc022e7926457f8 INFO: 172.20.0.1:33796 - "POST /node/69c3f5a6bbc022e7926457f8 HTTP/1.0" 202 Accepted 2026-03-27 14:48:13,227 - lava_callback - INFO - Starting async job submit for node 69c3f5a6bbc022e7926457f8 2026-03-27 14:48:13,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:48:13,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f5a6bbc022e7926457f8 HTTP/1.1" 200 None 2026-03-27 14:48:13,243 - lava_callback - DEBUG - Processing node 69c3f5a6bbc022e7926457f8: baseline-arm-baylibre 2026-03-27 14:48:13,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f5a6bbc022e7926457f8 2026-03-27 14:48:13,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:48:13,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f5a6bbc022e7926457f8 HTTP/1.1" 200 None 2026-03-27 14:48:13,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:48:13,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:48:13,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:48:13,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f5a6bbc022e7926457f8 HTTP/1.1" 200 None 2026-03-27 14:48:13,309 - lava_callback - ERROR - Error processing callback for node 69c3f5a6bbc022e7926457f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42912 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:48:53,220 - lava_callback - INFO - Received callback for node 69c3f5cfbbc022e792645809 INFO: 172.20.0.1:42904 - "POST /node/69c3f5cfbbc022e792645809 HTTP/1.0" 202 Accepted 2026-03-27 14:48:53,225 - lava_callback - INFO - Starting async job submit for node 69c3f5cfbbc022e792645809 2026-03-27 14:48:53,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:48:53,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f5cfbbc022e792645809 HTTP/1.1" 200 None 2026-03-27 14:48:53,251 - lava_callback - DEBUG - Processing node 69c3f5cfbbc022e792645809: baseline-arm-baylibre 2026-03-27 14:48:53,251 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f5cfbbc022e792645809 2026-03-27 14:48:53,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:48:53,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f5cfbbc022e792645809 HTTP/1.1" 200 None 2026-03-27 14:48:53,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:48:53,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:48:53,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:48:53,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f5cfbbc022e792645809 HTTP/1.1" 200 None 2026-03-27 14:48:53,323 - lava_callback - ERROR - Error processing callback for node 69c3f5cfbbc022e792645809: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54324 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:49:33,474 - lava_callback - INFO - Received callback for node 69c3f5f8bbc022e792645816 INFO: 172.20.0.1:38982 - "POST /node/69c3f5f8bbc022e792645816 HTTP/1.0" 202 Accepted 2026-03-27 14:49:33,479 - lava_callback - INFO - Starting async job submit for node 69c3f5f8bbc022e792645816 2026-03-27 14:49:33,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:49:33,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f5f8bbc022e792645816 HTTP/1.1" 200 None 2026-03-27 14:49:33,503 - lava_callback - DEBUG - Processing node 69c3f5f8bbc022e792645816: baseline-arm-baylibre 2026-03-27 14:49:33,503 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f5f8bbc022e792645816 2026-03-27 14:49:33,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:49:33,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f5f8bbc022e792645816 HTTP/1.1" 200 None 2026-03-27 14:49:33,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:49:33,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:49:33,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:49:33,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f5f8bbc022e792645816 HTTP/1.1" 200 None 2026-03-27 14:49:33,558 - lava_callback - ERROR - Error processing callback for node 69c3f5f8bbc022e792645816: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46396 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:50:33,293 - lava_callback - INFO - Received callback for node 69c3f624bbc022e792645826 INFO: 172.20.0.1:44698 - "POST /node/69c3f624bbc022e792645826 HTTP/1.0" 202 Accepted 2026-03-27 14:50:33,298 - lava_callback - INFO - Starting async job submit for node 69c3f624bbc022e792645826 2026-03-27 14:50:33,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:50:33,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f624bbc022e792645826 HTTP/1.1" 200 None 2026-03-27 14:50:33,315 - lava_callback - DEBUG - Processing node 69c3f624bbc022e792645826: baseline-arm-baylibre 2026-03-27 14:50:33,315 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f624bbc022e792645826 2026-03-27 14:50:33,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:50:33,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f624bbc022e792645826 HTTP/1.1" 200 None 2026-03-27 14:50:33,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:50:33,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:50:33,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:50:33,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f624bbc022e792645826 HTTP/1.1" 200 None 2026-03-27 14:50:33,371 - lava_callback - ERROR - Error processing callback for node 69c3f624bbc022e792645826: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48032 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:51:13,374 - lava_callback - INFO - Received callback for node 69c3f64dbbc022e79264582f INFO: 172.20.0.1:37588 - "POST /node/69c3f64dbbc022e79264582f HTTP/1.0" 202 Accepted 2026-03-27 14:51:13,379 - lava_callback - INFO - Starting async job submit for node 69c3f64dbbc022e79264582f 2026-03-27 14:51:13,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:51:13,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f64dbbc022e79264582f HTTP/1.1" 200 None 2026-03-27 14:51:13,396 - lava_callback - DEBUG - Processing node 69c3f64dbbc022e79264582f: baseline-arm-baylibre 2026-03-27 14:51:13,396 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f64dbbc022e79264582f 2026-03-27 14:51:13,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:51:13,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f64dbbc022e79264582f HTTP/1.1" 200 None 2026-03-27 14:51:13,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:51:13,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:51:13,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:51:13,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f64dbbc022e79264582f HTTP/1.1" 200 None 2026-03-27 14:51:13,467 - lava_callback - ERROR - Error processing callback for node 69c3f64dbbc022e79264582f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47872 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:51:53,341 - lava_callback - INFO - Received callback for node 69c3f676bbc022e79264583c INFO: 172.20.0.1:54614 - "POST /node/69c3f676bbc022e79264583c HTTP/1.0" 202 Accepted 2026-03-27 14:51:53,346 - lava_callback - INFO - Starting async job submit for node 69c3f676bbc022e79264583c 2026-03-27 14:51:53,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:51:53,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f676bbc022e79264583c HTTP/1.1" 200 None 2026-03-27 14:51:53,362 - lava_callback - DEBUG - Processing node 69c3f676bbc022e79264583c: baseline-arm-baylibre 2026-03-27 14:51:53,363 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f676bbc022e79264583c 2026-03-27 14:51:53,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:51:53,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f676bbc022e79264583c HTTP/1.1" 200 None 2026-03-27 14:51:53,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:51:53,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:51:53,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:51:53,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f676bbc022e79264583c HTTP/1.1" 200 None 2026-03-27 14:51:53,427 - lava_callback - ERROR - Error processing callback for node 69c3f676bbc022e79264583c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44878 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:52:33,389 - lava_callback - INFO - Received callback for node 69c3f69dbbc022e79264584a INFO: 172.20.0.1:59232 - "POST /node/69c3f69dbbc022e79264584a HTTP/1.0" 202 Accepted 2026-03-27 14:52:33,395 - lava_callback - INFO - Starting async job submit for node 69c3f69dbbc022e79264584a 2026-03-27 14:52:33,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:52:33,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f69dbbc022e79264584a HTTP/1.1" 200 None 2026-03-27 14:52:33,417 - lava_callback - DEBUG - Processing node 69c3f69dbbc022e79264584a: baseline-arm-baylibre 2026-03-27 14:52:33,418 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f69dbbc022e79264584a 2026-03-27 14:52:33,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:52:33,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f69dbbc022e79264584a HTTP/1.1" 200 None 2026-03-27 14:52:33,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:52:33,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:52:33,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:52:33,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f69dbbc022e79264584a HTTP/1.1" 200 None 2026-03-27 14:52:33,483 - lava_callback - ERROR - Error processing callback for node 69c3f69dbbc022e79264584a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34368 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:53:13,389 - lava_callback - INFO - Received callback for node 69c3f6c5bbc022e792645867 INFO: 172.20.0.1:56914 - "POST /node/69c3f6c5bbc022e792645867 HTTP/1.0" 202 Accepted 2026-03-27 14:53:13,395 - lava_callback - INFO - Starting async job submit for node 69c3f6c5bbc022e792645867 2026-03-27 14:53:13,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:53:13,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f6c5bbc022e792645867 HTTP/1.1" 200 None 2026-03-27 14:53:13,414 - lava_callback - DEBUG - Processing node 69c3f6c5bbc022e792645867: baseline-arm-baylibre 2026-03-27 14:53:13,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f6c5bbc022e792645867 2026-03-27 14:53:13,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:53:13,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f6c5bbc022e792645867 HTTP/1.1" 200 None 2026-03-27 14:53:13,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:53:13,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:53:13,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:53:13,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f6c5bbc022e792645867 HTTP/1.1" 200 None 2026-03-27 14:53:13,474 - lava_callback - ERROR - Error processing callback for node 69c3f6c5bbc022e792645867: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58652 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:53:53,537 - lava_callback - INFO - Received callback for node 69c3f6f0bbc022e79264587d INFO: 172.20.0.1:52062 - "POST /node/69c3f6f0bbc022e79264587d HTTP/1.0" 202 Accepted 2026-03-27 14:53:53,542 - lava_callback - INFO - Starting async job submit for node 69c3f6f0bbc022e79264587d 2026-03-27 14:53:53,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:53:53,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f6f0bbc022e79264587d HTTP/1.1" 200 None 2026-03-27 14:53:53,558 - lava_callback - DEBUG - Processing node 69c3f6f0bbc022e79264587d: baseline-arm-baylibre 2026-03-27 14:53:53,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f6f0bbc022e79264587d 2026-03-27 14:53:53,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:53:53,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f6f0bbc022e79264587d HTTP/1.1" 200 None 2026-03-27 14:53:53,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:53:53,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:53:53,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:53:53,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f6f0bbc022e79264587d HTTP/1.1" 200 None 2026-03-27 14:53:53,627 - lava_callback - ERROR - Error processing callback for node 69c3f6f0bbc022e79264587d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48752 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:54:33,492 - lava_callback - INFO - Received callback for node 69c3f719bbc022e7926458a5 INFO: 172.20.0.1:35338 - "POST /node/69c3f719bbc022e7926458a5 HTTP/1.0" 202 Accepted 2026-03-27 14:54:33,497 - lava_callback - INFO - Starting async job submit for node 69c3f719bbc022e7926458a5 2026-03-27 14:54:33,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:54:33,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f719bbc022e7926458a5 HTTP/1.1" 200 None 2026-03-27 14:54:33,513 - lava_callback - DEBUG - Processing node 69c3f719bbc022e7926458a5: baseline-arm-baylibre 2026-03-27 14:54:33,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f719bbc022e7926458a5 2026-03-27 14:54:33,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:54:33,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f719bbc022e7926458a5 HTTP/1.1" 200 None 2026-03-27 14:54:33,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:54:33,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:54:33,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:54:33,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f719bbc022e7926458a5 HTTP/1.1" 200 None 2026-03-27 14:54:33,568 - lava_callback - ERROR - Error processing callback for node 69c3f719bbc022e7926458a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53408 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:55:13,674 - lava_callback - INFO - Received callback for node 69c3f742bbc022e7926458b7 INFO: 172.20.0.1:42256 - "POST /node/69c3f742bbc022e7926458b7 HTTP/1.0" 202 Accepted 2026-03-27 14:55:13,679 - lava_callback - INFO - Starting async job submit for node 69c3f742bbc022e7926458b7 2026-03-27 14:55:13,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:55:13,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f742bbc022e7926458b7 HTTP/1.1" 200 None 2026-03-27 14:55:13,730 - lava_callback - DEBUG - Processing node 69c3f742bbc022e7926458b7: baseline-arm-baylibre 2026-03-27 14:55:13,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f742bbc022e7926458b7 2026-03-27 14:55:13,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:55:13,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f742bbc022e7926458b7 HTTP/1.1" 200 None 2026-03-27 14:55:13,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:55:13,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:55:13,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:55:13,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f742bbc022e7926458b7 HTTP/1.1" 200 None 2026-03-27 14:55:13,782 - lava_callback - ERROR - Error processing callback for node 69c3f742bbc022e7926458b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36704 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:55:53,576 - lava_callback - INFO - Received callback for node 69c3f768bbc022e7926458c2 INFO: 172.20.0.1:55820 - "POST /node/69c3f768bbc022e7926458c2 HTTP/1.0" 202 Accepted 2026-03-27 14:55:53,582 - lava_callback - INFO - Starting async job submit for node 69c3f768bbc022e7926458c2 2026-03-27 14:55:53,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:55:53,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f768bbc022e7926458c2 HTTP/1.1" 200 None 2026-03-27 14:55:53,617 - lava_callback - DEBUG - Processing node 69c3f768bbc022e7926458c2: baseline-arm-baylibre 2026-03-27 14:55:53,617 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f768bbc022e7926458c2 2026-03-27 14:55:53,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:55:53,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f768bbc022e7926458c2 HTTP/1.1" 200 None 2026-03-27 14:55:53,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:55:53,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:55:53,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:55:53,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f768bbc022e7926458c2 HTTP/1.1" 200 None 2026-03-27 14:55:53,690 - lava_callback - ERROR - Error processing callback for node 69c3f768bbc022e7926458c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48344 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:56:33,734 - lava_callback - INFO - Received callback for node 69c3f793bbc022e7926458d3 INFO: 172.20.0.1:33630 - "POST /node/69c3f793bbc022e7926458d3 HTTP/1.0" 202 Accepted 2026-03-27 14:56:33,740 - lava_callback - INFO - Starting async job submit for node 69c3f793bbc022e7926458d3 2026-03-27 14:56:33,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:56:33,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f793bbc022e7926458d3 HTTP/1.1" 200 None 2026-03-27 14:56:33,756 - lava_callback - DEBUG - Processing node 69c3f793bbc022e7926458d3: baseline-arm-baylibre 2026-03-27 14:56:33,756 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f793bbc022e7926458d3 2026-03-27 14:56:33,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:56:33,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f793bbc022e7926458d3 HTTP/1.1" 200 None 2026-03-27 14:56:33,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:56:33,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:56:33,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:56:33,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f793bbc022e7926458d3 HTTP/1.1" 200 None 2026-03-27 14:56:33,818 - lava_callback - ERROR - Error processing callback for node 69c3f793bbc022e7926458d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42568 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:57:13,641 - lava_callback - INFO - Received callback for node 69c3f7bbbbc022e7926458e1 INFO: 172.20.0.1:33626 - "POST /node/69c3f7bbbbc022e7926458e1 HTTP/1.0" 202 Accepted 2026-03-27 14:57:13,647 - lava_callback - INFO - Starting async job submit for node 69c3f7bbbbc022e7926458e1 2026-03-27 14:57:13,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:57:13,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f7bbbbc022e7926458e1 HTTP/1.1" 200 None 2026-03-27 14:57:13,664 - lava_callback - DEBUG - Processing node 69c3f7bbbbc022e7926458e1: baseline-arm-baylibre 2026-03-27 14:57:13,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f7bbbbc022e7926458e1 2026-03-27 14:57:13,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:57:13,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f7bbbbc022e7926458e1 HTTP/1.1" 200 None 2026-03-27 14:57:13,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:57:13,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:57:13,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:57:13,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f7bbbbc022e7926458e1 HTTP/1.1" 200 None 2026-03-27 14:57:13,720 - lava_callback - ERROR - Error processing callback for node 69c3f7bbbbc022e7926458e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56214 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:57:56,866 - lava_callback - INFO - Received callback for node 69c3f7e6bbc022e7926458f5 INFO: 172.20.0.1:56222 - "POST /node/69c3f7e6bbc022e7926458f5 HTTP/1.0" 202 Accepted 2026-03-27 14:57:56,872 - lava_callback - INFO - Starting async job submit for node 69c3f7e6bbc022e7926458f5 2026-03-27 14:57:56,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:57:56,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f7e6bbc022e7926458f5 HTTP/1.1" 200 None 2026-03-27 14:57:56,892 - lava_callback - DEBUG - Processing node 69c3f7e6bbc022e7926458f5: baseline-arm-clabbe 2026-03-27 14:57:56,892 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f7e6bbc022e7926458f5 2026-03-27 14:57:56,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:57:56,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f7e6bbc022e7926458f5 HTTP/1.1" 200 None 2026-03-27 14:57:56,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:57:56,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:57:56,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:57:56,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f7e6bbc022e7926458f5 HTTP/1.1" 200 None 2026-03-27 14:57:56,949 - lava_callback - ERROR - Error processing callback for node 69c3f7e6bbc022e7926458f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39488 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:58:33,723 - lava_callback - INFO - Received callback for node 69c3f80abbc022e7926458fe INFO: 172.20.0.1:56762 - "POST /node/69c3f80abbc022e7926458fe HTTP/1.0" 202 Accepted 2026-03-27 14:58:33,729 - lava_callback - INFO - Starting async job submit for node 69c3f80abbc022e7926458fe 2026-03-27 14:58:33,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:58:33,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f80abbc022e7926458fe HTTP/1.1" 200 None 2026-03-27 14:58:33,748 - lava_callback - DEBUG - Processing node 69c3f80abbc022e7926458fe: baseline-arm-baylibre 2026-03-27 14:58:33,748 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f80abbc022e7926458fe 2026-03-27 14:58:33,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:58:33,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f80abbc022e7926458fe HTTP/1.1" 200 None 2026-03-27 14:58:33,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:58:33,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:58:33,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:58:33,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f80abbc022e7926458fe HTTP/1.1" 200 None 2026-03-27 14:58:33,821 - lava_callback - ERROR - Error processing callback for node 69c3f80abbc022e7926458fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33114 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:59:16,864 - lava_callback - INFO - Received callback for node 69c3f83bbbc022e79264590e INFO: 172.20.0.1:33122 - "POST /node/69c3f83bbbc022e79264590e HTTP/1.0" 202 Accepted 2026-03-27 14:59:16,870 - lava_callback - INFO - Starting async job submit for node 69c3f83bbbc022e79264590e 2026-03-27 14:59:16,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:59:16,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f83bbbc022e79264590e HTTP/1.1" 200 None 2026-03-27 14:59:16,886 - lava_callback - DEBUG - Processing node 69c3f83bbbc022e79264590e: baseline-arm-clabbe 2026-03-27 14:59:16,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f83bbbc022e79264590e 2026-03-27 14:59:16,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:59:16,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f83bbbc022e79264590e HTTP/1.1" 200 None 2026-03-27 14:59:16,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:59:16,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:59:16,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:59:16,953 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f83bbbc022e79264590e HTTP/1.1" 200 None 2026-03-27 14:59:16,955 - lava_callback - ERROR - Error processing callback for node 69c3f83bbbc022e79264590e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK 2026-03-27 14:59:53,798 - lava_callback - INFO - Received callback for node 69c3f860bbc022e79264591f INFO: 172.20.0.1:51276 - "POST /node/69c3f860bbc022e79264591f HTTP/1.0" 202 Accepted 2026-03-27 14:59:53,804 - lava_callback - INFO - Starting async job submit for node 69c3f860bbc022e79264591f 2026-03-27 14:59:53,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:59:53,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f860bbc022e79264591f HTTP/1.1" 200 None 2026-03-27 14:59:53,840 - lava_callback - DEBUG - Processing node 69c3f860bbc022e79264591f: baseline-arm-baylibre 2026-03-27 14:59:53,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f860bbc022e79264591f 2026-03-27 14:59:53,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:59:53,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f860bbc022e79264591f HTTP/1.1" 200 None 2026-03-27 14:59:53,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:59:53,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 14:59:53,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 14:59:53,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f860bbc022e79264591f HTTP/1.1" 200 None 2026-03-27 14:59:53,915 - lava_callback - ERROR - Error processing callback for node 69c3f860bbc022e79264591f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46260 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:00:56,919 - lava_callback - INFO - Received callback for node 69c3f893bbc022e79264592e INFO: 172.20.0.1:46262 - "POST /node/69c3f893bbc022e79264592e HTTP/1.0" 202 Accepted 2026-03-27 15:00:56,924 - lava_callback - INFO - Starting async job submit for node 69c3f893bbc022e79264592e 2026-03-27 15:00:56,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:00:56,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f893bbc022e79264592e HTTP/1.1" 200 None 2026-03-27 15:00:56,962 - lava_callback - DEBUG - Processing node 69c3f893bbc022e79264592e: baseline-arm-clabbe 2026-03-27 15:00:56,962 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f893bbc022e79264592e 2026-03-27 15:00:56,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:00:56,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f893bbc022e79264592e HTTP/1.1" 200 None 2026-03-27 15:00:56,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:00:57,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:00:57,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:00:57,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f893bbc022e79264592e HTTP/1.1" 200 None 2026-03-27 15:00:57,065 - lava_callback - ERROR - Error processing callback for node 69c3f893bbc022e79264592e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47318 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:01:33,987 - lava_callback - INFO - Received callback for node 69c3f8b9bbc022e792645946 INFO: 172.20.0.1:40378 - "POST /node/69c3f8b9bbc022e792645946 HTTP/1.0" 202 Accepted 2026-03-27 15:01:33,992 - lava_callback - INFO - Starting async job submit for node 69c3f8b9bbc022e792645946 2026-03-27 15:01:33,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:01:34,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f8b9bbc022e792645946 HTTP/1.1" 200 None 2026-03-27 15:01:34,011 - lava_callback - DEBUG - Processing node 69c3f8b9bbc022e792645946: baseline-arm-baylibre 2026-03-27 15:01:34,011 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f8b9bbc022e792645946 2026-03-27 15:01:34,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:01:34,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f8b9bbc022e792645946 HTTP/1.1" 200 None 2026-03-27 15:01:34,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:01:34,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:01:34,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:01:34,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f8b9bbc022e792645946 HTTP/1.1" 200 None 2026-03-27 15:01:34,070 - lava_callback - ERROR - Error processing callback for node 69c3f8b9bbc022e792645946: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42376 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42384 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:02:17,004 - lava_callback - INFO - Received callback for node 69c3f8e5bbc022e792645954 INFO: 172.20.0.1:42388 - "POST /node/69c3f8e5bbc022e792645954 HTTP/1.0" 202 Accepted 2026-03-27 15:02:17,010 - lava_callback - INFO - Starting async job submit for node 69c3f8e5bbc022e792645954 2026-03-27 15:02:17,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:02:17,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f8e5bbc022e792645954 HTTP/1.1" 200 None 2026-03-27 15:02:17,026 - lava_callback - DEBUG - Processing node 69c3f8e5bbc022e792645954: baseline-arm-clabbe 2026-03-27 15:02:17,026 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f8e5bbc022e792645954 2026-03-27 15:02:17,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:02:17,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f8e5bbc022e792645954 HTTP/1.1" 200 None 2026-03-27 15:02:17,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:02:17,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:02:17,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:02:17,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f8e5bbc022e792645954 HTTP/1.1" 200 None 2026-03-27 15:02:17,267 - lava_callback - ERROR - Error processing callback for node 69c3f8e5bbc022e792645954: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32932 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:02:54,045 - lava_callback - INFO - Received callback for node 69c3f90dbbc022e792645966 INFO: 172.20.0.1:59800 - "POST /node/69c3f90dbbc022e792645966 HTTP/1.0" 202 Accepted 2026-03-27 15:02:54,050 - lava_callback - INFO - Starting async job submit for node 69c3f90dbbc022e792645966 2026-03-27 15:02:54,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:02:54,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f90dbbc022e792645966 HTTP/1.1" 200 None 2026-03-27 15:02:54,068 - lava_callback - DEBUG - Processing node 69c3f90dbbc022e792645966: baseline-arm-baylibre 2026-03-27 15:02:54,069 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3f90dbbc022e792645966 2026-03-27 15:02:54,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:02:54,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3f90dbbc022e792645966 HTTP/1.1" 200 None 2026-03-27 15:02:54,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:02:54,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:02:54,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:02:54,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3f90dbbc022e792645966 HTTP/1.1" 200 None 2026-03-27 15:02:54,148 - lava_callback - ERROR - Error processing callback for node 69c3f90dbbc022e792645966: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34128 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:10:34,417 - lava_callback - INFO - Received callback for node 69c3fad9bbc022e7926459b0 INFO: 172.20.0.1:55174 - "POST /node/69c3fad9bbc022e7926459b0 HTTP/1.0" 202 Accepted 2026-03-27 15:10:34,423 - lava_callback - INFO - Starting async job submit for node 69c3fad9bbc022e7926459b0 2026-03-27 15:10:34,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:10:34,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fad9bbc022e7926459b0 HTTP/1.1" 200 None 2026-03-27 15:10:34,439 - lava_callback - DEBUG - Processing node 69c3fad9bbc022e7926459b0: baseline-arm64-baylibre 2026-03-27 15:10:34,439 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fad9bbc022e7926459b0 2026-03-27 15:10:34,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:10:34,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fad9bbc022e7926459b0 HTTP/1.1" 200 None 2026-03-27 15:10:34,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:10:34,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:10:34,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:10:34,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fad9bbc022e7926459b0 HTTP/1.1" 200 None 2026-03-27 15:10:34,499 - lava_callback - ERROR - Error processing callback for node 69c3fad9bbc022e7926459b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36462 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:11:54,477 - lava_callback - INFO - Received callback for node 69c3fb32bbc022e7926459be INFO: 172.20.0.1:43642 - "POST /node/69c3fb32bbc022e7926459be HTTP/1.0" 202 Accepted 2026-03-27 15:11:54,482 - lava_callback - INFO - Starting async job submit for node 69c3fb32bbc022e7926459be 2026-03-27 15:11:54,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:11:54,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fb32bbc022e7926459be HTTP/1.1" 200 None 2026-03-27 15:11:54,498 - lava_callback - DEBUG - Processing node 69c3fb32bbc022e7926459be: baseline-arm64-baylibre 2026-03-27 15:11:54,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fb32bbc022e7926459be 2026-03-27 15:11:54,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:11:54,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fb32bbc022e7926459be HTTP/1.1" 200 None 2026-03-27 15:11:54,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:11:54,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:11:54,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:11:54,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fb32bbc022e7926459be HTTP/1.1" 200 None 2026-03-27 15:11:54,554 - lava_callback - ERROR - Error processing callback for node 69c3fb32bbc022e7926459be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49630 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:13:14,532 - lava_callback - INFO - Received callback for node 69c3fb85bbc022e7926459c9 INFO: 172.20.0.1:47896 - "POST /node/69c3fb85bbc022e7926459c9 HTTP/1.0" 202 Accepted 2026-03-27 15:13:14,538 - lava_callback - INFO - Starting async job submit for node 69c3fb85bbc022e7926459c9 2026-03-27 15:13:14,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:13:14,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fb85bbc022e7926459c9 HTTP/1.1" 200 None 2026-03-27 15:13:14,554 - lava_callback - DEBUG - Processing node 69c3fb85bbc022e7926459c9: baseline-arm64-baylibre 2026-03-27 15:13:14,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fb85bbc022e7926459c9 2026-03-27 15:13:14,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:13:14,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fb85bbc022e7926459c9 HTTP/1.1" 200 None 2026-03-27 15:13:14,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:13:14,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:13:14,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:13:14,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fb85bbc022e7926459c9 HTTP/1.1" 200 None 2026-03-27 15:13:14,608 - lava_callback - ERROR - Error processing callback for node 69c3fb85bbc022e7926459c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45656 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:14:34,685 - lava_callback - INFO - Received callback for node 69c3fbcabbc022e7926459d7 INFO: 172.20.0.1:56806 - "POST /node/69c3fbcabbc022e7926459d7 HTTP/1.0" 202 Accepted 2026-03-27 15:14:34,690 - lava_callback - INFO - Starting async job submit for node 69c3fbcabbc022e7926459d7 2026-03-27 15:14:34,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:14:34,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fbcabbc022e7926459d7 HTTP/1.1" 200 None 2026-03-27 15:14:34,706 - lava_callback - DEBUG - Processing node 69c3fbcabbc022e7926459d7: baseline-arm-baylibre 2026-03-27 15:14:34,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fbcabbc022e7926459d7 2026-03-27 15:14:34,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:14:34,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fbcabbc022e7926459d7 HTTP/1.1" 200 None 2026-03-27 15:14:34,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:14:34,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:14:34,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:14:34,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fbcabbc022e7926459d7 HTTP/1.1" 200 None 2026-03-27 15:14:34,759 - lava_callback - ERROR - Error processing callback for node 69c3fbcabbc022e7926459d7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59434 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:15:34,854 - lava_callback - INFO - Received callback for node 69c3fc01bbc022e7926459e9 INFO: 172.20.0.1:56498 - "POST /node/69c3fc01bbc022e7926459e9 HTTP/1.0" 202 Accepted 2026-03-27 15:15:34,860 - lava_callback - INFO - Starting async job submit for node 69c3fc01bbc022e7926459e9 2026-03-27 15:15:34,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:15:34,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fc01bbc022e7926459e9 HTTP/1.1" 200 None 2026-03-27 15:15:34,877 - lava_callback - DEBUG - Processing node 69c3fc01bbc022e7926459e9: baseline-arm64-baylibre 2026-03-27 15:15:34,877 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fc01bbc022e7926459e9 2026-03-27 15:15:34,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:15:34,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fc01bbc022e7926459e9 HTTP/1.1" 200 None 2026-03-27 15:15:34,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:15:34,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:15:34,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:15:34,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fc01bbc022e7926459e9 HTTP/1.1" 200 None 2026-03-27 15:15:34,939 - lava_callback - ERROR - Error processing callback for node 69c3fc01bbc022e7926459e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57298 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:16:54,743 - lava_callback - INFO - Received callback for node 69c3fc51bbc022e7926459f4 INFO: 172.20.0.1:51742 - "POST /node/69c3fc51bbc022e7926459f4 HTTP/1.0" 202 Accepted 2026-03-27 15:16:54,749 - lava_callback - INFO - Starting async job submit for node 69c3fc51bbc022e7926459f4 2026-03-27 15:16:54,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:16:54,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fc51bbc022e7926459f4 HTTP/1.1" 200 None 2026-03-27 15:16:54,764 - lava_callback - DEBUG - Processing node 69c3fc51bbc022e7926459f4: baseline-arm64-baylibre 2026-03-27 15:16:54,764 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fc51bbc022e7926459f4 2026-03-27 15:16:54,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:16:54,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fc51bbc022e7926459f4 HTTP/1.1" 200 None 2026-03-27 15:16:54,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:16:54,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:16:54,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:16:54,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fc51bbc022e7926459f4 HTTP/1.1" 200 None 2026-03-27 15:16:54,820 - lava_callback - ERROR - Error processing callback for node 69c3fc51bbc022e7926459f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54184 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:18:14,775 - lava_callback - INFO - Received callback for node 69c3fca5bbc022e792645a04 INFO: 172.20.0.1:59742 - "POST /node/69c3fca5bbc022e792645a04 HTTP/1.0" 202 Accepted 2026-03-27 15:18:14,781 - lava_callback - INFO - Starting async job submit for node 69c3fca5bbc022e792645a04 2026-03-27 15:18:14,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:18:14,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fca5bbc022e792645a04 HTTP/1.1" 200 None 2026-03-27 15:18:14,799 - lava_callback - DEBUG - Processing node 69c3fca5bbc022e792645a04: baseline-arm64-baylibre 2026-03-27 15:18:14,800 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fca5bbc022e792645a04 2026-03-27 15:18:14,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:18:14,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fca5bbc022e792645a04 HTTP/1.1" 200 None 2026-03-27 15:18:14,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:18:14,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:18:14,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:18:14,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fca5bbc022e792645a04 HTTP/1.1" 200 None 2026-03-27 15:18:14,868 - lava_callback - ERROR - Error processing callback for node 69c3fca5bbc022e792645a04: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49938 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:19:37,773 - lava_callback - INFO - Received callback for node 69c3fcfebbc022e792645a13 INFO: 172.20.0.1:49954 - "POST /node/69c3fcfebbc022e792645a13 HTTP/1.0" 202 Accepted 2026-03-27 15:19:37,778 - lava_callback - INFO - Starting async job submit for node 69c3fcfebbc022e792645a13 2026-03-27 15:19:37,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:19:37,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fcfebbc022e792645a13 HTTP/1.1" 200 None 2026-03-27 15:19:37,794 - lava_callback - DEBUG - Processing node 69c3fcfebbc022e792645a13: baseline-arm64-clabbe 2026-03-27 15:19:37,795 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fcfebbc022e792645a13 2026-03-27 15:19:37,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:19:37,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fcfebbc022e792645a13 HTTP/1.1" 200 None 2026-03-27 15:19:37,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:19:37,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:19:37,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:19:37,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fcfebbc022e792645a13 HTTP/1.1" 200 None 2026-03-27 15:19:37,863 - lava_callback - ERROR - Error processing callback for node 69c3fcfebbc022e792645a13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48854 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:20:57,831 - lava_callback - INFO - Received callback for node 69c3fd52bbc022e792645a21 INFO: 172.20.0.1:48858 - "POST /node/69c3fd52bbc022e792645a21 HTTP/1.0" 202 Accepted 2026-03-27 15:20:57,837 - lava_callback - INFO - Starting async job submit for node 69c3fd52bbc022e792645a21 2026-03-27 15:20:57,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:20:57,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fd52bbc022e792645a21 HTTP/1.1" 200 None 2026-03-27 15:20:57,852 - lava_callback - DEBUG - Processing node 69c3fd52bbc022e792645a21: baseline-arm64-clabbe 2026-03-27 15:20:57,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fd52bbc022e792645a21 2026-03-27 15:20:57,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:20:57,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fd52bbc022e792645a21 HTTP/1.1" 200 None 2026-03-27 15:20:57,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:20:57,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:20:57,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:20:57,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fd52bbc022e792645a21 HTTP/1.1" 200 None 2026-03-27 15:20:57,903 - lava_callback - ERROR - Error processing callback for node 69c3fd52bbc022e792645a21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55848 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:22:17,906 - lava_callback - INFO - Received callback for node 69c3fda6bbc022e792645a2b INFO: 172.20.0.1:55862 - "POST /node/69c3fda6bbc022e792645a2b HTTP/1.0" 202 Accepted 2026-03-27 15:22:17,913 - lava_callback - INFO - Starting async job submit for node 69c3fda6bbc022e792645a2b 2026-03-27 15:22:17,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:22:17,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fda6bbc022e792645a2b HTTP/1.1" 200 None 2026-03-27 15:22:17,932 - lava_callback - DEBUG - Processing node 69c3fda6bbc022e792645a2b: baseline-arm64-clabbe 2026-03-27 15:22:17,932 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fda6bbc022e792645a2b 2026-03-27 15:22:17,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:22:17,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fda6bbc022e792645a2b HTTP/1.1" 200 None 2026-03-27 15:22:17,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:22:17,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:22:17,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:22:17,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fda6bbc022e792645a2b HTTP/1.1" 200 None 2026-03-27 15:22:17,984 - lava_callback - ERROR - Error processing callback for node 69c3fda6bbc022e792645a2b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42784 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:23:58,017 - lava_callback - INFO - Received callback for node 69c3fdfbbbc022e792645a34 INFO: 172.20.0.1:42796 - "POST /node/69c3fdfbbbc022e792645a34 HTTP/1.0" 202 Accepted 2026-03-27 15:23:58,022 - lava_callback - INFO - Starting async job submit for node 69c3fdfbbbc022e792645a34 2026-03-27 15:23:58,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:23:58,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fdfbbbc022e792645a34 HTTP/1.1" 200 None 2026-03-27 15:23:58,039 - lava_callback - DEBUG - Processing node 69c3fdfbbbc022e792645a34: baseline-arm64-clabbe 2026-03-27 15:23:58,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fdfbbbc022e792645a34 2026-03-27 15:23:58,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:23:58,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fdfbbbc022e792645a34 HTTP/1.1" 200 None 2026-03-27 15:23:58,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:23:58,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:23:58,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:23:58,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fdfbbbc022e792645a34 HTTP/1.1" 200 None 2026-03-27 15:23:58,098 - lava_callback - ERROR - Error processing callback for node 69c3fdfbbbc022e792645a34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44420 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:25:18,008 - lava_callback - INFO - Received callback for node 69c3fe52bbc022e792645a3f INFO: 172.20.0.1:44430 - "POST /node/69c3fe52bbc022e792645a3f HTTP/1.0" 202 Accepted 2026-03-27 15:25:18,014 - lava_callback - INFO - Starting async job submit for node 69c3fe52bbc022e792645a3f 2026-03-27 15:25:18,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:25:18,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fe52bbc022e792645a3f HTTP/1.1" 200 None 2026-03-27 15:25:18,051 - lava_callback - DEBUG - Processing node 69c3fe52bbc022e792645a3f: baseline-arm64-clabbe 2026-03-27 15:25:18,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fe52bbc022e792645a3f 2026-03-27 15:25:18,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:25:18,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fe52bbc022e792645a3f HTTP/1.1" 200 None 2026-03-27 15:25:18,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:25:18,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:25:18,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:25:18,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fe52bbc022e792645a3f HTTP/1.1" 200 None 2026-03-27 15:25:18,125 - lava_callback - ERROR - Error processing callback for node 69c3fe52bbc022e792645a3f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36634 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:26:38,109 - lava_callback - INFO - Received callback for node 69c3fea7bbc022e792645a4a INFO: 172.20.0.1:36638 - "POST /node/69c3fea7bbc022e792645a4a HTTP/1.0" 202 Accepted 2026-03-27 15:26:38,115 - lava_callback - INFO - Starting async job submit for node 69c3fea7bbc022e792645a4a 2026-03-27 15:26:38,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:26:38,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fea7bbc022e792645a4a HTTP/1.1" 200 None 2026-03-27 15:26:38,131 - lava_callback - DEBUG - Processing node 69c3fea7bbc022e792645a4a: baseline-arm64-clabbe 2026-03-27 15:26:38,131 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3fea7bbc022e792645a4a 2026-03-27 15:26:38,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:26:38,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3fea7bbc022e792645a4a HTTP/1.1" 200 None 2026-03-27 15:26:38,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:26:38,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-27 15:26:38,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:26:38,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3fea7bbc022e792645a4a HTTP/1.1" 200 None 2026-03-27 15:26:38,207 - lava_callback - ERROR - Error processing callback for node 69c3fea7bbc022e792645a4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43982 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:29:18,299 - lava_callback - INFO - Received callback for node 69c3ff42bbc022e792645a66 INFO: 172.20.0.1:43996 - "POST /node/69c3ff42bbc022e792645a66 HTTP/1.0" 202 Accepted 2026-03-27 15:29:18,305 - lava_callback - INFO - Starting async job submit for node 69c3ff42bbc022e792645a66 2026-03-27 15:29:18,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:29:18,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ff42bbc022e792645a66 HTTP/1.1" 200 None 2026-03-27 15:29:18,321 - lava_callback - DEBUG - Processing node 69c3ff42bbc022e792645a66: baseline-arm-clabbe 2026-03-27 15:29:18,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ff42bbc022e792645a66 2026-03-27 15:29:18,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:29:18,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ff42bbc022e792645a66 HTTP/1.1" 200 None 2026-03-27 15:29:18,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:29:18,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:29:18,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:29:18,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ff42bbc022e792645a66 HTTP/1.1" 200 None 2026-03-27 15:29:18,386 - lava_callback - ERROR - Error processing callback for node 69c3ff42bbc022e792645a66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32914 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:29:58,192 - lava_callback - INFO - Received callback for node 69c3ff6cbbc022e792645a78 INFO: 172.20.0.1:32920 - "POST /node/69c3ff6cbbc022e792645a78 HTTP/1.0" 202 Accepted 2026-03-27 15:29:58,198 - lava_callback - INFO - Starting async job submit for node 69c3ff6cbbc022e792645a78 2026-03-27 15:29:58,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:29:58,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ff6cbbc022e792645a78 HTTP/1.1" 200 None 2026-03-27 15:29:58,224 - lava_callback - DEBUG - Processing node 69c3ff6cbbc022e792645a78: baseline-arm-clabbe 2026-03-27 15:29:58,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ff6cbbc022e792645a78 2026-03-27 15:29:58,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:29:58,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ff6cbbc022e792645a78 HTTP/1.1" 200 None 2026-03-27 15:29:58,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:29:58,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:29:58,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:29:58,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ff6cbbc022e792645a78 HTTP/1.1" 200 None 2026-03-27 15:29:58,275 - lava_callback - ERROR - Error processing callback for node 69c3ff6cbbc022e792645a78: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43856 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57338 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:30:35,446 - lava_callback - INFO - Received callback for node 69c3ff92bbc022e792645a82 INFO: 172.20.0.1:36810 - "POST /node/69c3ff92bbc022e792645a82 HTTP/1.0" 202 Accepted 2026-03-27 15:30:35,452 - lava_callback - INFO - Starting async job submit for node 69c3ff92bbc022e792645a82 2026-03-27 15:30:35,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:30:35,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ff92bbc022e792645a82 HTTP/1.1" 200 None 2026-03-27 15:30:35,468 - lava_callback - DEBUG - Processing node 69c3ff92bbc022e792645a82: baseline-arm-baylibre 2026-03-27 15:30:35,469 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ff92bbc022e792645a82 2026-03-27 15:30:35,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:30:35,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ff92bbc022e792645a82 HTTP/1.1" 200 None 2026-03-27 15:30:35,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:30:35,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:30:35,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:30:35,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ff92bbc022e792645a82 HTTP/1.1" 200 None 2026-03-27 15:30:35,531 - lava_callback - ERROR - Error processing callback for node 69c3ff92bbc022e792645a82: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60520 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:31:18,297 - lava_callback - INFO - Received callback for node 69c3ffc3bbc022e792645a96 INFO: 172.20.0.1:60532 - "POST /node/69c3ffc3bbc022e792645a96 HTTP/1.0" 202 Accepted 2026-03-27 15:31:18,302 - lava_callback - INFO - Starting async job submit for node 69c3ffc3bbc022e792645a96 2026-03-27 15:31:18,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:31:18,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ffc3bbc022e792645a96 HTTP/1.1" 200 None 2026-03-27 15:31:18,317 - lava_callback - DEBUG - Processing node 69c3ffc3bbc022e792645a96: baseline-arm-clabbe 2026-03-27 15:31:18,318 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ffc3bbc022e792645a96 2026-03-27 15:31:18,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:31:18,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ffc3bbc022e792645a96 HTTP/1.1" 200 None 2026-03-27 15:31:18,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:31:18,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:31:18,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:31:18,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ffc3bbc022e792645a96 HTTP/1.1" 200 None 2026-03-27 15:31:18,377 - lava_callback - ERROR - Error processing callback for node 69c3ffc3bbc022e792645a96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38620 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:31:55,511 - lava_callback - INFO - Received callback for node 69c3ffe7bbc022e792645a9e INFO: 172.20.0.1:54196 - "POST /node/69c3ffe7bbc022e792645a9e HTTP/1.0" 202 Accepted 2026-03-27 15:31:55,516 - lava_callback - INFO - Starting async job submit for node 69c3ffe7bbc022e792645a9e 2026-03-27 15:31:55,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:31:55,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ffe7bbc022e792645a9e HTTP/1.1" 200 None 2026-03-27 15:31:55,532 - lava_callback - DEBUG - Processing node 69c3ffe7bbc022e792645a9e: baseline-arm-baylibre 2026-03-27 15:31:55,533 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c3ffe7bbc022e792645a9e 2026-03-27 15:31:55,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:31:55,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c3ffe7bbc022e792645a9e HTTP/1.1" 200 None 2026-03-27 15:31:55,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:31:55,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:31:55,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:31:55,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c3ffe7bbc022e792645a9e HTTP/1.1" 200 None 2026-03-27 15:31:55,589 - lava_callback - ERROR - Error processing callback for node 69c3ffe7bbc022e792645a9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44022 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:32:55,494 - lava_callback - INFO - Received callback for node 69c40013bbc022e792645aaa INFO: 172.20.0.1:41322 - "POST /node/69c40013bbc022e792645aaa HTTP/1.0" 202 Accepted 2026-03-27 15:32:55,499 - lava_callback - INFO - Starting async job submit for node 69c40013bbc022e792645aaa 2026-03-27 15:32:55,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:32:55,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40013bbc022e792645aaa HTTP/1.1" 200 None 2026-03-27 15:32:55,546 - lava_callback - DEBUG - Processing node 69c40013bbc022e792645aaa: baseline-arm-baylibre 2026-03-27 15:32:55,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40013bbc022e792645aaa 2026-03-27 15:32:55,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:32:55,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40013bbc022e792645aaa HTTP/1.1" 200 None 2026-03-27 15:32:55,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:32:55,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:32:55,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:32:55,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40013bbc022e792645aaa HTTP/1.1" 200 None 2026-03-27 15:32:55,651 - lava_callback - ERROR - Error processing callback for node 69c40013bbc022e792645aaa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49376 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:33:35,559 - lava_callback - INFO - Received callback for node 69c4003bbbc022e792645ab7 INFO: 172.20.0.1:38814 - "POST /node/69c4003bbbc022e792645ab7 HTTP/1.0" 202 Accepted 2026-03-27 15:33:35,565 - lava_callback - INFO - Starting async job submit for node 69c4003bbbc022e792645ab7 2026-03-27 15:33:35,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:33:35,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4003bbbc022e792645ab7 HTTP/1.1" 200 None 2026-03-27 15:33:35,583 - lava_callback - DEBUG - Processing node 69c4003bbbc022e792645ab7: baseline-arm-baylibre 2026-03-27 15:33:35,583 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4003bbbc022e792645ab7 2026-03-27 15:33:35,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:33:35,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4003bbbc022e792645ab7 HTTP/1.1" 200 None 2026-03-27 15:33:35,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:33:35,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:33:35,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:33:35,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4003bbbc022e792645ab7 HTTP/1.1" 200 None 2026-03-27 15:33:35,635 - lava_callback - ERROR - Error processing callback for node 69c4003bbbc022e792645ab7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35022 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:34:58,447 - lava_callback - INFO - Received callback for node 69c40092bbc022e792645acd INFO: 172.20.0.1:35032 - "POST /node/69c40092bbc022e792645acd HTTP/1.0" 202 Accepted 2026-03-27 15:34:58,453 - lava_callback - INFO - Starting async job submit for node 69c40092bbc022e792645acd 2026-03-27 15:34:58,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:34:58,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40092bbc022e792645acd HTTP/1.1" 200 None 2026-03-27 15:34:58,480 - lava_callback - DEBUG - Processing node 69c40092bbc022e792645acd: baseline-arm-clabbe 2026-03-27 15:34:58,481 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40092bbc022e792645acd 2026-03-27 15:34:58,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:34:58,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40092bbc022e792645acd HTTP/1.1" 200 None 2026-03-27 15:34:58,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:34:58,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:34:58,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:34:58,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40092bbc022e792645acd HTTP/1.1" 200 None 2026-03-27 15:34:58,535 - lava_callback - ERROR - Error processing callback for node 69c40092bbc022e792645acd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49954 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:35:35,678 - lava_callback - INFO - Received callback for node 69c400b7bbc022e792645ad8 INFO: 172.20.0.1:40130 - "POST /node/69c400b7bbc022e792645ad8 HTTP/1.0" 202 Accepted 2026-03-27 15:35:35,684 - lava_callback - INFO - Starting async job submit for node 69c400b7bbc022e792645ad8 2026-03-27 15:35:35,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:35:35,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c400b7bbc022e792645ad8 HTTP/1.1" 200 None 2026-03-27 15:35:35,701 - lava_callback - DEBUG - Processing node 69c400b7bbc022e792645ad8: baseline-arm-baylibre 2026-03-27 15:35:35,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c400b7bbc022e792645ad8 2026-03-27 15:35:35,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:35:35,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c400b7bbc022e792645ad8 HTTP/1.1" 200 None 2026-03-27 15:35:35,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:35:35,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:35:35,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:35:35,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c400b7bbc022e792645ad8 HTTP/1.1" 200 None 2026-03-27 15:35:35,755 - lava_callback - ERROR - Error processing callback for node 69c400b7bbc022e792645ad8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51526 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-27 15:36:15,688 - lava_callback - INFO - Received callback for node 69c400e0bbc022e792645ae5 INFO: 172.20.0.1:51534 - "POST /node/69c400e0bbc022e792645ae5 HTTP/1.0" 202 Accepted 2026-03-27 15:36:15,693 - lava_callback - INFO - Starting async job submit for node 69c400e0bbc022e792645ae5 2026-03-27 15:36:15,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:36:15,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c400e0bbc022e792645ae5 HTTP/1.1" 200 None 2026-03-27 15:36:15,708 - lava_callback - DEBUG - Processing node 69c400e0bbc022e792645ae5: baseline-arm-baylibre 2026-03-27 15:36:15,709 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c400e0bbc022e792645ae5 2026-03-27 15:36:15,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:36:15,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c400e0bbc022e792645ae5 HTTP/1.1" 200 None 2026-03-27 15:36:15,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:36:15,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:36:15,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:36:15,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c400e0bbc022e792645ae5 HTTP/1.1" 200 None 2026-03-27 15:36:15,761 - lava_callback - ERROR - Error processing callback for node 69c400e0bbc022e792645ae5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41682 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:36:58,545 - lava_callback - INFO - Received callback for node 69c4010ebbc022e792645af3 INFO: 172.20.0.1:41690 - "POST /node/69c4010ebbc022e792645af3 HTTP/1.0" 202 Accepted 2026-03-27 15:36:58,551 - lava_callback - INFO - Starting async job submit for node 69c4010ebbc022e792645af3 2026-03-27 15:36:58,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:36:58,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4010ebbc022e792645af3 HTTP/1.1" 200 None 2026-03-27 15:36:58,567 - lava_callback - DEBUG - Processing node 69c4010ebbc022e792645af3: baseline-arm-clabbe 2026-03-27 15:36:58,567 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4010ebbc022e792645af3 2026-03-27 15:36:58,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:36:58,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4010ebbc022e792645af3 HTTP/1.1" 200 None 2026-03-27 15:36:58,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:36:58,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:36:58,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:36:58,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4010ebbc022e792645af3 HTTP/1.1" 200 None 2026-03-27 15:36:58,620 - lava_callback - ERROR - Error processing callback for node 69c4010ebbc022e792645af3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57670 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:37:35,791 - lava_callback - INFO - Received callback for node 69c40136bbc022e792645afb INFO: 172.20.0.1:51700 - "POST /node/69c40136bbc022e792645afb HTTP/1.0" 202 Accepted 2026-03-27 15:37:35,796 - lava_callback - INFO - Starting async job submit for node 69c40136bbc022e792645afb 2026-03-27 15:37:35,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:37:35,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40136bbc022e792645afb HTTP/1.1" 200 None 2026-03-27 15:37:35,813 - lava_callback - DEBUG - Processing node 69c40136bbc022e792645afb: baseline-arm-baylibre 2026-03-27 15:37:35,813 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40136bbc022e792645afb 2026-03-27 15:37:35,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:37:35,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40136bbc022e792645afb HTTP/1.1" 200 None 2026-03-27 15:37:35,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:37:35,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:37:35,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:37:35,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40136bbc022e792645afb HTTP/1.1" 200 None 2026-03-27 15:37:35,875 - lava_callback - ERROR - Error processing callback for node 69c40136bbc022e792645afb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55530 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:38:18,616 - lava_callback - INFO - Received callback for node 69c40162bbc022e792645b06 INFO: 172.20.0.1:55536 - "POST /node/69c40162bbc022e792645b06 HTTP/1.0" 202 Accepted 2026-03-27 15:38:18,622 - lava_callback - INFO - Starting async job submit for node 69c40162bbc022e792645b06 2026-03-27 15:38:18,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:38:18,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40162bbc022e792645b06 HTTP/1.1" 200 None 2026-03-27 15:38:18,638 - lava_callback - DEBUG - Processing node 69c40162bbc022e792645b06: baseline-arm-clabbe 2026-03-27 15:38:18,638 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40162bbc022e792645b06 2026-03-27 15:38:18,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:38:18,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40162bbc022e792645b06 HTTP/1.1" 200 None 2026-03-27 15:38:18,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:38:18,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:38:18,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:38:18,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40162bbc022e792645b06 HTTP/1.1" 200 None 2026-03-27 15:38:18,710 - lava_callback - ERROR - Error processing callback for node 69c40162bbc022e792645b06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41490 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:38:55,871 - lava_callback - INFO - Received callback for node 69c4018abbc022e792645b12 INFO: 172.20.0.1:44484 - "POST /node/69c4018abbc022e792645b12 HTTP/1.0" 202 Accepted 2026-03-27 15:38:55,877 - lava_callback - INFO - Starting async job submit for node 69c4018abbc022e792645b12 2026-03-27 15:38:55,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:38:55,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4018abbc022e792645b12 HTTP/1.1" 200 None 2026-03-27 15:38:55,894 - lava_callback - DEBUG - Processing node 69c4018abbc022e792645b12: baseline-arm-baylibre 2026-03-27 15:38:55,894 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4018abbc022e792645b12 2026-03-27 15:38:55,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:38:55,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4018abbc022e792645b12 HTTP/1.1" 200 None 2026-03-27 15:38:55,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:38:55,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:38:55,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:38:55,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4018abbc022e792645b12 HTTP/1.1" 200 None 2026-03-27 15:38:55,962 - lava_callback - ERROR - Error processing callback for node 69c4018abbc022e792645b12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50778 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:39:38,703 - lava_callback - INFO - Received callback for node 69c401b5bbc022e792645b1e INFO: 172.20.0.1:50782 - "POST /node/69c401b5bbc022e792645b1e HTTP/1.0" 202 Accepted 2026-03-27 15:39:38,709 - lava_callback - INFO - Starting async job submit for node 69c401b5bbc022e792645b1e 2026-03-27 15:39:38,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:39:38,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c401b5bbc022e792645b1e HTTP/1.1" 200 None 2026-03-27 15:39:38,726 - lava_callback - DEBUG - Processing node 69c401b5bbc022e792645b1e: baseline-arm-clabbe 2026-03-27 15:39:38,726 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c401b5bbc022e792645b1e 2026-03-27 15:39:38,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:39:38,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c401b5bbc022e792645b1e HTTP/1.1" 200 None 2026-03-27 15:39:38,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:39:38,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:39:38,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:39:38,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c401b5bbc022e792645b1e HTTP/1.1" 200 None 2026-03-27 15:39:38,782 - lava_callback - ERROR - Error processing callback for node 69c401b5bbc022e792645b1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60602 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:40:18,678 - lava_callback - INFO - Received callback for node 69c401e0bbc022e792645b30 INFO: 172.20.0.1:60612 - "POST /node/69c401e0bbc022e792645b30 HTTP/1.0" 202 Accepted 2026-03-27 15:40:18,684 - lava_callback - INFO - Starting async job submit for node 69c401e0bbc022e792645b30 2026-03-27 15:40:18,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:40:18,698 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c401e0bbc022e792645b30 HTTP/1.1" 200 None 2026-03-27 15:40:18,699 - lava_callback - DEBUG - Processing node 69c401e0bbc022e792645b30: baseline-arm-clabbe 2026-03-27 15:40:18,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c401e0bbc022e792645b30 2026-03-27 15:40:18,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:40:18,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c401e0bbc022e792645b30 HTTP/1.1" 200 None 2026-03-27 15:40:18,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:40:18,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:40:18,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:40:18,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c401e0bbc022e792645b30 HTTP/1.1" 200 None 2026-03-27 15:40:18,761 - lava_callback - ERROR - Error processing callback for node 69c401e0bbc022e792645b30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53846 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:41:18,795 - lava_callback - INFO - Received callback for node 69c4020abbc022e792645b39 INFO: 172.20.0.1:53862 - "POST /node/69c4020abbc022e792645b39 HTTP/1.0" 202 Accepted 2026-03-27 15:41:18,801 - lava_callback - INFO - Starting async job submit for node 69c4020abbc022e792645b39 2026-03-27 15:41:18,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:41:18,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4020abbc022e792645b39 HTTP/1.1" 200 None 2026-03-27 15:41:18,817 - lava_callback - DEBUG - Processing node 69c4020abbc022e792645b39: baseline-arm-clabbe 2026-03-27 15:41:18,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4020abbc022e792645b39 2026-03-27 15:41:18,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:41:18,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4020abbc022e792645b39 HTTP/1.1" 200 None 2026-03-27 15:41:18,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:41:18,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:41:18,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:41:18,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4020abbc022e792645b39 HTTP/1.1" 200 None 2026-03-27 15:41:18,873 - lava_callback - ERROR - Error processing callback for node 69c4020abbc022e792645b39: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33330 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:41:58,761 - lava_callback - INFO - Received callback for node 69c40235bbc022e792645b45 INFO: 172.20.0.1:33342 - "POST /node/69c40235bbc022e792645b45 HTTP/1.0" 202 Accepted 2026-03-27 15:41:58,767 - lava_callback - INFO - Starting async job submit for node 69c40235bbc022e792645b45 2026-03-27 15:41:58,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:41:58,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40235bbc022e792645b45 HTTP/1.1" 200 None 2026-03-27 15:41:58,783 - lava_callback - DEBUG - Processing node 69c40235bbc022e792645b45: baseline-arm-clabbe 2026-03-27 15:41:58,783 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40235bbc022e792645b45 2026-03-27 15:41:58,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:41:58,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40235bbc022e792645b45 HTTP/1.1" 200 None 2026-03-27 15:41:58,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:41:58,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:41:58,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:41:58,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40235bbc022e792645b45 HTTP/1.1" 200 None 2026-03-27 15:41:58,840 - lava_callback - ERROR - Error processing callback for node 69c40235bbc022e792645b45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40244 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:42:38,871 - lava_callback - INFO - Received callback for node 69c40261bbc022e792645b50 INFO: 172.20.0.1:40246 - "POST /node/69c40261bbc022e792645b50 HTTP/1.0" 202 Accepted 2026-03-27 15:42:38,877 - lava_callback - INFO - Starting async job submit for node 69c40261bbc022e792645b50 2026-03-27 15:42:38,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:42:38,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40261bbc022e792645b50 HTTP/1.1" 200 None 2026-03-27 15:42:38,892 - lava_callback - DEBUG - Processing node 69c40261bbc022e792645b50: baseline-arm-clabbe 2026-03-27 15:42:38,892 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40261bbc022e792645b50 2026-03-27 15:42:38,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:42:38,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40261bbc022e792645b50 HTTP/1.1" 200 None 2026-03-27 15:42:38,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:42:38,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:42:38,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:42:38,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40261bbc022e792645b50 HTTP/1.1" 200 None 2026-03-27 15:42:38,941 - lava_callback - ERROR - Error processing callback for node 69c40261bbc022e792645b50: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38044 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:43:18,834 - lava_callback - INFO - Received callback for node 69c4028cbbc022e792645b5d INFO: 172.20.0.1:38052 - "POST /node/69c4028cbbc022e792645b5d HTTP/1.0" 202 Accepted 2026-03-27 15:43:18,839 - lava_callback - INFO - Starting async job submit for node 69c4028cbbc022e792645b5d 2026-03-27 15:43:18,840 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:43:18,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4028cbbc022e792645b5d HTTP/1.1" 200 None 2026-03-27 15:43:18,864 - lava_callback - DEBUG - Processing node 69c4028cbbc022e792645b5d: baseline-arm-clabbe 2026-03-27 15:43:18,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4028cbbc022e792645b5d 2026-03-27 15:43:18,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:43:18,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4028cbbc022e792645b5d HTTP/1.1" 200 None 2026-03-27 15:43:18,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:43:18,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:43:18,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:43:18,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4028cbbc022e792645b5d HTTP/1.1" 200 None 2026-03-27 15:43:18,924 - lava_callback - ERROR - Error processing callback for node 69c4028cbbc022e792645b5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33384 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:43:58,845 - lava_callback - INFO - Received callback for node 69c402b7bbc022e792645b69 INFO: 172.20.0.1:33392 - "POST /node/69c402b7bbc022e792645b69 HTTP/1.0" 202 Accepted 2026-03-27 15:43:58,851 - lava_callback - INFO - Starting async job submit for node 69c402b7bbc022e792645b69 2026-03-27 15:43:58,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:43:58,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c402b7bbc022e792645b69 HTTP/1.1" 200 None 2026-03-27 15:43:58,876 - lava_callback - DEBUG - Processing node 69c402b7bbc022e792645b69: baseline-arm-clabbe 2026-03-27 15:43:58,876 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c402b7bbc022e792645b69 2026-03-27 15:43:58,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:43:58,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c402b7bbc022e792645b69 HTTP/1.1" 200 None 2026-03-27 15:43:58,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:43:58,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:43:58,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:43:58,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c402b7bbc022e792645b69 HTTP/1.1" 200 None 2026-03-27 15:43:58,948 - lava_callback - ERROR - Error processing callback for node 69c402b7bbc022e792645b69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42718 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:44:38,842 - lava_callback - INFO - Received callback for node 69c402e1bbc022e792645b73 INFO: 172.20.0.1:42730 - "POST /node/69c402e1bbc022e792645b73 HTTP/1.0" 202 Accepted 2026-03-27 15:44:38,847 - lava_callback - INFO - Starting async job submit for node 69c402e1bbc022e792645b73 2026-03-27 15:44:38,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:44:38,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c402e1bbc022e792645b73 HTTP/1.1" 200 None 2026-03-27 15:44:38,873 - lava_callback - DEBUG - Processing node 69c402e1bbc022e792645b73: baseline-arm-clabbe 2026-03-27 15:44:38,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c402e1bbc022e792645b73 2026-03-27 15:44:38,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:44:38,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c402e1bbc022e792645b73 HTTP/1.1" 200 None 2026-03-27 15:44:38,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:44:38,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:44:38,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:44:38,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c402e1bbc022e792645b73 HTTP/1.1" 200 None 2026-03-27 15:44:38,929 - lava_callback - ERROR - Error processing callback for node 69c402e1bbc022e792645b73: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55410 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:55146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56760 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:46:58,994 - lava_callback - INFO - Received callback for node 69c40360bbc022e792645b87 INFO: 172.20.0.1:56764 - "POST /node/69c40360bbc022e792645b87 HTTP/1.0" 202 Accepted 2026-03-27 15:46:59,002 - lava_callback - INFO - Starting async job submit for node 69c40360bbc022e792645b87 2026-03-27 15:46:59,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:46:59,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40360bbc022e792645b87 HTTP/1.1" 200 None 2026-03-27 15:46:59,025 - lava_callback - DEBUG - Processing node 69c40360bbc022e792645b87: baseline-arm-clabbe 2026-03-27 15:46:59,026 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40360bbc022e792645b87 2026-03-27 15:46:59,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:46:59,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40360bbc022e792645b87 HTTP/1.1" 200 None 2026-03-27 15:46:59,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:46:59,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:46:59,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:46:59,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40360bbc022e792645b87 HTTP/1.1" 200 None 2026-03-27 15:46:59,097 - lava_callback - ERROR - Error processing callback for node 69c40360bbc022e792645b87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39284 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:47:38,972 - lava_callback - INFO - Received callback for node 69c4038abbc022e792645b95 INFO: 172.20.0.1:39288 - "POST /node/69c4038abbc022e792645b95 HTTP/1.0" 202 Accepted 2026-03-27 15:47:38,977 - lava_callback - INFO - Starting async job submit for node 69c4038abbc022e792645b95 2026-03-27 15:47:38,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:47:38,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4038abbc022e792645b95 HTTP/1.1" 200 None 2026-03-27 15:47:38,995 - lava_callback - DEBUG - Processing node 69c4038abbc022e792645b95: baseline-arm-clabbe 2026-03-27 15:47:38,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4038abbc022e792645b95 2026-03-27 15:47:38,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:47:39,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4038abbc022e792645b95 HTTP/1.1" 200 None 2026-03-27 15:47:39,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:47:39,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:47:39,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:47:39,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4038abbc022e792645b95 HTTP/1.1" 200 None 2026-03-27 15:47:39,066 - lava_callback - ERROR - Error processing callback for node 69c4038abbc022e792645b95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46692 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:48:19,030 - lava_callback - INFO - Received callback for node 69c403b3bbc022e792645b9e INFO: 172.20.0.1:46706 - "POST /node/69c403b3bbc022e792645b9e HTTP/1.0" 202 Accepted 2026-03-27 15:48:19,035 - lava_callback - INFO - Starting async job submit for node 69c403b3bbc022e792645b9e 2026-03-27 15:48:19,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:48:19,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c403b3bbc022e792645b9e HTTP/1.1" 200 None 2026-03-27 15:48:19,051 - lava_callback - DEBUG - Processing node 69c403b3bbc022e792645b9e: baseline-arm-clabbe 2026-03-27 15:48:19,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c403b3bbc022e792645b9e 2026-03-27 15:48:19,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:48:19,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c403b3bbc022e792645b9e HTTP/1.1" 200 None 2026-03-27 15:48:19,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:48:19,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f72facdb2e26c83cca20 HTTP/1.1" 200 None 2026-03-27 15:48:19,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:48:19,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c403b3bbc022e792645b9e HTTP/1.1" 200 None 2026-03-27 15:48:19,112 - lava_callback - ERROR - Error processing callback for node 69c403b3bbc022e792645b9e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33322 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:48:59,054 - lava_callback - INFO - Received callback for node 69c403debbc022e792645bab INFO: 172.20.0.1:33332 - "POST /node/69c403debbc022e792645bab HTTP/1.0" 202 Accepted 2026-03-27 15:48:59,059 - lava_callback - INFO - Starting async job submit for node 69c403debbc022e792645bab 2026-03-27 15:48:59,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:48:59,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c403debbc022e792645bab HTTP/1.1" 200 None 2026-03-27 15:48:59,075 - lava_callback - DEBUG - Processing node 69c403debbc022e792645bab: baseline-arm-clabbe 2026-03-27 15:48:59,075 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c403debbc022e792645bab 2026-03-27 15:48:59,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:48:59,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c403debbc022e792645bab HTTP/1.1" 200 None 2026-03-27 15:48:59,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:48:59,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:48:59,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:48:59,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c403debbc022e792645bab HTTP/1.1" 200 None 2026-03-27 15:48:59,135 - lava_callback - ERROR - Error processing callback for node 69c403debbc022e792645bab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36284 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:49:39,150 - lava_callback - INFO - Received callback for node 69c40408bbc022e792645bb4 INFO: 172.20.0.1:36298 - "POST /node/69c40408bbc022e792645bb4 HTTP/1.0" 202 Accepted 2026-03-27 15:49:39,156 - lava_callback - INFO - Starting async job submit for node 69c40408bbc022e792645bb4 2026-03-27 15:49:39,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:49:39,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40408bbc022e792645bb4 HTTP/1.1" 200 None 2026-03-27 15:49:39,187 - lava_callback - DEBUG - Processing node 69c40408bbc022e792645bb4: baseline-arm-clabbe 2026-03-27 15:49:39,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40408bbc022e792645bb4 2026-03-27 15:49:39,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:49:39,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40408bbc022e792645bb4 HTTP/1.1" 200 None 2026-03-27 15:49:39,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:49:39,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:49:39,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:49:39,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40408bbc022e792645bb4 HTTP/1.1" 200 None 2026-03-27 15:49:39,245 - lava_callback - ERROR - Error processing callback for node 69c40408bbc022e792645bb4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47616 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:50:19,110 - lava_callback - INFO - Received callback for node 69c40431bbc022e792645bbd INFO: 172.20.0.1:47618 - "POST /node/69c40431bbc022e792645bbd HTTP/1.0" 202 Accepted 2026-03-27 15:50:19,115 - lava_callback - INFO - Starting async job submit for node 69c40431bbc022e792645bbd 2026-03-27 15:50:19,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:50:19,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40431bbc022e792645bbd HTTP/1.1" 200 None 2026-03-27 15:50:19,131 - lava_callback - DEBUG - Processing node 69c40431bbc022e792645bbd: baseline-arm-clabbe 2026-03-27 15:50:19,132 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40431bbc022e792645bbd 2026-03-27 15:50:19,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:50:19,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40431bbc022e792645bbd HTTP/1.1" 200 None 2026-03-27 15:50:19,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:50:19,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69be544f448e074b3ffe162d HTTP/1.1" 200 None 2026-03-27 15:50:19,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:50:19,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40431bbc022e792645bbd HTTP/1.1" 200 None 2026-03-27 15:50:19,191 - lava_callback - ERROR - Error processing callback for node 69c40431bbc022e792645bbd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50972 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:51:16,668 - lava_callback - INFO - Received callback for node 69c40461bbc022e792645bc7 INFO: 172.20.0.1:50980 - "POST /node/69c40461bbc022e792645bc7 HTTP/1.0" 202 Accepted 2026-03-27 15:51:16,674 - lava_callback - INFO - Starting async job submit for node 69c40461bbc022e792645bc7 2026-03-27 15:51:16,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:51:16,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40461bbc022e792645bc7 HTTP/1.1" 200 None 2026-03-27 15:51:16,690 - lava_callback - DEBUG - Processing node 69c40461bbc022e792645bc7: baseline-arm64-baylibre 2026-03-27 15:51:16,690 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40461bbc022e792645bc7 2026-03-27 15:51:16,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:51:16,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40461bbc022e792645bc7 HTTP/1.1" 200 None 2026-03-27 15:51:16,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:51:16,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 15:51:16,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:51:16,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40461bbc022e792645bc7 HTTP/1.1" 200 None 2026-03-27 15:51:16,743 - lava_callback - ERROR - Error processing callback for node 69c40461bbc022e792645bc7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49982 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:52:36,843 - lava_callback - INFO - Received callback for node 69c404b7bbc022e792645bd4 INFO: 172.20.0.1:49994 - "POST /node/69c404b7bbc022e792645bd4 HTTP/1.0" 202 Accepted 2026-03-27 15:52:36,849 - lava_callback - INFO - Starting async job submit for node 69c404b7bbc022e792645bd4 2026-03-27 15:52:36,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:52:36,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c404b7bbc022e792645bd4 HTTP/1.1" 200 None 2026-03-27 15:52:36,865 - lava_callback - DEBUG - Processing node 69c404b7bbc022e792645bd4: baseline-arm64-baylibre 2026-03-27 15:52:36,865 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c404b7bbc022e792645bd4 2026-03-27 15:52:36,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:52:36,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c404b7bbc022e792645bd4 HTTP/1.1" 200 None 2026-03-27 15:52:36,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:52:36,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 15:52:36,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:52:36,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c404b7bbc022e792645bd4 HTTP/1.1" 200 None 2026-03-27 15:52:36,919 - lava_callback - ERROR - Error processing callback for node 69c404b7bbc022e792645bd4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52042 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:53:56,738 - lava_callback - INFO - Received callback for node 69c4050ebbc022e792645be4 INFO: 172.20.0.1:52044 - "POST /node/69c4050ebbc022e792645be4 HTTP/1.0" 202 Accepted 2026-03-27 15:53:56,744 - lava_callback - INFO - Starting async job submit for node 69c4050ebbc022e792645be4 2026-03-27 15:53:56,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:53:56,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4050ebbc022e792645be4 HTTP/1.1" 200 None 2026-03-27 15:53:56,760 - lava_callback - DEBUG - Processing node 69c4050ebbc022e792645be4: baseline-arm64-baylibre 2026-03-27 15:53:56,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4050ebbc022e792645be4 2026-03-27 15:53:56,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:53:56,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4050ebbc022e792645be4 HTTP/1.1" 200 None 2026-03-27 15:53:56,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:53:56,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 15:53:56,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:53:56,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4050ebbc022e792645be4 HTTP/1.1" 200 None 2026-03-27 15:53:56,817 - lava_callback - ERROR - Error processing callback for node 69c4050ebbc022e792645be4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55656 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:55:36,794 - lava_callback - INFO - Received callback for node 69c40564bbc022e792645bfa INFO: 172.20.0.1:55670 - "POST /node/69c40564bbc022e792645bfa HTTP/1.0" 202 Accepted 2026-03-27 15:55:36,800 - lava_callback - INFO - Starting async job submit for node 69c40564bbc022e792645bfa 2026-03-27 15:55:36,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:55:36,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40564bbc022e792645bfa HTTP/1.1" 200 None 2026-03-27 15:55:36,829 - lava_callback - DEBUG - Processing node 69c40564bbc022e792645bfa: baseline-arm64-baylibre 2026-03-27 15:55:36,829 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40564bbc022e792645bfa 2026-03-27 15:55:36,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:55:36,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40564bbc022e792645bfa HTTP/1.1" 200 None 2026-03-27 15:55:36,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:55:36,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 15:55:36,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:55:36,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40564bbc022e792645bfa HTTP/1.1" 200 None 2026-03-27 15:55:36,919 - lava_callback - ERROR - Error processing callback for node 69c40564bbc022e792645bfa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47628 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:56:56,888 - lava_callback - INFO - Received callback for node 69c405b9bbc022e792645c07 INFO: 172.20.0.1:47630 - "POST /node/69c405b9bbc022e792645c07 HTTP/1.0" 202 Accepted 2026-03-27 15:56:56,894 - lava_callback - INFO - Starting async job submit for node 69c405b9bbc022e792645c07 2026-03-27 15:56:56,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:56:56,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c405b9bbc022e792645c07 HTTP/1.1" 200 None 2026-03-27 15:56:56,910 - lava_callback - DEBUG - Processing node 69c405b9bbc022e792645c07: baseline-arm64-baylibre 2026-03-27 15:56:56,910 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c405b9bbc022e792645c07 2026-03-27 15:56:56,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:56:56,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c405b9bbc022e792645c07 HTTP/1.1" 200 None 2026-03-27 15:56:56,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:56:56,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 15:56:56,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:56:56,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c405b9bbc022e792645c07 HTTP/1.1" 200 None 2026-03-27 15:56:56,968 - lava_callback - ERROR - Error processing callback for node 69c405b9bbc022e792645c07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46066 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:58:17,048 - lava_callback - INFO - Received callback for node 69c4060dbbc022e792645c1c INFO: 172.20.0.1:46072 - "POST /node/69c4060dbbc022e792645c1c HTTP/1.0" 202 Accepted 2026-03-27 15:58:17,054 - lava_callback - INFO - Starting async job submit for node 69c4060dbbc022e792645c1c 2026-03-27 15:58:17,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:58:17,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4060dbbc022e792645c1c HTTP/1.1" 200 None 2026-03-27 15:58:17,070 - lava_callback - DEBUG - Processing node 69c4060dbbc022e792645c1c: baseline-arm64-baylibre 2026-03-27 15:58:17,070 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4060dbbc022e792645c1c 2026-03-27 15:58:17,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:58:17,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4060dbbc022e792645c1c HTTP/1.1" 200 None 2026-03-27 15:58:17,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:58:17,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 15:58:17,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:58:17,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4060dbbc022e792645c1c HTTP/1.1" 200 None 2026-03-27 15:58:17,121 - lava_callback - ERROR - Error processing callback for node 69c4060dbbc022e792645c1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56560 - "GET / HTTP/1.0" 200 OK 2026-03-27 15:59:59,727 - lava_callback - INFO - Received callback for node 69c4066ebbc022e792645c2f INFO: 172.20.0.1:48712 - "POST /node/69c4066ebbc022e792645c2f HTTP/1.0" 202 Accepted 2026-03-27 15:59:59,733 - lava_callback - INFO - Starting async job submit for node 69c4066ebbc022e792645c2f 2026-03-27 15:59:59,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:59:59,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4066ebbc022e792645c2f HTTP/1.1" 200 None 2026-03-27 15:59:59,749 - lava_callback - DEBUG - Processing node 69c4066ebbc022e792645c2f: baseline-arm64-clabbe 2026-03-27 15:59:59,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4066ebbc022e792645c2f 2026-03-27 15:59:59,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:59:59,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4066ebbc022e792645c2f HTTP/1.1" 200 None 2026-03-27 15:59:59,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:59:59,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 15:59:59,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 15:59:59,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4066ebbc022e792645c2f HTTP/1.1" 200 None 2026-03-27 15:59:59,801 - lava_callback - ERROR - Error processing callback for node 69c4066ebbc022e792645c2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58930 - "GET / HTTP/1.0" 200 OK 2026-03-27 16:01:17,105 - lava_callback - INFO - Received callback for node 69c406bbbbc022e792645c45 INFO: 172.20.0.1:58938 - "POST /node/69c406bbbbc022e792645c45 HTTP/1.0" 202 Accepted 2026-03-27 16:01:17,111 - lava_callback - INFO - Starting async job submit for node 69c406bbbbc022e792645c45 2026-03-27 16:01:17,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:01:17,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c406bbbbc022e792645c45 HTTP/1.1" 200 None 2026-03-27 16:01:17,127 - lava_callback - DEBUG - Processing node 69c406bbbbc022e792645c45: baseline-arm64-baylibre 2026-03-27 16:01:17,127 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c406bbbbc022e792645c45 2026-03-27 16:01:17,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:01:17,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c406bbbbc022e792645c45 HTTP/1.1" 200 None 2026-03-27 16:01:17,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:01:17,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 16:01:17,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:01:17,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c406bbbbc022e792645c45 HTTP/1.1" 200 None 2026-03-27 16:01:17,197 - lava_callback - ERROR - Error processing callback for node 69c406bbbbc022e792645c45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41294 - "GET / HTTP/1.0" 200 OK 2026-03-27 16:04:19,755 - lava_callback - INFO - Received callback for node 69c4076ebbc022e792645c6a INFO: 172.20.0.1:36246 - "POST /node/69c4076ebbc022e792645c6a HTTP/1.0" 202 Accepted 2026-03-27 16:04:19,760 - lava_callback - INFO - Starting async job submit for node 69c4076ebbc022e792645c6a 2026-03-27 16:04:19,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:04:19,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4076ebbc022e792645c6a HTTP/1.1" 200 None 2026-03-27 16:04:19,777 - lava_callback - DEBUG - Processing node 69c4076ebbc022e792645c6a: baseline-arm64-clabbe 2026-03-27 16:04:19,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4076ebbc022e792645c6a 2026-03-27 16:04:19,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:04:19,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4076ebbc022e792645c6a HTTP/1.1" 200 None 2026-03-27 16:04:19,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:04:19,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 16:04:19,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:04:19,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4076ebbc022e792645c6a HTTP/1.1" 200 None 2026-03-27 16:04:19,845 - lava_callback - ERROR - Error processing callback for node 69c4076ebbc022e792645c6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58910 - "GET / HTTP/1.0" 200 OK 2026-03-27 16:05:17,310 - lava_callback - INFO - Received callback for node 69c407babbc022e792645c7f INFO: 172.20.0.1:58922 - "POST /node/69c407babbc022e792645c7f HTTP/1.0" 202 Accepted 2026-03-27 16:05:17,316 - lava_callback - INFO - Starting async job submit for node 69c407babbc022e792645c7f 2026-03-27 16:05:17,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:05:17,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c407babbc022e792645c7f HTTP/1.1" 200 None 2026-03-27 16:05:17,332 - lava_callback - DEBUG - Processing node 69c407babbc022e792645c7f: baseline-arm64-baylibre 2026-03-27 16:05:17,332 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c407babbc022e792645c7f 2026-03-27 16:05:17,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:05:17,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c407babbc022e792645c7f HTTP/1.1" 200 None 2026-03-27 16:05:17,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:05:17,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 16:05:17,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:05:17,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c407babbc022e792645c7f HTTP/1.1" 200 None 2026-03-27 16:05:17,401 - lava_callback - ERROR - Error processing callback for node 69c407babbc022e792645c7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34980 - "GET / HTTP/1.0" 200 OK 2026-03-27 16:07:00,030 - lava_callback - INFO - Received callback for node 69c40818bbc022e792645c96 INFO: 172.20.0.1:33792 - "POST /node/69c40818bbc022e792645c96 HTTP/1.0" 202 Accepted 2026-03-27 16:07:00,036 - lava_callback - INFO - Starting async job submit for node 69c40818bbc022e792645c96 2026-03-27 16:07:00,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:07:00,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40818bbc022e792645c96 HTTP/1.1" 200 None 2026-03-27 16:07:00,052 - lava_callback - DEBUG - Processing node 69c40818bbc022e792645c96: baseline-arm64-clabbe 2026-03-27 16:07:00,053 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40818bbc022e792645c96 2026-03-27 16:07:00,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:07:00,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40818bbc022e792645c96 HTTP/1.1" 200 None 2026-03-27 16:07:00,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:07:00,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 16:07:00,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:07:00,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40818bbc022e792645c96 HTTP/1.1" 200 None 2026-03-27 16:07:00,108 - lava_callback - ERROR - Error processing callback for node 69c40818bbc022e792645c96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52146 - "GET / HTTP/1.0" 200 OK 2026-03-27 16:08:19,875 - lava_callback - INFO - Received callback for node 69c4086ebbc022e792645ca3 INFO: 172.20.0.1:32980 - "POST /node/69c4086ebbc022e792645ca3 HTTP/1.0" 202 Accepted 2026-03-27 16:08:19,881 - lava_callback - INFO - Starting async job submit for node 69c4086ebbc022e792645ca3 2026-03-27 16:08:19,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:08:19,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4086ebbc022e792645ca3 HTTP/1.1" 200 None 2026-03-27 16:08:19,896 - lava_callback - DEBUG - Processing node 69c4086ebbc022e792645ca3: baseline-arm64-clabbe 2026-03-27 16:08:19,896 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4086ebbc022e792645ca3 2026-03-27 16:08:19,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:08:19,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4086ebbc022e792645ca3 HTTP/1.1" 200 None 2026-03-27 16:08:19,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:08:19,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 16:08:19,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:08:19,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4086ebbc022e792645ca3 HTTP/1.1" 200 None 2026-03-27 16:08:19,956 - lava_callback - ERROR - Error processing callback for node 69c4086ebbc022e792645ca3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35948 - "GET / HTTP/1.0" 200 OK 2026-03-27 16:09:37,567 - lava_callback - INFO - Received callback for node 69c408babbc022e792645cb8 INFO: 172.20.0.1:35954 - "POST /node/69c408babbc022e792645cb8 HTTP/1.0" 202 Accepted 2026-03-27 16:09:37,573 - lava_callback - INFO - Starting async job submit for node 69c408babbc022e792645cb8 2026-03-27 16:09:37,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:09:37,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c408babbc022e792645cb8 HTTP/1.1" 200 None 2026-03-27 16:09:37,590 - lava_callback - DEBUG - Processing node 69c408babbc022e792645cb8: baseline-arm64-baylibre 2026-03-27 16:09:37,590 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c408babbc022e792645cb8 2026-03-27 16:09:37,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:09:37,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c408babbc022e792645cb8 HTTP/1.1" 200 None 2026-03-27 16:09:37,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:09:37,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 16:09:37,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:09:37,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c408babbc022e792645cb8 HTTP/1.1" 200 None 2026-03-27 16:09:37,674 - lava_callback - ERROR - Error processing callback for node 69c408babbc022e792645cb8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36230 - "GET / HTTP/1.0" 200 OK 2026-03-27 16:11:20,013 - lava_callback - INFO - Received callback for node 69c40917bbc022e792645ccf INFO: 172.20.0.1:58774 - "POST /node/69c40917bbc022e792645ccf HTTP/1.0" 202 Accepted 2026-03-27 16:11:20,020 - lava_callback - INFO - Starting async job submit for node 69c40917bbc022e792645ccf 2026-03-27 16:11:20,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:11:20,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40917bbc022e792645ccf HTTP/1.1" 200 None 2026-03-27 16:11:20,040 - lava_callback - DEBUG - Processing node 69c40917bbc022e792645ccf: baseline-arm64-clabbe 2026-03-27 16:11:20,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40917bbc022e792645ccf 2026-03-27 16:11:20,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:11:20,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40917bbc022e792645ccf HTTP/1.1" 200 None 2026-03-27 16:11:20,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:11:20,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 16:11:20,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:11:20,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40917bbc022e792645ccf HTTP/1.1" 200 None 2026-03-27 16:11:20,107 - lava_callback - ERROR - Error processing callback for node 69c40917bbc022e792645ccf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57968 - "GET / HTTP/1.0" 200 OK 2026-03-27 16:12:37,709 - lava_callback - INFO - Received callback for node 69c40965bbc022e792645ce4 INFO: 172.20.0.1:57970 - "POST /node/69c40965bbc022e792645ce4 HTTP/1.0" 202 Accepted 2026-03-27 16:12:37,715 - lava_callback - INFO - Starting async job submit for node 69c40965bbc022e792645ce4 2026-03-27 16:12:37,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:12:37,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40965bbc022e792645ce4 HTTP/1.1" 200 None 2026-03-27 16:12:37,731 - lava_callback - DEBUG - Processing node 69c40965bbc022e792645ce4: baseline-arm64-baylibre 2026-03-27 16:12:37,731 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c40965bbc022e792645ce4 2026-03-27 16:12:37,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:12:37,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40965bbc022e792645ce4 HTTP/1.1" 200 None 2026-03-27 16:12:37,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:12:37,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c0f73bacdb2e26c83cca2f HTTP/1.1" 200 None 2026-03-27 16:12:37,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-27 16:12:37,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c40965bbc022e792645ce4 HTTP/1.1" 200 None 2026-03-27 16:12:37,783 - lava_callback - ERROR - Error processing callback for node 69c40965bbc022e792645ce4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57742 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]