INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44840 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:14:10,505 - lava_callback - INFO - Received callback for node 69cc63468df9359ff489a793 2026-04-03 00:14:10,511 - lava_callback - INFO - Starting async job submit for node 69cc63468df9359ff489a793 INFO: 172.21.0.1:56330 - "POST /node/69cc63468df9359ff489a793 HTTP/1.0" 202 Accepted 2026-04-03 00:14:10,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:14:10,532 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc63468df9359ff489a793 HTTP/1.1" 200 None 2026-04-03 00:14:10,532 - lava_callback - DEBUG - Processing node 69cc63468df9359ff489a793: baseline-arm64-clabbe 2026-04-03 00:14:10,533 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc63468df9359ff489a793 2026-04-03 00:14:10,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:14:10,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc63468df9359ff489a793 HTTP/1.1" 200 None 2026-04-03 00:14:10,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:14:10,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 00:14:10,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:14:10,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc63468df9359ff489a793 HTTP/1.1" 200 None 2026-04-03 00:14:10,630 - lava_callback - ERROR - Error processing callback for node 69cc63468df9359ff489a793: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54204 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56656 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:17:52,058 - lava_callback - INFO - Received callback for node 69cc641d8cd9cbcdc9459624 INFO: 172.21.0.1:49196 - "POST /node/69cc641d8cd9cbcdc9459624 HTTP/1.0" 202 Accepted 2026-04-03 00:17:52,064 - lava_callback - INFO - Starting async job submit for node 69cc641d8cd9cbcdc9459624 2026-04-03 00:17:52,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:17:52,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc641d8cd9cbcdc9459624 HTTP/1.1" 200 None 2026-04-03 00:17:52,081 - lava_callback - DEBUG - Processing node 69cc641d8cd9cbcdc9459624: baseline-arm-baylibre 2026-04-03 00:17:52,081 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc641d8cd9cbcdc9459624 2026-04-03 00:17:52,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:17:52,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc641d8cd9cbcdc9459624 HTTP/1.1" 200 None 2026-04-03 00:17:52,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:17:52,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:17:52,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:17:52,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc641d8cd9cbcdc9459624 HTTP/1.1" 200 None 2026-04-03 00:17:52,149 - lava_callback - ERROR - Error processing callback for node 69cc641d8cd9cbcdc9459624: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37662 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:18:52,077 - lava_callback - INFO - Received callback for node 69cc645a8cd9cbcdc9459632 INFO: 172.21.0.1:39464 - "POST /node/69cc645a8cd9cbcdc9459632 HTTP/1.0" 202 Accepted 2026-04-03 00:18:52,083 - lava_callback - INFO - Starting async job submit for node 69cc645a8cd9cbcdc9459632 2026-04-03 00:18:52,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:18:52,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc645a8cd9cbcdc9459632 HTTP/1.1" 200 None 2026-04-03 00:18:52,104 - lava_callback - DEBUG - Processing node 69cc645a8cd9cbcdc9459632: baseline-arm-baylibre 2026-04-03 00:18:52,104 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc645a8cd9cbcdc9459632 2026-04-03 00:18:52,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:18:52,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc645a8cd9cbcdc9459632 HTTP/1.1" 200 None 2026-04-03 00:18:52,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:18:52,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:18:52,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:18:52,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc645a8cd9cbcdc9459632 HTTP/1.1" 200 None 2026-04-03 00:18:52,157 - lava_callback - ERROR - Error processing callback for node 69cc645a8cd9cbcdc9459632: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41406 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:20:52,213 - lava_callback - INFO - Received callback for node 69cc64d68cd9cbcdc945968a INFO: 172.21.0.1:52974 - "POST /node/69cc64d68cd9cbcdc945968a HTTP/1.0" 202 Accepted 2026-04-03 00:20:52,218 - lava_callback - INFO - Starting async job submit for node 69cc64d68cd9cbcdc945968a 2026-04-03 00:20:52,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:20:52,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64d68cd9cbcdc945968a HTTP/1.1" 200 None 2026-04-03 00:20:52,234 - lava_callback - DEBUG - Processing node 69cc64d68cd9cbcdc945968a: baseline-arm-baylibre 2026-04-03 00:20:52,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc64d68cd9cbcdc945968a 2026-04-03 00:20:52,235 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:20:52,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc64d68cd9cbcdc945968a HTTP/1.1" 200 None 2026-04-03 00:20:52,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:20:52,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:20:52,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:20:52,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc64d68cd9cbcdc945968a HTTP/1.1" 200 None 2026-04-03 00:20:52,292 - lava_callback - ERROR - Error processing callback for node 69cc64d68cd9cbcdc945968a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36438 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:21:52,247 - lava_callback - INFO - Received callback for node 69cc650e8cd9cbcdc945969a INFO: 172.21.0.1:45692 - "POST /node/69cc650e8cd9cbcdc945969a HTTP/1.0" 202 Accepted 2026-04-03 00:21:52,253 - lava_callback - INFO - Starting async job submit for node 69cc650e8cd9cbcdc945969a 2026-04-03 00:21:52,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:21:52,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc650e8cd9cbcdc945969a HTTP/1.1" 200 None 2026-04-03 00:21:52,269 - lava_callback - DEBUG - Processing node 69cc650e8cd9cbcdc945969a: baseline-arm-baylibre 2026-04-03 00:21:52,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc650e8cd9cbcdc945969a 2026-04-03 00:21:52,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:21:52,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc650e8cd9cbcdc945969a HTTP/1.1" 200 None 2026-04-03 00:21:52,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:21:52,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:21:52,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:21:52,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc650e8cd9cbcdc945969a HTTP/1.1" 200 None 2026-04-03 00:21:52,338 - lava_callback - ERROR - Error processing callback for node 69cc650e8cd9cbcdc945969a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37928 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:22:32,301 - lava_callback - INFO - Received callback for node 69cc65398cd9cbcdc94596a8 INFO: 172.21.0.1:50990 - "POST /node/69cc65398cd9cbcdc94596a8 HTTP/1.0" 202 Accepted 2026-04-03 00:22:32,307 - lava_callback - INFO - Starting async job submit for node 69cc65398cd9cbcdc94596a8 2026-04-03 00:22:32,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:22:32,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc65398cd9cbcdc94596a8 HTTP/1.1" 200 None 2026-04-03 00:22:32,323 - lava_callback - DEBUG - Processing node 69cc65398cd9cbcdc94596a8: baseline-arm-baylibre 2026-04-03 00:22:32,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc65398cd9cbcdc94596a8 2026-04-03 00:22:32,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:22:32,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc65398cd9cbcdc94596a8 HTTP/1.1" 200 None 2026-04-03 00:22:32,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:22:32,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:22:32,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:22:32,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc65398cd9cbcdc94596a8 HTTP/1.1" 200 None 2026-04-03 00:22:32,407 - lava_callback - ERROR - Error processing callback for node 69cc65398cd9cbcdc94596a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60362 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:23:52,388 - lava_callback - INFO - Received callback for node 69cc658c8cd9cbcdc94596b7 INFO: 172.21.0.1:39472 - "POST /node/69cc658c8cd9cbcdc94596b7 HTTP/1.0" 202 Accepted 2026-04-03 00:23:52,393 - lava_callback - INFO - Starting async job submit for node 69cc658c8cd9cbcdc94596b7 2026-04-03 00:23:52,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:23:52,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc658c8cd9cbcdc94596b7 HTTP/1.1" 200 None 2026-04-03 00:23:52,434 - lava_callback - DEBUG - Processing node 69cc658c8cd9cbcdc94596b7: baseline-arm-baylibre 2026-04-03 00:23:52,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc658c8cd9cbcdc94596b7 2026-04-03 00:23:52,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:23:52,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc658c8cd9cbcdc94596b7 HTTP/1.1" 200 None 2026-04-03 00:23:52,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:23:52,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:23:52,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:23:52,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc658c8cd9cbcdc94596b7 HTTP/1.1" 200 None 2026-04-03 00:23:52,511 - lava_callback - ERROR - Error processing callback for node 69cc658c8cd9cbcdc94596b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54860 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:24:51,029 - lava_callback - INFO - Received callback for node 69cc65cb8cd9cbcdc94596c6 INFO: 172.21.0.1:33786 - "POST /node/69cc65cb8cd9cbcdc94596c6 HTTP/1.0" 202 Accepted 2026-04-03 00:24:51,035 - lava_callback - INFO - Starting async job submit for node 69cc65cb8cd9cbcdc94596c6 2026-04-03 00:24:51,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:24:51,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc65cb8cd9cbcdc94596c6 HTTP/1.1" 200 None 2026-04-03 00:24:51,053 - lava_callback - DEBUG - Processing node 69cc65cb8cd9cbcdc94596c6: baseline-arm-clabbe 2026-04-03 00:24:51,053 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc65cb8cd9cbcdc94596c6 2026-04-03 00:24:51,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:24:51,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc65cb8cd9cbcdc94596c6 HTTP/1.1" 200 None 2026-04-03 00:24:51,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:24:51,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:24:51,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:24:51,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc65cb8cd9cbcdc94596c6 HTTP/1.1" 200 None 2026-04-03 00:24:51,114 - lava_callback - ERROR - Error processing callback for node 69cc65cb8cd9cbcdc94596c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43644 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:25:51,035 - lava_callback - INFO - Received callback for node 69cc66078cd9cbcdc94596d8 INFO: 172.21.0.1:41430 - "POST /node/69cc66078cd9cbcdc94596d8 HTTP/1.0" 202 Accepted 2026-04-03 00:25:51,040 - lava_callback - INFO - Starting async job submit for node 69cc66078cd9cbcdc94596d8 2026-04-03 00:25:51,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:25:51,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc66078cd9cbcdc94596d8 HTTP/1.1" 200 None 2026-04-03 00:25:51,059 - lava_callback - DEBUG - Processing node 69cc66078cd9cbcdc94596d8: baseline-arm-clabbe 2026-04-03 00:25:51,059 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc66078cd9cbcdc94596d8 2026-04-03 00:25:51,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:25:51,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc66078cd9cbcdc94596d8 HTTP/1.1" 200 None 2026-04-03 00:25:51,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:25:51,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:25:51,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:25:51,126 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc66078cd9cbcdc94596d8 HTTP/1.1" 200 None 2026-04-03 00:25:51,128 - lava_callback - ERROR - Error processing callback for node 69cc66078cd9cbcdc94596d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48736 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:26:51,101 - lava_callback - INFO - Received callback for node 69cc66348cd9cbcdc94596ed INFO: 172.21.0.1:45898 - "POST /node/69cc66348cd9cbcdc94596ed HTTP/1.0" 202 Accepted 2026-04-03 00:26:51,107 - lava_callback - INFO - Starting async job submit for node 69cc66348cd9cbcdc94596ed 2026-04-03 00:26:51,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:26:51,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc66348cd9cbcdc94596ed HTTP/1.1" 200 None 2026-04-03 00:26:51,124 - lava_callback - DEBUG - Processing node 69cc66348cd9cbcdc94596ed: baseline-arm-clabbe 2026-04-03 00:26:51,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc66348cd9cbcdc94596ed 2026-04-03 00:26:51,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:26:51,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc66348cd9cbcdc94596ed HTTP/1.1" 200 None 2026-04-03 00:26:51,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:26:51,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:26:51,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:26:51,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc66348cd9cbcdc94596ed HTTP/1.1" 200 None 2026-04-03 00:26:51,177 - lava_callback - ERROR - Error processing callback for node 69cc66348cd9cbcdc94596ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48576 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:27:31,046 - lava_callback - INFO - Received callback for node 69cc66608cd9cbcdc94596fb INFO: 172.21.0.1:37610 - "POST /node/69cc66608cd9cbcdc94596fb HTTP/1.0" 202 Accepted 2026-04-03 00:27:31,052 - lava_callback - INFO - Starting async job submit for node 69cc66608cd9cbcdc94596fb 2026-04-03 00:27:31,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:27:31,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc66608cd9cbcdc94596fb HTTP/1.1" 200 None 2026-04-03 00:27:31,072 - lava_callback - DEBUG - Processing node 69cc66608cd9cbcdc94596fb: baseline-arm-clabbe 2026-04-03 00:27:31,072 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc66608cd9cbcdc94596fb 2026-04-03 00:27:31,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:27:31,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc66608cd9cbcdc94596fb HTTP/1.1" 200 None 2026-04-03 00:27:31,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:27:31,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:27:31,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:27:31,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc66608cd9cbcdc94596fb HTTP/1.1" 200 None 2026-04-03 00:27:31,124 - lava_callback - ERROR - Error processing callback for node 69cc66608cd9cbcdc94596fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54826 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:28:11,114 - lava_callback - INFO - Received callback for node 69cc668a8cd9cbcdc9459711 INFO: 172.21.0.1:58506 - "POST /node/69cc668a8cd9cbcdc9459711 HTTP/1.0" 202 Accepted 2026-04-03 00:28:11,123 - lava_callback - INFO - Starting async job submit for node 69cc668a8cd9cbcdc9459711 2026-04-03 00:28:11,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:28:11,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc668a8cd9cbcdc9459711 HTTP/1.1" 200 None 2026-04-03 00:28:11,154 - lava_callback - DEBUG - Processing node 69cc668a8cd9cbcdc9459711: baseline-arm-clabbe 2026-04-03 00:28:11,155 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc668a8cd9cbcdc9459711 2026-04-03 00:28:11,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:28:11,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc668a8cd9cbcdc9459711 HTTP/1.1" 200 None 2026-04-03 00:28:11,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:28:11,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c6ae0102375b4506df898a HTTP/1.1" 200 None 2026-04-03 00:28:11,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:28:11,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc668a8cd9cbcdc9459711 HTTP/1.1" 200 None 2026-04-03 00:28:11,248 - lava_callback - ERROR - Error processing callback for node 69cc668a8cd9cbcdc9459711: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50902 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35954 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:33:01,750 - lava_callback - INFO - Received callback for node 69cf0a529e2394c38fe01336 INFO: 172.21.0.1:58526 - "POST /node/69cf0a529e2394c38fe01336 HTTP/1.0" 202 Accepted 2026-04-03 00:33:01,761 - lava_callback - INFO - Starting async job submit for node 69cf0a529e2394c38fe01336 2026-04-03 00:33:01,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:33:01,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0a529e2394c38fe01336 HTTP/1.1" 200 None 2026-04-03 00:33:01,803 - lava_callback - DEBUG - Processing node 69cf0a529e2394c38fe01336: baseline-arm 2026-04-03 00:33:01,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:33:02,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:33:02,189 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0a529e2394c38fe01336/log.txt.gz 2026-04-03 00:33:02,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:33:02,665 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:33:02,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0a529e2394c38fe01336/lava-logs.yaml 2026-04-03 00:33:02,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:33:02,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:33:02,841 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0a529e2394c38fe01336/lava_callback.json.gz 2026-04-03 00:33:02,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:33:02,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0a529e2394c38fe01336 HTTP/1.1" 200 None 2026-04-03 00:33:02,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:33:02,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf07039e2394c38fe00f45 HTTP/1.1" 200 None 2026-04-03 00:33:02,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:33:02,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0a529e2394c38fe01336 HTTP/1.1" 200 None 2026-04-03 00:33:02,932 - lava_callback - INFO - Completed processing callback for node 69cf0a529e2394c38fe01336 INFO: 172.21.0.1:58532 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:33:10,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:33:10,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35766 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:36:13,095 - lava_callback - INFO - Received callback for node 69cc687a8cd9cbcdc9459ad9 INFO: 172.21.0.1:46606 - "POST /node/69cc687a8cd9cbcdc9459ad9 HTTP/1.0" 202 Accepted 2026-04-03 00:36:13,101 - lava_callback - INFO - Starting async job submit for node 69cc687a8cd9cbcdc9459ad9 2026-04-03 00:36:13,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:13,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc687a8cd9cbcdc9459ad9 HTTP/1.1" 200 None 2026-04-03 00:36:13,141 - lava_callback - DEBUG - Processing node 69cc687a8cd9cbcdc9459ad9: baseline-arm-baylibre 2026-04-03 00:36:13,141 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc687a8cd9cbcdc9459ad9 2026-04-03 00:36:13,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:13,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc687a8cd9cbcdc9459ad9 HTTP/1.1" 200 None 2026-04-03 00:36:13,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:13,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 00:36:13,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:13,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc687a8cd9cbcdc9459ad9 HTTP/1.1" 200 None 2026-04-03 00:36:13,208 - lava_callback - ERROR - Error processing callback for node 69cc687a8cd9cbcdc9459ad9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 00:36:13,741 - lava_callback - INFO - Received callback for node 69cc68788cd9cbcdc9459ace INFO: 172.21.0.1:46620 - "POST /node/69cc68788cd9cbcdc9459ace HTTP/1.0" 202 Accepted 2026-04-03 00:36:13,746 - lava_callback - INFO - Starting async job submit for node 69cc68788cd9cbcdc9459ace 2026-04-03 00:36:13,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:13,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68788cd9cbcdc9459ace HTTP/1.1" 200 None 2026-04-03 00:36:13,763 - lava_callback - DEBUG - Processing node 69cc68788cd9cbcdc9459ace: baseline-arm-baylibre 2026-04-03 00:36:13,763 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc68788cd9cbcdc9459ace 2026-04-03 00:36:13,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:13,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68788cd9cbcdc9459ace HTTP/1.1" 200 None 2026-04-03 00:36:13,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:13,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 00:36:13,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:13,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68788cd9cbcdc9459ace HTTP/1.1" 200 None 2026-04-03 00:36:13,817 - lava_callback - ERROR - Error processing callback for node 69cc68788cd9cbcdc9459ace: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34956 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:36:31,561 - lava_callback - INFO - Received callback for node 69cc68848cd9cbcdc9459af7 INFO: 172.21.0.1:33906 - "POST /node/69cc68848cd9cbcdc9459af7 HTTP/1.0" 202 Accepted 2026-04-03 00:36:31,567 - lava_callback - INFO - Starting async job submit for node 69cc68848cd9cbcdc9459af7 2026-04-03 00:36:31,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:31,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68848cd9cbcdc9459af7 HTTP/1.1" 200 None 2026-04-03 00:36:31,589 - lava_callback - DEBUG - Processing node 69cc68848cd9cbcdc9459af7: baseline-arm-clabbe 2026-04-03 00:36:31,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc68848cd9cbcdc9459af7 2026-04-03 00:36:31,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:31,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68848cd9cbcdc9459af7 HTTP/1.1" 200 None 2026-04-03 00:36:31,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:31,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 00:36:31,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:31,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68848cd9cbcdc9459af7 HTTP/1.1" 200 None 2026-04-03 00:36:31,644 - lava_callback - ERROR - Error processing callback for node 69cc68848cd9cbcdc9459af7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 00:36:32,233 - lava_callback - INFO - Received callback for node 69cc68878cd9cbcdc9459afd INFO: 172.21.0.1:33914 - "POST /node/69cc68878cd9cbcdc9459afd HTTP/1.0" 202 Accepted 2026-04-03 00:36:32,239 - lava_callback - INFO - Starting async job submit for node 69cc68878cd9cbcdc9459afd 2026-04-03 00:36:32,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:32,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68878cd9cbcdc9459afd HTTP/1.1" 200 None 2026-04-03 00:36:32,258 - lava_callback - DEBUG - Processing node 69cc68878cd9cbcdc9459afd: baseline-arm-clabbe 2026-04-03 00:36:32,258 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc68878cd9cbcdc9459afd 2026-04-03 00:36:32,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:32,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68878cd9cbcdc9459afd HTTP/1.1" 200 None 2026-04-03 00:36:32,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:32,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 00:36:32,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:32,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68878cd9cbcdc9459afd HTTP/1.1" 200 None 2026-04-03 00:36:32,308 - lava_callback - ERROR - Error processing callback for node 69cc68878cd9cbcdc9459afd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 00:36:32,936 - lava_callback - INFO - Received callback for node 69cc68898cd9cbcdc9459b2d INFO: 172.21.0.1:33922 - "POST /node/69cc68898cd9cbcdc9459b2d HTTP/1.0" 202 Accepted 2026-04-03 00:36:32,942 - lava_callback - INFO - Starting async job submit for node 69cc68898cd9cbcdc9459b2d 2026-04-03 00:36:32,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:32,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68898cd9cbcdc9459b2d HTTP/1.1" 200 None 2026-04-03 00:36:32,961 - lava_callback - DEBUG - Processing node 69cc68898cd9cbcdc9459b2d: baseline-arm-clabbe 2026-04-03 00:36:32,961 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc68898cd9cbcdc9459b2d 2026-04-03 00:36:32,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:32,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68898cd9cbcdc9459b2d HTTP/1.1" 200 None 2026-04-03 00:36:32,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:32,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 00:36:32,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:33,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68898cd9cbcdc9459b2d HTTP/1.1" 200 None 2026-04-03 00:36:33,018 - lava_callback - ERROR - Error processing callback for node 69cc68898cd9cbcdc9459b2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 00:36:33,074 - lava_callback - INFO - Received callback for node 69cc687c8cd9cbcdc9459adc INFO: 172.21.0.1:33928 - "POST /node/69cc687c8cd9cbcdc9459adc HTTP/1.0" 202 Accepted 2026-04-03 00:36:33,084 - lava_callback - INFO - Starting async job submit for node 69cc687c8cd9cbcdc9459adc 2026-04-03 00:36:33,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:33,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc687c8cd9cbcdc9459adc HTTP/1.1" 200 None 2026-04-03 00:36:33,108 - lava_callback - DEBUG - Processing node 69cc687c8cd9cbcdc9459adc: baseline-arm-baylibre 2026-04-03 00:36:33,108 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc687c8cd9cbcdc9459adc 2026-04-03 00:36:33,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:33,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc687c8cd9cbcdc9459adc HTTP/1.1" 200 None 2026-04-03 00:36:33,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:33,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 00:36:33,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:33,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc687c8cd9cbcdc9459adc HTTP/1.1" 200 None 2026-04-03 00:36:33,187 - lava_callback - ERROR - Error processing callback for node 69cc687c8cd9cbcdc9459adc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 00:36:33,658 - lava_callback - INFO - Received callback for node 69cc68898cd9cbcdc9459b30 INFO: 172.21.0.1:33944 - "POST /node/69cc68898cd9cbcdc9459b30 HTTP/1.0" 202 Accepted 2026-04-03 00:36:33,663 - lava_callback - INFO - Starting async job submit for node 69cc68898cd9cbcdc9459b30 2026-04-03 00:36:33,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:33,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68898cd9cbcdc9459b30 HTTP/1.1" 200 None 2026-04-03 00:36:33,693 - lava_callback - DEBUG - Processing node 69cc68898cd9cbcdc9459b30: baseline-arm-clabbe 2026-04-03 00:36:33,693 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc68898cd9cbcdc9459b30 2026-04-03 00:36:33,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:33,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc68898cd9cbcdc9459b30 HTTP/1.1" 200 None 2026-04-03 00:36:33,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:33,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 00:36:33,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:33,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc68898cd9cbcdc9459b30 HTTP/1.1" 200 None 2026-04-03 00:36:33,749 - lava_callback - ERROR - Error processing callback for node 69cc68898cd9cbcdc9459b30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 00:36:34,346 - lava_callback - INFO - Received callback for node 69cc688a8cd9cbcdc9459b33 INFO: 172.21.0.1:33950 - "POST /node/69cc688a8cd9cbcdc9459b33 HTTP/1.0" 202 Accepted 2026-04-03 00:36:34,352 - lava_callback - INFO - Starting async job submit for node 69cc688a8cd9cbcdc9459b33 2026-04-03 00:36:34,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:34,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc688a8cd9cbcdc9459b33 HTTP/1.1" 200 None 2026-04-03 00:36:34,368 - lava_callback - DEBUG - Processing node 69cc688a8cd9cbcdc9459b33: baseline-arm-clabbe 2026-04-03 00:36:34,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc688a8cd9cbcdc9459b33 2026-04-03 00:36:34,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:34,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc688a8cd9cbcdc9459b33 HTTP/1.1" 200 None 2026-04-03 00:36:34,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:34,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 00:36:34,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:34,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc688a8cd9cbcdc9459b33 HTTP/1.1" 200 None 2026-04-03 00:36:34,427 - lava_callback - ERROR - Error processing callback for node 69cc688a8cd9cbcdc9459b33: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 00:36:35,096 - lava_callback - INFO - Received callback for node 69cc688d8cd9cbcdc9459b3c INFO: 172.21.0.1:33966 - "POST /node/69cc688d8cd9cbcdc9459b3c HTTP/1.0" 202 Accepted 2026-04-03 00:36:35,101 - lava_callback - INFO - Starting async job submit for node 69cc688d8cd9cbcdc9459b3c 2026-04-03 00:36:35,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:35,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc688d8cd9cbcdc9459b3c HTTP/1.1" 200 None 2026-04-03 00:36:35,117 - lava_callback - DEBUG - Processing node 69cc688d8cd9cbcdc9459b3c: baseline-arm-clabbe 2026-04-03 00:36:35,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc688d8cd9cbcdc9459b3c 2026-04-03 00:36:35,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:35,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc688d8cd9cbcdc9459b3c HTTP/1.1" 200 None 2026-04-03 00:36:35,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:35,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 00:36:35,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:36:35,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc688d8cd9cbcdc9459b3c HTTP/1.1" 200 None 2026-04-03 00:36:35,172 - lava_callback - ERROR - Error processing callback for node 69cc688d8cd9cbcdc9459b3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55534 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:38:18,258 - lava_callback - INFO - Received callback for node 69cf0be19e2394c38fe013b0 INFO: 172.21.0.1:55548 - "POST /node/69cf0be19e2394c38fe013b0 HTTP/1.0" 202 Accepted 2026-04-03 00:38:18,265 - lava_callback - INFO - Starting async job submit for node 69cf0be19e2394c38fe013b0 2026-04-03 00:38:18,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:38:18,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0be19e2394c38fe013b0 HTTP/1.1" 200 None 2026-04-03 00:38:18,299 - lava_callback - DEBUG - Processing node 69cf0be19e2394c38fe013b0: baseline-x86 2026-04-03 00:38:18,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:38:18,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:38:18,615 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be19e2394c38fe013b0/log.txt.gz 2026-04-03 00:38:18,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:38:18,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:38:18,888 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be19e2394c38fe013b0/lava-logs.yaml 2026-04-03 00:38:18,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:38:19,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:38:19,023 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be19e2394c38fe013b0/lava_callback.json.gz 2026-04-03 00:38:19,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:38:19,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0be19e2394c38fe013b0 HTTP/1.1" 200 None 2026-04-03 00:38:19,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:38:19,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf071c9e2394c38fe00f6d HTTP/1.1" 200 None 2026-04-03 00:38:19,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:38:19,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0be19e2394c38fe013b0 HTTP/1.1" 200 None 2026-04-03 00:38:19,098 - lava_callback - INFO - Completed processing callback for node 69cf0be19e2394c38fe013b0 2026-04-03 00:38:22,111 - lava_callback - INFO - Received callback for node 69cf0bdf9e2394c38fe013a6 INFO: 172.21.0.1:37968 - "POST /node/69cf0bdf9e2394c38fe013a6 HTTP/1.0" 202 Accepted 2026-04-03 00:38:22,117 - lava_callback - INFO - Starting async job submit for node 69cf0bdf9e2394c38fe013a6 2026-04-03 00:38:22,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:38:22,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0bdf9e2394c38fe013a6 HTTP/1.1" 200 None 2026-04-03 00:38:22,152 - lava_callback - DEBUG - Processing node 69cf0bdf9e2394c38fe013a6: baseline-x86 2026-04-03 00:38:22,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:38:22,457 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:38:22,459 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0bdf9e2394c38fe013a6/log.txt.gz 2026-04-03 00:38:22,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:38:22,739 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:38:22,741 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0bdf9e2394c38fe013a6/lava-logs.yaml 2026-04-03 00:38:22,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:38:22,905 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:38:22,906 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0bdf9e2394c38fe013a6/lava_callback.json.gz 2026-04-03 00:38:22,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:38:22,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0bdf9e2394c38fe013a6 HTTP/1.1" 200 None 2026-04-03 00:38:22,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:38:22,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf071c9e2394c38fe00f6d HTTP/1.1" 200 None 2026-04-03 00:38:22,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:38:23,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0bdf9e2394c38fe013a6 HTTP/1.1" 200 None 2026-04-03 00:38:23,004 - lava_callback - INFO - Completed processing callback for node 69cf0bdf9e2394c38fe013a6 INFO: 172.21.0.1:37970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37068 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:38:40,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:38:40,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49166 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:39:32,587 - lava_callback - INFO - Received callback for node 69cf0be29e2394c38fe013b3 INFO: 172.21.0.1:33874 - "POST /node/69cf0be29e2394c38fe013b3 HTTP/1.0" 202 Accepted 2026-04-03 00:39:32,592 - lava_callback - INFO - Starting async job submit for node 69cf0be29e2394c38fe013b3 2026-04-03 00:39:32,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:39:32,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0be29e2394c38fe013b3 HTTP/1.1" 200 None 2026-04-03 00:39:32,627 - lava_callback - DEBUG - Processing node 69cf0be29e2394c38fe013b3: baseline-x86 2026-04-03 00:39:32,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:39:33,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:39:33,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be29e2394c38fe013b3/log.txt.gz 2026-04-03 00:39:33,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:39:33,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:39:33,619 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be29e2394c38fe013b3/lava-logs.yaml 2026-04-03 00:39:33,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:39:33,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:39:33,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be29e2394c38fe013b3/lava_callback.json.gz 2026-04-03 00:39:33,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:39:33,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0be29e2394c38fe013b3 HTTP/1.1" 200 None 2026-04-03 00:39:33,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:39:33,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf071c9e2394c38fe00f6d HTTP/1.1" 200 None 2026-04-03 00:39:33,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:39:33,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0be29e2394c38fe013b3 HTTP/1.1" 200 None 2026-04-03 00:39:33,859 - lava_callback - INFO - Completed processing callback for node 69cf0be29e2394c38fe013b3 INFO: 172.21.0.1:33878 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:39:40,205 - lava_callback - INFO - Received callback for node 69cf0be09e2394c38fe013a9 INFO: 172.21.0.1:52764 - "POST /node/69cf0be09e2394c38fe013a9 HTTP/1.0" 202 Accepted 2026-04-03 00:39:40,211 - lava_callback - INFO - Starting async job submit for node 69cf0be09e2394c38fe013a9 2026-04-03 00:39:40,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:39:40,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0be09e2394c38fe013a9 HTTP/1.1" 200 None 2026-04-03 00:39:40,251 - lava_callback - DEBUG - Processing node 69cf0be09e2394c38fe013a9: baseline-x86 2026-04-03 00:39:40,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:39:40,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:39:40,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 00:39:40,960 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:39:40,961 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be09e2394c38fe013a9/log.txt.gz 2026-04-03 00:39:41,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:39:41,352 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:39:41,354 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be09e2394c38fe013a9/lava-logs.yaml 2026-04-03 00:39:41,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:39:41,507 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:39:41,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be09e2394c38fe013a9/lava_callback.json.gz 2026-04-03 00:39:41,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:39:41,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0be09e2394c38fe013a9 HTTP/1.1" 200 None 2026-04-03 00:39:41,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:39:41,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf071c9e2394c38fe00f6d HTTP/1.1" 200 None 2026-04-03 00:39:41,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:39:41,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0be09e2394c38fe013a9 HTTP/1.1" 200 None 2026-04-03 00:39:41,599 - lava_callback - INFO - Completed processing callback for node 69cf0be09e2394c38fe013a9 INFO: 172.21.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42208 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:40:10,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:40:10,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60794 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43716 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:46:53,705 - lava_callback - INFO - Received callback for node 69cc6af38cd9cbcdc9459cd8 INFO: 172.21.0.1:50048 - "POST /node/69cc6af38cd9cbcdc9459cd8 HTTP/1.0" 202 Accepted 2026-04-03 00:46:53,714 - lava_callback - INFO - Starting async job submit for node 69cc6af38cd9cbcdc9459cd8 2026-04-03 00:46:53,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:46:53,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6af38cd9cbcdc9459cd8 HTTP/1.1" 200 None 2026-04-03 00:46:53,734 - lava_callback - DEBUG - Processing node 69cc6af38cd9cbcdc9459cd8: baseline-arm-baylibre 2026-04-03 00:46:53,734 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6af38cd9cbcdc9459cd8 2026-04-03 00:46:53,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:46:53,754 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6af38cd9cbcdc9459cd8 HTTP/1.1" 200 None 2026-04-03 00:46:53,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:46:53,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 00:46:53,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:46:53,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6af38cd9cbcdc9459cd8 HTTP/1.1" 200 None 2026-04-03 00:46:53,820 - lava_callback - ERROR - Error processing callback for node 69cc6af38cd9cbcdc9459cd8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43300 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:47:33,668 - lava_callback - INFO - Received callback for node 69cc6b1d8cd9cbcdc9459d02 INFO: 172.21.0.1:44750 - "POST /node/69cc6b1d8cd9cbcdc9459d02 HTTP/1.0" 202 Accepted 2026-04-03 00:47:33,676 - lava_callback - INFO - Starting async job submit for node 69cc6b1d8cd9cbcdc9459d02 2026-04-03 00:47:33,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:47:33,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b1d8cd9cbcdc9459d02 HTTP/1.1" 200 None 2026-04-03 00:47:33,697 - lava_callback - DEBUG - Processing node 69cc6b1d8cd9cbcdc9459d02: baseline-arm-baylibre 2026-04-03 00:47:33,697 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6b1d8cd9cbcdc9459d02 2026-04-03 00:47:33,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:47:33,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b1d8cd9cbcdc9459d02 HTTP/1.1" 200 None 2026-04-03 00:47:33,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:47:33,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 00:47:33,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:47:33,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6b1d8cd9cbcdc9459d02 HTTP/1.1" 200 None 2026-04-03 00:47:33,790 - lava_callback - ERROR - Error processing callback for node 69cc6b1d8cd9cbcdc9459d02: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58852 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:48:13,788 - lava_callback - INFO - Received callback for node 69cc6b478cd9cbcdc9459d12 INFO: 172.21.0.1:38948 - "POST /node/69cc6b478cd9cbcdc9459d12 HTTP/1.0" 202 Accepted 2026-04-03 00:48:13,797 - lava_callback - INFO - Starting async job submit for node 69cc6b478cd9cbcdc9459d12 2026-04-03 00:48:13,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:48:13,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b478cd9cbcdc9459d12 HTTP/1.1" 200 None 2026-04-03 00:48:13,825 - lava_callback - DEBUG - Processing node 69cc6b478cd9cbcdc9459d12: baseline-arm-baylibre 2026-04-03 00:48:13,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6b478cd9cbcdc9459d12 2026-04-03 00:48:13,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:48:13,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b478cd9cbcdc9459d12 HTTP/1.1" 200 None 2026-04-03 00:48:13,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:48:13,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 00:48:13,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:48:13,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6b478cd9cbcdc9459d12 HTTP/1.1" 200 None 2026-04-03 00:48:13,914 - lava_callback - ERROR - Error processing callback for node 69cc6b478cd9cbcdc9459d12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42318 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:48:53,758 - lava_callback - INFO - Received callback for node 69cc6b728cd9cbcdc9459d2e INFO: 172.21.0.1:36990 - "POST /node/69cc6b728cd9cbcdc9459d2e HTTP/1.0" 202 Accepted 2026-04-03 00:48:53,767 - lava_callback - INFO - Starting async job submit for node 69cc6b728cd9cbcdc9459d2e 2026-04-03 00:48:53,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:48:53,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b728cd9cbcdc9459d2e HTTP/1.1" 200 None 2026-04-03 00:48:53,795 - lava_callback - DEBUG - Processing node 69cc6b728cd9cbcdc9459d2e: baseline-arm-baylibre 2026-04-03 00:48:53,796 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6b728cd9cbcdc9459d2e 2026-04-03 00:48:53,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:48:53,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b728cd9cbcdc9459d2e HTTP/1.1" 200 None 2026-04-03 00:48:53,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:48:53,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 00:48:53,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:48:53,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6b728cd9cbcdc9459d2e HTTP/1.1" 200 None 2026-04-03 00:48:53,951 - lava_callback - ERROR - Error processing callback for node 69cc6b728cd9cbcdc9459d2e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56314 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:49:10,651 - lava_callback - INFO - Received callback for node 69cf0e719e2394c38fe0151e INFO: 172.21.0.1:36220 - "POST /node/69cf0e719e2394c38fe0151e HTTP/1.0" 202 Accepted 2026-04-03 00:49:10,660 - lava_callback - INFO - Starting async job submit for node 69cf0e719e2394c38fe0151e 2026-04-03 00:49:10,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:10,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e719e2394c38fe0151e HTTP/1.1" 200 None 2026-04-03 00:49:10,698 - lava_callback - DEBUG - Processing node 69cf0e719e2394c38fe0151e: baseline-arm 2026-04-03 00:49:10,706 - lava_callback - WARNING - Failed to upload log 2026-04-03 00:49:10,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:49:11,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:49:11,025 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e719e2394c38fe0151e/lava-logs.yaml 2026-04-03 00:49:11,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:49:11,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:49:11,177 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e719e2394c38fe0151e/lava_callback.json.gz 2026-04-03 00:49:11,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:11,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e719e2394c38fe0151e HTTP/1.1" 200 None 2026-04-03 00:49:11,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:11,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:49:11,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:11,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0e719e2394c38fe0151e HTTP/1.1" 200 None 2026-04-03 00:49:11,248 - lava_callback - INFO - Completed processing callback for node 69cf0e719e2394c38fe0151e INFO: 172.21.0.1:36230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36236 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:49:24,841 - lava_callback - INFO - Received callback for node 69cf0e759e2394c38fe0152f INFO: 172.21.0.1:59652 - "POST /node/69cf0e759e2394c38fe0152f HTTP/1.0" 202 Accepted 2026-04-03 00:49:24,893 - lava_callback - INFO - Starting async job submit for node 69cf0e759e2394c38fe0152f 2026-04-03 00:49:24,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:24,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e759e2394c38fe0152f HTTP/1.1" 200 None 2026-04-03 00:49:24,961 - lava_callback - DEBUG - Processing node 69cf0e759e2394c38fe0152f: baseline-arm 2026-04-03 00:49:25,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:49:25,353 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:49:25,355 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e759e2394c38fe0152f/log.txt.gz 2026-04-03 00:49:25,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:49:26,385 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:49:26,388 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e759e2394c38fe0152f/lava-logs.yaml 2026-04-03 00:49:26,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:59660 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:49:26,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:49:26,609 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e759e2394c38fe0152f/lava_callback.json.gz 2026-04-03 00:49:26,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:26,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e759e2394c38fe0152f HTTP/1.1" 200 None 2026-04-03 00:49:26,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:26,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:49:26,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:26,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0e759e2394c38fe0152f HTTP/1.1" 200 None 2026-04-03 00:49:26,798 - lava_callback - INFO - Completed processing callback for node 69cf0e759e2394c38fe0152f 2026-04-03 00:49:27,917 - lava_callback - INFO - Received callback for node 69cf0e729e2394c38fe01521 INFO: 172.21.0.1:59668 - "POST /node/69cf0e729e2394c38fe01521 HTTP/1.0" 202 Accepted 2026-04-03 00:49:27,970 - lava_callback - INFO - Starting async job submit for node 69cf0e729e2394c38fe01521 2026-04-03 00:49:28,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:28,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e729e2394c38fe01521 HTTP/1.1" 200 None 2026-04-03 00:49:28,032 - lava_callback - DEBUG - Processing node 69cf0e729e2394c38fe01521: baseline-arm 2026-04-03 00:49:28,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:49:28,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:49:28,528 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e729e2394c38fe01521/log.txt.gz 2026-04-03 00:49:28,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:49:29,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:49:29,563 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e729e2394c38fe01521/lava-logs.yaml 2026-04-03 00:49:29,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:49:29,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:49:29,927 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e729e2394c38fe01521/lava_callback.json.gz 2026-04-03 00:49:29,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:29,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e729e2394c38fe01521 HTTP/1.1" 200 None 2026-04-03 00:49:29,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:30,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:49:30,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:30,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0e729e2394c38fe01521 HTTP/1.1" 200 None 2026-04-03 00:49:30,066 - lava_callback - INFO - Completed processing callback for node 69cf0e729e2394c38fe01521 INFO: 172.21.0.1:39558 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:49:40,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:40,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55010 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:49:53,858 - lava_callback - INFO - Received callback for node 69cc6b9d8cd9cbcdc9459d44 INFO: 172.21.0.1:55762 - "POST /node/69cc6b9d8cd9cbcdc9459d44 HTTP/1.0" 202 Accepted 2026-04-03 00:49:53,867 - lava_callback - INFO - Starting async job submit for node 69cc6b9d8cd9cbcdc9459d44 2026-04-03 00:49:53,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:53,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b9d8cd9cbcdc9459d44 HTTP/1.1" 200 None 2026-04-03 00:49:53,891 - lava_callback - DEBUG - Processing node 69cc6b9d8cd9cbcdc9459d44: baseline-arm-baylibre 2026-04-03 00:49:53,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6b9d8cd9cbcdc9459d44 2026-04-03 00:49:53,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:53,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6b9d8cd9cbcdc9459d44 HTTP/1.1" 200 None 2026-04-03 00:49:53,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:53,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 00:49:53,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:49:53,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6b9d8cd9cbcdc9459d44 HTTP/1.1" 200 None 2026-04-03 00:49:53,989 - lava_callback - ERROR - Error processing callback for node 69cc6b9d8cd9cbcdc9459d44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55768 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:50:00,508 - lava_callback - INFO - Received callback for node 69cf0e749e2394c38fe01529 INFO: 172.21.0.1:56870 - "POST /node/69cf0e749e2394c38fe01529 HTTP/1.0" 202 Accepted 2026-04-03 00:50:00,517 - lava_callback - INFO - Starting async job submit for node 69cf0e749e2394c38fe01529 2026-04-03 00:50:00,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:00,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e749e2394c38fe01529 HTTP/1.1" 200 None 2026-04-03 00:50:00,565 - lava_callback - DEBUG - Processing node 69cf0e749e2394c38fe01529: baseline-arm 2026-04-03 00:50:00,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:01,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:01,110 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e749e2394c38fe01529/log.txt.gz 2026-04-03 00:50:01,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:01,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:01,486 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e749e2394c38fe01529/lava-logs.yaml 2026-04-03 00:50:01,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:01,624 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:01,626 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e749e2394c38fe01529/lava_callback.json.gz 2026-04-03 00:50:01,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:01,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e749e2394c38fe01529 HTTP/1.1" 200 None 2026-04-03 00:50:01,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:01,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:50:01,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:01,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0e749e2394c38fe01529 HTTP/1.1" 200 None 2026-04-03 00:50:01,732 - lava_callback - INFO - Completed processing callback for node 69cf0e749e2394c38fe01529 INFO: 172.21.0.1:56874 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:50:10,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:10,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58252 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:50:32,005 - lava_callback - INFO - Received callback for node 69cc6bca8cd9cbcdc9459d55 INFO: 172.21.0.1:53422 - "POST /node/69cc6bca8cd9cbcdc9459d55 HTTP/1.0" 202 Accepted 2026-04-03 00:50:32,020 - lava_callback - INFO - Starting async job submit for node 69cc6bca8cd9cbcdc9459d55 2026-04-03 00:50:32,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:32,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6bca8cd9cbcdc9459d55 HTTP/1.1" 200 None 2026-04-03 00:50:32,074 - lava_callback - DEBUG - Processing node 69cc6bca8cd9cbcdc9459d55: baseline-arm-clabbe 2026-04-03 00:50:32,074 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6bca8cd9cbcdc9459d55 2026-04-03 00:50:32,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:32,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6bca8cd9cbcdc9459d55 HTTP/1.1" 200 None 2026-04-03 00:50:32,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:32,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 00:50:32,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:32,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6bca8cd9cbcdc9459d55 HTTP/1.1" 200 None 2026-04-03 00:50:32,180 - lava_callback - ERROR - Error processing callback for node 69cc6bca8cd9cbcdc9459d55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 00:50:35,262 - lava_callback - INFO - Received callback for node 69cf0e709e2394c38fe0151b INFO: 172.21.0.1:53424 - "POST /node/69cf0e709e2394c38fe0151b HTTP/1.0" 202 Accepted 2026-04-03 00:50:35,272 - lava_callback - INFO - Starting async job submit for node 69cf0e709e2394c38fe0151b 2026-04-03 00:50:35,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:35,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e709e2394c38fe0151b HTTP/1.1" 200 None 2026-04-03 00:50:35,344 - lava_callback - DEBUG - Processing node 69cf0e709e2394c38fe0151b: baseline-arm 2026-04-03 00:50:35,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:36,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:36,477 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e709e2394c38fe0151b/log.txt.gz INFO: 172.21.0.1:53440 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:50:36,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:37,218 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:37,221 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e709e2394c38fe0151b/lava-logs.yaml 2026-04-03 00:50:37,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:38,018 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:38,020 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e709e2394c38fe0151b/lava_callback.json.gz 2026-04-03 00:50:38,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:38,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e709e2394c38fe0151b HTTP/1.1" 200 None 2026-04-03 00:50:38,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:38,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:50:38,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:38,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0e709e2394c38fe0151b HTTP/1.1" 200 None 2026-04-03 00:50:38,160 - lava_callback - INFO - Completed processing callback for node 69cf0e709e2394c38fe0151b 2026-04-03 00:50:40,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:40,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43656 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:50:49,274 - lava_callback - INFO - Received callback for node 69cf0eaf9e2394c38fe015e1 INFO: 172.21.0.1:43670 - "POST /node/69cf0eaf9e2394c38fe015e1 HTTP/1.0" 202 Accepted 2026-04-03 00:50:49,329 - lava_callback - INFO - Starting async job submit for node 69cf0eaf9e2394c38fe015e1 2026-04-03 00:50:49,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:49,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0eaf9e2394c38fe015e1 HTTP/1.1" 200 None 2026-04-03 00:50:49,433 - lava_callback - DEBUG - Processing node 69cf0eaf9e2394c38fe015e1: baseline-arm-pengutronix 2026-04-03 00:50:49,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:50,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:50,262 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0eaf9e2394c38fe015e1/log.txt.gz 2026-04-03 00:50:50,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:50,481 - lava_callback - INFO - Received callback for node 69cf0ea99e2394c38fe015d8 2026-04-03 00:50:50,492 - lava_callback - INFO - Starting async job submit for node 69cf0ea99e2394c38fe015d8 INFO: 172.21.0.1:38632 - "POST /node/69cf0ea99e2394c38fe015d8 HTTP/1.0" 202 Accepted 2026-04-03 00:50:50,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:50,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ea99e2394c38fe015d8 HTTP/1.1" 200 None 2026-04-03 00:50:50,562 - lava_callback - DEBUG - Processing node 69cf0ea99e2394c38fe015d8: baseline-arm-pengutronix 2026-04-03 00:50:50,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:51,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:51,198 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0eaf9e2394c38fe015e1/lava-logs.yaml 2026-04-03 00:50:51,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:51,523 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:51,525 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0ea99e2394c38fe015d8/log.txt.gz 2026-04-03 00:50:51,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:51,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:51,781 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0eaf9e2394c38fe015e1/lava_callback.json.gz 2026-04-03 00:50:51,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:51,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0eaf9e2394c38fe015e1 HTTP/1.1" 200 None 2026-04-03 00:50:51,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:51,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:50:51,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:51,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0eaf9e2394c38fe015e1 HTTP/1.1" 200 None 2026-04-03 00:50:51,945 - lava_callback - INFO - Completed processing callback for node 69cf0eaf9e2394c38fe015e1 2026-04-03 00:50:52,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:52,413 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0ea99e2394c38fe015d8/lava-logs.yaml 2026-04-03 00:50:52,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:52,456 - lava_callback - INFO - Received callback for node 69cf0eab9e2394c38fe015db INFO: 172.21.0.1:38640 - "POST /node/69cf0eab9e2394c38fe015db HTTP/1.0" 202 Accepted 2026-04-03 00:50:52,469 - lava_callback - INFO - Starting async job submit for node 69cf0eab9e2394c38fe015db 2026-04-03 00:50:52,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:52,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0eab9e2394c38fe015db HTTP/1.1" 200 None 2026-04-03 00:50:52,588 - lava_callback - DEBUG - Processing node 69cf0eab9e2394c38fe015db: baseline-arm-pengutronix 2026-04-03 00:50:52,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:53,355 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:53,358 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0ea99e2394c38fe015d8/lava_callback.json.gz 2026-04-03 00:50:53,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:53,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ea99e2394c38fe015d8 HTTP/1.1" 200 None 2026-04-03 00:50:53,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:53,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:50:53,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:53,488 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:53,490 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0eab9e2394c38fe015db/log.txt.gz 2026-04-03 00:50:53,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ea99e2394c38fe015d8 HTTP/1.1" 200 None 2026-04-03 00:50:53,530 - lava_callback - INFO - Completed processing callback for node 69cf0ea99e2394c38fe015d8 2026-04-03 00:50:53,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:54,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:54,502 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0eab9e2394c38fe015db/lava-logs.yaml 2026-04-03 00:50:54,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:50:55,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:50:55,160 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0eab9e2394c38fe015db/lava_callback.json.gz 2026-04-03 00:50:55,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:55,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0eab9e2394c38fe015db HTTP/1.1" 200 None 2026-04-03 00:50:55,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:55,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:50:55,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:50:55,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0eab9e2394c38fe015db HTTP/1.1" 200 None 2026-04-03 00:50:55,283 - lava_callback - INFO - Completed processing callback for node 69cf0eab9e2394c38fe015db INFO: 172.21.0.1:38646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49756 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:51:10,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:10,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 00:51:12,003 - lava_callback - INFO - Received callback for node 69cc6bf38cd9cbcdc9459d66 INFO: 172.21.0.1:43732 - "POST /node/69cc6bf38cd9cbcdc9459d66 HTTP/1.0" 202 Accepted 2026-04-03 00:51:12,012 - lava_callback - INFO - Starting async job submit for node 69cc6bf38cd9cbcdc9459d66 2026-04-03 00:51:12,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:12,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6bf38cd9cbcdc9459d66 HTTP/1.1" 200 None 2026-04-03 00:51:12,054 - lava_callback - DEBUG - Processing node 69cc6bf38cd9cbcdc9459d66: baseline-arm-clabbe 2026-04-03 00:51:12,055 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6bf38cd9cbcdc9459d66 2026-04-03 00:51:12,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:12,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6bf38cd9cbcdc9459d66 HTTP/1.1" 200 None 2026-04-03 00:51:12,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:12,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 00:51:12,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:12,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6bf38cd9cbcdc9459d66 HTTP/1.1" 200 None 2026-04-03 00:51:12,161 - lava_callback - ERROR - Error processing callback for node 69cc6bf38cd9cbcdc9459d66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43756 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:51:21,140 - lava_callback - INFO - Received callback for node 69cf0ea49e2394c38fe015c3 INFO: 172.21.0.1:52928 - "POST /node/69cf0ea49e2394c38fe015c3 HTTP/1.0" 202 Accepted 2026-04-03 00:51:21,150 - lava_callback - INFO - Starting async job submit for node 69cf0ea49e2394c38fe015c3 2026-04-03 00:51:21,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:21,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ea49e2394c38fe015c3 HTTP/1.1" 200 None 2026-04-03 00:51:21,230 - lava_callback - DEBUG - Processing node 69cf0ea49e2394c38fe015c3: baseline-arm-kontron 2026-04-03 00:51:21,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:51:22,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:51:22,377 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf0ea49e2394c38fe015c3/log.txt.gz 2026-04-03 00:51:22,388 - lava_callback - INFO - Received callback for node 69cf0ea59e2394c38fe015cd INFO: 172.21.0.1:52944 - "POST /node/69cf0ea59e2394c38fe015cd HTTP/1.0" 202 Accepted 2026-04-03 00:51:22,438 - lava_callback - INFO - Starting async job submit for node 69cf0ea59e2394c38fe015cd 2026-04-03 00:51:22,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:22,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ea59e2394c38fe015cd HTTP/1.1" 200 None 2026-04-03 00:51:22,573 - lava_callback - DEBUG - Processing node 69cf0ea59e2394c38fe015cd: baseline-arm-kontron 2026-04-03 00:51:22,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:51:23,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:51:23,910 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:51:23,912 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf0ea59e2394c38fe015cd/log.txt.gz 2026-04-03 00:51:24,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:51:24,362 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:51:24,365 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf0ea49e2394c38fe015c3/lava-logs.yaml 2026-04-03 00:51:24,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:51:25,217 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:51:25,219 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf0ea49e2394c38fe015c3/lava_callback.json.gz 2026-04-03 00:51:25,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:25,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ea49e2394c38fe015c3 HTTP/1.1" 200 None 2026-04-03 00:51:25,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:25,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:51:25,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:25,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ea49e2394c38fe015c3 HTTP/1.1" 200 None 2026-04-03 00:51:25,382 - lava_callback - INFO - Completed processing callback for node 69cf0ea49e2394c38fe015c3 2026-04-03 00:51:25,576 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:51:25,579 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf0ea59e2394c38fe015cd/lava-logs.yaml 2026-04-03 00:51:25,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:52956 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:51:26,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:51:26,762 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69cf0ea59e2394c38fe015cd/lava_callback.json.gz 2026-04-03 00:51:26,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:26,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ea59e2394c38fe015cd HTTP/1.1" 200 None 2026-04-03 00:51:26,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:26,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:51:26,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:26,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ea59e2394c38fe015cd HTTP/1.1" 200 None 2026-04-03 00:51:26,916 - lava_callback - INFO - Completed processing callback for node 69cf0ea59e2394c38fe015cd 2026-04-03 00:51:30,166 - lava_callback - INFO - Received callback for node 69cf0e9a9e2394c38fe015ae INFO: 172.21.0.1:53316 - "POST /node/69cf0e9a9e2394c38fe015ae HTTP/1.0" 202 Accepted 2026-04-03 00:51:30,176 - lava_callback - INFO - Starting async job submit for node 69cf0e9a9e2394c38fe015ae 2026-04-03 00:51:30,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:30,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e9a9e2394c38fe015ae HTTP/1.1" 200 None 2026-04-03 00:51:30,260 - lava_callback - DEBUG - Processing node 69cf0e9a9e2394c38fe015ae: baseline-arm-clabbe 2026-04-03 00:51:30,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:51:31,375 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:51:31,377 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf0e9a9e2394c38fe015ae/log.txt.gz 2026-04-03 00:51:31,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:51:33,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:51:33,078 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf0e9a9e2394c38fe015ae/lava-logs.yaml 2026-04-03 00:51:33,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:51:34,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:51:34,204 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf0e9a9e2394c38fe015ae/lava_callback.json.gz 2026-04-03 00:51:34,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:34,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e9a9e2394c38fe015ae HTTP/1.1" 200 None 2026-04-03 00:51:34,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:34,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:51:34,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:34,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0e9a9e2394c38fe015ae HTTP/1.1" 200 None 2026-04-03 00:51:34,350 - lava_callback - INFO - Completed processing callback for node 69cf0e9a9e2394c38fe015ae INFO: 172.21.0.1:53324 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:51:40,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:40,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60202 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:51:51,935 - lava_callback - INFO - Received callback for node 69cc6c1f8cd9cbcdc9459d6e INFO: 172.21.0.1:33278 - "POST /node/69cc6c1f8cd9cbcdc9459d6e HTTP/1.0" 202 Accepted 2026-04-03 00:51:51,946 - lava_callback - INFO - Starting async job submit for node 69cc6c1f8cd9cbcdc9459d6e 2026-04-03 00:51:51,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:51,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6c1f8cd9cbcdc9459d6e HTTP/1.1" 200 None 2026-04-03 00:51:51,990 - lava_callback - DEBUG - Processing node 69cc6c1f8cd9cbcdc9459d6e: baseline-arm-clabbe 2026-04-03 00:51:51,990 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6c1f8cd9cbcdc9459d6e 2026-04-03 00:51:51,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:52,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6c1f8cd9cbcdc9459d6e HTTP/1.1" 200 None 2026-04-03 00:51:52,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:52,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 00:51:52,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:51:52,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6c1f8cd9cbcdc9459d6e HTTP/1.1" 200 None 2026-04-03 00:51:52,089 - lava_callback - ERROR - Error processing callback for node 69cc6c1f8cd9cbcdc9459d6e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51734 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:52:34,202 - lava_callback - INFO - Received callback for node 69cc6c4b8cd9cbcdc9459d7d INFO: 172.21.0.1:47688 - "POST /node/69cc6c4b8cd9cbcdc9459d7d HTTP/1.0" 202 Accepted 2026-04-03 00:52:34,211 - lava_callback - INFO - Starting async job submit for node 69cc6c4b8cd9cbcdc9459d7d 2026-04-03 00:52:34,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:52:34,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6c4b8cd9cbcdc9459d7d HTTP/1.1" 200 None 2026-04-03 00:52:34,242 - lava_callback - DEBUG - Processing node 69cc6c4b8cd9cbcdc9459d7d: baseline-arm-clabbe 2026-04-03 00:52:34,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6c4b8cd9cbcdc9459d7d 2026-04-03 00:52:34,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:52:34,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6c4b8cd9cbcdc9459d7d HTTP/1.1" 200 None 2026-04-03 00:52:34,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:52:34,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 00:52:34,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:52:34,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6c4b8cd9cbcdc9459d7d HTTP/1.1" 200 None 2026-04-03 00:52:34,341 - lava_callback - ERROR - Error processing callback for node 69cc6c4b8cd9cbcdc9459d7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36362 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:53:09,190 - lava_callback - INFO - Received callback for node 69cf0e9f9e2394c38fe015ba INFO: 172.21.0.1:36374 - "POST /node/69cf0e9f9e2394c38fe015ba HTTP/1.0" 202 Accepted 2026-04-03 00:53:09,200 - lava_callback - INFO - Starting async job submit for node 69cf0e9f9e2394c38fe015ba 2026-04-03 00:53:09,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:09,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e9f9e2394c38fe015ba HTTP/1.1" 200 None 2026-04-03 00:53:09,279 - lava_callback - DEBUG - Processing node 69cf0e9f9e2394c38fe015ba: baseline-arm-clabbe 2026-04-03 00:53:09,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:53:10,065 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:10,067 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf0e9f9e2394c38fe015ba/log.txt.gz 2026-04-03 00:53:10,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:53:10,650 - lava_callback - INFO - Received callback for node 69cf0f699e2394c38fe01712 INFO: 172.21.0.1:60222 - "POST /node/69cf0f699e2394c38fe01712 HTTP/1.0" 202 Accepted 2026-04-03 00:53:10,660 - lava_callback - INFO - Starting async job submit for node 69cf0f699e2394c38fe01712 2026-04-03 00:53:10,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:10,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0f699e2394c38fe01712 HTTP/1.1" 200 None 2026-04-03 00:53:10,704 - lava_callback - DEBUG - Processing node 69cf0f699e2394c38fe01712: baseline-arm 2026-04-03 00:53:10,714 - lava_callback - WARNING - Failed to upload log 2026-04-03 00:53:10,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:53:10,902 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:10,905 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf0e9f9e2394c38fe015ba/lava-logs.yaml 2026-04-03 00:53:10,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:53:11,104 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:11,107 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0f699e2394c38fe01712/lava-logs.yaml 2026-04-03 00:53:11,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:53:11,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:11,433 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf0e9f9e2394c38fe015ba/lava_callback.json.gz 2026-04-03 00:53:11,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:11,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:11,478 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0f699e2394c38fe01712/lava_callback.json.gz 2026-04-03 00:53:11,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:11,506 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e9f9e2394c38fe015ba HTTP/1.1" 200 None 2026-04-03 00:53:11,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:11,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0f699e2394c38fe01712 HTTP/1.1" 200 None 2026-04-03 00:53:11,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:11,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:53:11,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:11,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:53:11,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:11,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0f699e2394c38fe01712 HTTP/1.1" 200 None 2026-04-03 00:53:11,601 - lava_callback - INFO - Completed processing callback for node 69cf0f699e2394c38fe01712 2026-04-03 00:53:11,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0e9f9e2394c38fe015ba HTTP/1.1" 200 None 2026-04-03 00:53:11,606 - lava_callback - INFO - Completed processing callback for node 69cf0e9f9e2394c38fe015ba INFO: 172.21.0.1:60232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60234 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:53:25,022 - lava_callback - INFO - Received callback for node 69cf0ed69e2394c38fe01646 INFO: 172.21.0.1:40600 - "POST /node/69cf0ed69e2394c38fe01646 HTTP/1.0" 202 Accepted 2026-04-03 00:53:25,031 - lava_callback - INFO - Starting async job submit for node 69cf0ed69e2394c38fe01646 2026-04-03 00:53:25,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:25,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ed69e2394c38fe01646 HTTP/1.1" 200 None 2026-04-03 00:53:25,077 - lava_callback - DEBUG - Processing node 69cf0ed69e2394c38fe01646: ltp-dio 2026-04-03 00:53:25,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:53:25,852 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:25,854 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69cf0ed69e2394c38fe01646/log.txt.gz 2026-04-03 00:53:26,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40608 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:53:26,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:26,896 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69cf0ed69e2394c38fe01646/lava-logs.yaml 2026-04-03 00:53:26,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:53:27,346 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:27,348 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69cf0ed69e2394c38fe01646/lava_callback.json.gz 2026-04-03 00:53:27,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:27,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ed69e2394c38fe01646 HTTP/1.1" 200 None 2026-04-03 00:53:27,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:27,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:53:27,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:27,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ed69e2394c38fe01646 HTTP/1.1" 200 None 2026-04-03 00:53:27,494 - lava_callback - INFO - Completed processing callback for node 69cf0ed69e2394c38fe01646 INFO: 172.21.0.1:51832 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:53:40,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:40,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:53:52,091 - lava_callback - INFO - Received callback for node 69cf0be09e2394c38fe013ac INFO: 172.21.0.1:45990 - "POST /node/69cf0be09e2394c38fe013ac HTTP/1.0" 202 Accepted 2026-04-03 00:53:52,098 - lava_callback - INFO - Starting async job submit for node 69cf0be09e2394c38fe013ac 2026-04-03 00:53:52,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:52,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0be09e2394c38fe013ac HTTP/1.1" 200 None 2026-04-03 00:53:52,167 - lava_callback - DEBUG - Processing node 69cf0be09e2394c38fe013ac: baseline-x86 2026-04-03 00:53:53,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:53:53,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:53,280 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be09e2394c38fe013ac/log.txt.gz 2026-04-03 00:53:53,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:53:53,990 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:53,992 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be09e2394c38fe013ac/lava-logs.yaml 2026-04-03 00:53:54,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:53:54,176 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:53:54,178 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf0be09e2394c38fe013ac/lava_callback.json.gz DEBUG: Setup failed for 69cf0be09e2394c38fe013ac. Transitting job node result: incomplete -> fail 2026-04-03 00:53:54,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:54,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0be09e2394c38fe013ac HTTP/1.1" 200 None 2026-04-03 00:53:54,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:54,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf071c9e2394c38fe00f6d HTTP/1.1" 200 None 2026-04-03 00:53:54,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:53:54,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0be09e2394c38fe013ac HTTP/1.1" 200 None 2026-04-03 00:53:54,482 - lava_callback - INFO - Completed processing callback for node 69cf0be09e2394c38fe013ac INFO: 172.21.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33292 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:54:10,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:10,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 00:54:12,122 - lava_callback - INFO - Received callback for node 69cc6cac8cd9cbcdc9459d8f INFO: 172.21.0.1:33510 - "POST /node/69cc6cac8cd9cbcdc9459d8f HTTP/1.0" 202 Accepted 2026-04-03 00:54:12,128 - lava_callback - INFO - Starting async job submit for node 69cc6cac8cd9cbcdc9459d8f 2026-04-03 00:54:12,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:12,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6cac8cd9cbcdc9459d8f HTTP/1.1" 200 None 2026-04-03 00:54:12,144 - lava_callback - DEBUG - Processing node 69cc6cac8cd9cbcdc9459d8f: baseline-arm-clabbe 2026-04-03 00:54:12,144 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6cac8cd9cbcdc9459d8f 2026-04-03 00:54:12,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:12,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6cac8cd9cbcdc9459d8f HTTP/1.1" 200 None 2026-04-03 00:54:12,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:12,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 00:54:12,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:12,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6cac8cd9cbcdc9459d8f HTTP/1.1" 200 None 2026-04-03 00:54:12,202 - lava_callback - ERROR - Error processing callback for node 69cc6cac8cd9cbcdc9459d8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33516 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:54:17,623 - lava_callback - INFO - Received callback for node 69cf0f9d9e2394c38fe01746 INFO: 172.21.0.1:33526 - "POST /node/69cf0f9d9e2394c38fe01746 HTTP/1.0" 202 Accepted 2026-04-03 00:54:17,630 - lava_callback - INFO - Starting async job submit for node 69cf0f9d9e2394c38fe01746 2026-04-03 00:54:17,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:17,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0f9d9e2394c38fe01746 HTTP/1.1" 200 None 2026-04-03 00:54:17,663 - lava_callback - DEBUG - Processing node 69cf0f9d9e2394c38fe01746: baseline-arm64 2026-04-03 00:54:17,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:18,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:18,132 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0f9d9e2394c38fe01746/log.txt.gz 2026-04-03 00:54:18,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:18,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:18,407 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0f9d9e2394c38fe01746/lava-logs.yaml 2026-04-03 00:54:18,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:18,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:18,556 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0f9d9e2394c38fe01746/lava_callback.json.gz 2026-04-03 00:54:18,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:18,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0f9d9e2394c38fe01746 HTTP/1.1" 200 None 2026-04-03 00:54:18,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:18,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:54:18,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:18,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0f9d9e2394c38fe01746 HTTP/1.1" 200 None 2026-04-03 00:54:18,652 - lava_callback - INFO - Completed processing callback for node 69cf0f9d9e2394c38fe01746 INFO: 172.21.0.1:40524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53302 - "GET /node/693b3ada4b47c3f87266d862%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:53306 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:54:39,437 - lava_callback - INFO - Received callback for node 69cf0fba9e2394c38fe017ba INFO: 172.21.0.1:53318 - "POST /node/69cf0fba9e2394c38fe017ba HTTP/1.0" 202 Accepted 2026-04-03 00:54:39,447 - lava_callback - INFO - Starting async job submit for node 69cf0fba9e2394c38fe017ba 2026-04-03 00:54:39,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:39,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fba9e2394c38fe017ba HTTP/1.1" 200 None 2026-04-03 00:54:39,466 - lava_callback - DEBUG - Processing node 69cf0fba9e2394c38fe017ba: baseline-nfs-arm64 2026-04-03 00:54:39,468 - lava_callback - WARNING - Failed to upload log 2026-04-03 00:54:39,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:39,657 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:39,659 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fba9e2394c38fe017ba/lava-logs.yaml 2026-04-03 00:54:39,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:39,807 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:39,809 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fba9e2394c38fe017ba/lava_callback.json.gz 2026-04-03 00:54:39,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:39,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fba9e2394c38fe017ba HTTP/1.1" 200 None 2026-04-03 00:54:39,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:39,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:54:39,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:39,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fba9e2394c38fe017ba HTTP/1.1" 200 None 2026-04-03 00:54:39,892 - lava_callback - INFO - Completed processing callback for node 69cf0fba9e2394c38fe017ba 2026-04-03 00:54:40,664 - lava_callback - INFO - Received callback for node 69cf0faa9e2394c38fe0178e INFO: 172.21.0.1:46176 - "POST /node/69cf0faa9e2394c38fe0178e HTTP/1.0" 202 Accepted 2026-04-03 00:54:40,677 - lava_callback - INFO - Starting async job submit for node 69cf0faa9e2394c38fe0178e 2026-04-03 00:54:40,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:40,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0faa9e2394c38fe0178e HTTP/1.1" 200 None 2026-04-03 00:54:40,740 - lava_callback - DEBUG - Processing node 69cf0faa9e2394c38fe0178e: baseline-arm64 2026-04-03 00:54:40,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:40,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:40,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 00:54:41,016 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:41,018 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0faa9e2394c38fe0178e/log.txt.gz 2026-04-03 00:54:41,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:41,290 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:41,292 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0faa9e2394c38fe0178e/lava-logs.yaml 2026-04-03 00:54:41,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:41,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:41,432 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0faa9e2394c38fe0178e/lava_callback.json.gz 2026-04-03 00:54:41,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:41,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0faa9e2394c38fe0178e HTTP/1.1" 200 None 2026-04-03 00:54:41,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:41,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:54:41,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:41,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0faa9e2394c38fe0178e HTTP/1.1" 200 None 2026-04-03 00:54:41,550 - lava_callback - INFO - Completed processing callback for node 69cf0faa9e2394c38fe0178e INFO: 172.21.0.1:46190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46196 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:54:48,262 - lava_callback - INFO - Received callback for node 69cf0ea09e2394c38fe015bd INFO: 172.21.0.1:46206 - "POST /node/69cf0ea09e2394c38fe015bd HTTP/1.0" 202 Accepted 2026-04-03 00:54:48,273 - lava_callback - INFO - Starting async job submit for node 69cf0ea09e2394c38fe015bd 2026-04-03 00:54:48,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:48,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ea09e2394c38fe015bd HTTP/1.1" 200 None 2026-04-03 00:54:48,373 - lava_callback - DEBUG - Processing node 69cf0ea09e2394c38fe015bd: baseline-arm-clabbe 2026-04-03 00:54:48,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:48,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:48,827 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf0ea09e2394c38fe015bd/log.txt.gz 2026-04-03 00:54:48,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:49,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:49,228 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf0ea09e2394c38fe015bd/lava-logs.yaml 2026-04-03 00:54:49,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:49,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:49,369 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69cf0ea09e2394c38fe015bd/lava_callback.json.gz 2026-04-03 00:54:49,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:49,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ea09e2394c38fe015bd HTTP/1.1" 200 None 2026-04-03 00:54:49,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:49,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 00:54:49,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:49,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ea09e2394c38fe015bd HTTP/1.1" 200 None 2026-04-03 00:54:49,449 - lava_callback - INFO - Completed processing callback for node 69cf0ea09e2394c38fe015bd 2026-04-03 00:54:52,964 - lava_callback - INFO - Received callback for node 69cf0f9f9e2394c38fe0174c INFO: 172.21.0.1:38716 - "POST /node/69cf0f9f9e2394c38fe0174c HTTP/1.0" 202 Accepted 2026-04-03 00:54:52,971 - lava_callback - INFO - Starting async job submit for node 69cf0f9f9e2394c38fe0174c 2026-04-03 00:54:52,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:53,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0f9f9e2394c38fe0174c HTTP/1.1" 200 None 2026-04-03 00:54:53,013 - lava_callback - DEBUG - Processing node 69cf0f9f9e2394c38fe0174c: baseline-arm64 2026-04-03 00:54:53,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:53,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:53,599 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0f9f9e2394c38fe0174c/log.txt.gz 2026-04-03 00:54:53,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:54,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:54,075 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0f9f9e2394c38fe0174c/lava-logs.yaml 2026-04-03 00:54:54,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:54:54,303 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:54:54,305 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0f9f9e2394c38fe0174c/lava_callback.json.gz 2026-04-03 00:54:54,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:54,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0f9f9e2394c38fe0174c HTTP/1.1" 200 None 2026-04-03 00:54:54,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:54,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:54:54,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:54:54,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0f9f9e2394c38fe0174c HTTP/1.1" 200 None 2026-04-03 00:54:54,423 - lava_callback - INFO - Completed processing callback for node 69cf0f9f9e2394c38fe0174c INFO: 172.21.0.1:38718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40578 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:55:10,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:55:10,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55554 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:55:28,865 - lava_callback - INFO - Received callback for node 69cf0f9e9e2394c38fe01749 INFO: 172.21.0.1:55560 - "POST /node/69cf0f9e9e2394c38fe01749 HTTP/1.0" 202 Accepted 2026-04-03 00:55:28,875 - lava_callback - INFO - Starting async job submit for node 69cf0f9e9e2394c38fe01749 2026-04-03 00:55:28,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:55:28,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0f9e9e2394c38fe01749 HTTP/1.1" 200 None 2026-04-03 00:55:28,913 - lava_callback - DEBUG - Processing node 69cf0f9e9e2394c38fe01749: baseline-arm64 2026-04-03 00:55:29,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:55:29,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:55:29,349 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0f9e9e2394c38fe01749/log.txt.gz 2026-04-03 00:55:29,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:55:29,773 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:55:29,775 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0f9e9e2394c38fe01749/lava-logs.yaml 2026-04-03 00:55:29,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:55:29,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:55:29,982 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0f9e9e2394c38fe01749/lava_callback.json.gz 2026-04-03 00:55:30,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:55:30,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0f9e9e2394c38fe01749 HTTP/1.1" 200 None 2026-04-03 00:55:30,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:55:30,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:55:30,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:55:30,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0f9e9e2394c38fe01749 HTTP/1.1" 200 None 2026-04-03 00:55:30,089 - lava_callback - INFO - Completed processing callback for node 69cf0f9e9e2394c38fe01749 INFO: 172.21.0.1:40312 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:55:40,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:55:40,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33618 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:56:02,134 - lava_callback - INFO - Received callback for node 69cf0fe49e2394c38fe0189a INFO: 172.21.0.1:59516 - "POST /node/69cf0fe49e2394c38fe0189a HTTP/1.0" 202 Accepted 2026-04-03 00:56:02,140 - lava_callback - INFO - Starting async job submit for node 69cf0fe49e2394c38fe0189a 2026-04-03 00:56:02,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:02,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fe49e2394c38fe0189a HTTP/1.1" 200 None 2026-04-03 00:56:02,180 - lava_callback - DEBUG - Processing node 69cf0fe49e2394c38fe0189a: baseline-arm64-foundriesio 2026-04-03 00:56:02,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:02,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:02,623 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf0fe49e2394c38fe0189a/log.txt.gz 2026-04-03 00:56:02,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:03,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:03,083 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf0fe49e2394c38fe0189a/lava-logs.yaml 2026-04-03 00:56:03,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:03,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:03,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf0fe49e2394c38fe0189a/lava_callback.json.gz 2026-04-03 00:56:03,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:03,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fe49e2394c38fe0189a HTTP/1.1" 200 None 2026-04-03 00:56:03,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:03,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:03,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:03,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fe49e2394c38fe0189a HTTP/1.1" 200 None 2026-04-03 00:56:03,363 - lava_callback - INFO - Completed processing callback for node 69cf0fe49e2394c38fe0189a INFO: 172.21.0.1:59526 - "GET /node/6938d841e767fbc36e03ef7b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:59528 - "GET /node/6928c7ce3a49357e496c8a3a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:59532 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:56:07,984 - lava_callback - INFO - Received callback for node 69cf0fe29e2394c38fe01894 INFO: 172.21.0.1:59544 - "POST /node/69cf0fe29e2394c38fe01894 HTTP/1.0" 202 Accepted 2026-04-03 00:56:07,994 - lava_callback - INFO - Starting async job submit for node 69cf0fe29e2394c38fe01894 2026-04-03 00:56:08,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:08,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fe29e2394c38fe01894 HTTP/1.1" 200 None 2026-04-03 00:56:08,052 - lava_callback - DEBUG - Processing node 69cf0fe29e2394c38fe01894: baseline-arm64-pengutronix 2026-04-03 00:56:08,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:08,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:08,613 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf0fe29e2394c38fe01894/log.txt.gz 2026-04-03 00:56:08,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:09,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:09,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf0fe29e2394c38fe01894/lava-logs.yaml 2026-04-03 00:56:09,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:09,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:09,175 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf0fe29e2394c38fe01894/lava_callback.json.gz 2026-04-03 00:56:09,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:09,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fe29e2394c38fe01894 HTTP/1.1" 200 None 2026-04-03 00:56:09,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:09,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:09,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:09,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fe29e2394c38fe01894 HTTP/1.1" 200 None 2026-04-03 00:56:09,282 - lava_callback - INFO - Completed processing callback for node 69cf0fe29e2394c38fe01894 2026-04-03 00:56:10,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:11,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 00:56:11,944 - lava_callback - INFO - Received callback for node 69cf0fe39e2394c38fe01897 INFO: 172.21.0.1:41576 - "POST /node/69cf0fe39e2394c38fe01897 HTTP/1.0" 202 Accepted 2026-04-03 00:56:11,950 - lava_callback - INFO - Starting async job submit for node 69cf0fe39e2394c38fe01897 2026-04-03 00:56:11,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:11,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fe39e2394c38fe01897 HTTP/1.1" 200 None 2026-04-03 00:56:11,995 - lava_callback - DEBUG - Processing node 69cf0fe39e2394c38fe01897: baseline-arm64-pengutronix 2026-04-03 00:56:12,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:12,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:12,380 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf0fe39e2394c38fe01897/log.txt.gz 2026-04-03 00:56:12,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:12,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:12,766 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf0fe39e2394c38fe01897/lava-logs.yaml 2026-04-03 00:56:12,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:12,908 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:12,910 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69cf0fe39e2394c38fe01897/lava_callback.json.gz 2026-04-03 00:56:12,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:12,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fe39e2394c38fe01897 HTTP/1.1" 200 None 2026-04-03 00:56:12,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:12,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:12,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:12,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fe39e2394c38fe01897 HTTP/1.1" 200 None 2026-04-03 00:56:12,992 - lava_callback - INFO - Completed processing callback for node 69cf0fe39e2394c38fe01897 INFO: 172.21.0.1:41588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41600 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:56:21,277 - lava_callback - INFO - Received callback for node 69cf0fdf9e2394c38fe01889 INFO: 172.21.0.1:33466 - "POST /node/69cf0fdf9e2394c38fe01889 HTTP/1.0" 202 Accepted 2026-04-03 00:56:21,283 - lava_callback - INFO - Starting async job submit for node 69cf0fdf9e2394c38fe01889 2026-04-03 00:56:21,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:21,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fdf9e2394c38fe01889 HTTP/1.1" 200 None 2026-04-03 00:56:21,320 - lava_callback - DEBUG - Processing node 69cf0fdf9e2394c38fe01889: baseline-arm64-kontron 2026-04-03 00:56:21,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:21,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:21,677 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf0fdf9e2394c38fe01889/log.txt.gz 2026-04-03 00:56:21,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:22,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:22,011 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf0fdf9e2394c38fe01889/lava-logs.yaml 2026-04-03 00:56:22,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:22,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:22,176 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf0fdf9e2394c38fe01889/lava_callback.json.gz 2026-04-03 00:56:22,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:22,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fdf9e2394c38fe01889 HTTP/1.1" 200 None 2026-04-03 00:56:22,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:22,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:22,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:22,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fdf9e2394c38fe01889 HTTP/1.1" 200 None 2026-04-03 00:56:22,275 - lava_callback - INFO - Completed processing callback for node 69cf0fdf9e2394c38fe01889 2026-04-03 00:56:22,839 - lava_callback - INFO - Received callback for node 69cf0fe09e2394c38fe01891 INFO: 172.21.0.1:33480 - "POST /node/69cf0fe09e2394c38fe01891 HTTP/1.0" 202 Accepted 2026-04-03 00:56:22,849 - lava_callback - INFO - Starting async job submit for node 69cf0fe09e2394c38fe01891 2026-04-03 00:56:22,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:22,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fe09e2394c38fe01891 HTTP/1.1" 200 None 2026-04-03 00:56:22,901 - lava_callback - DEBUG - Processing node 69cf0fe09e2394c38fe01891: baseline-arm64-kontron 2026-04-03 00:56:23,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:23,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:23,381 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf0fe09e2394c38fe01891/log.txt.gz 2026-04-03 00:56:23,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:23,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:23,756 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf0fe09e2394c38fe01891/lava-logs.yaml 2026-04-03 00:56:23,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:24,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:24,134 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69cf0fe09e2394c38fe01891/lava_callback.json.gz 2026-04-03 00:56:24,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:24,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fe09e2394c38fe01891 HTTP/1.1" 200 None 2026-04-03 00:56:24,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:24,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:24,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:24,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fe09e2394c38fe01891 HTTP/1.1" 200 None 2026-04-03 00:56:24,228 - lava_callback - INFO - Completed processing callback for node 69cf0fe09e2394c38fe01891 2026-04-03 00:56:24,722 - lava_callback - INFO - Received callback for node 69cf0fa19e2394c38fe01753 INFO: 172.21.0.1:33490 - "POST /node/69cf0fa19e2394c38fe01753 HTTP/1.0" 202 Accepted 2026-04-03 00:56:24,770 - lava_callback - INFO - Starting async job submit for node 69cf0fa19e2394c38fe01753 2026-04-03 00:56:24,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:24,817 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa19e2394c38fe01753 HTTP/1.1" 200 None 2026-04-03 00:56:24,818 - lava_callback - DEBUG - Processing node 69cf0fa19e2394c38fe01753: baseline-arm64 2026-04-03 00:56:24,968 - lava_callback - WARNING - Failed to upload log 2026-04-03 00:56:25,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:25,212 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:25,214 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa19e2394c38fe01753/lava-logs.yaml 2026-04-03 00:56:25,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:25,374 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:25,377 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa19e2394c38fe01753/lava_callback.json.gz 2026-04-03 00:56:25,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:25,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa19e2394c38fe01753 HTTP/1.1" 200 None 2026-04-03 00:56:25,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:25,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:25,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:25,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fa19e2394c38fe01753 HTTP/1.1" 200 None 2026-04-03 00:56:25,547 - lava_callback - INFO - Completed processing callback for node 69cf0fa19e2394c38fe01753 2026-04-03 00:56:26,006 - lava_callback - INFO - Received callback for node 69cf0fa29e2394c38fe0175b INFO: 172.21.0.1:33496 - "POST /node/69cf0fa29e2394c38fe0175b HTTP/1.0" 202 Accepted 2026-04-03 00:56:26,017 - lava_callback - INFO - Starting async job submit for node 69cf0fa29e2394c38fe0175b 2026-04-03 00:56:26,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:26,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa29e2394c38fe0175b HTTP/1.1" 200 None 2026-04-03 00:56:26,076 - lava_callback - DEBUG - Processing node 69cf0fa29e2394c38fe0175b: baseline-arm64 2026-04-03 00:56:26,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:26,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:26,362 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa29e2394c38fe0175b/log.txt.gz 2026-04-03 00:56:26,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:33504 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:56:26,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:26,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa29e2394c38fe0175b/lava-logs.yaml 2026-04-03 00:56:26,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:26,895 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:26,897 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa29e2394c38fe0175b/lava_callback.json.gz 2026-04-03 00:56:26,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:26,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa29e2394c38fe0175b HTTP/1.1" 200 None 2026-04-03 00:56:26,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:26,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:26,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:26,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fa29e2394c38fe0175b HTTP/1.1" 200 None 2026-04-03 00:56:27,043 - lava_callback - INFO - Completed processing callback for node 69cf0fa29e2394c38fe0175b 2026-04-03 00:56:28,703 - lava_callback - INFO - Received callback for node 69cf0fe59e2394c38fe0189d INFO: 172.21.0.1:33518 - "POST /node/69cf0fe59e2394c38fe0189d HTTP/1.0" 202 Accepted 2026-04-03 00:56:28,714 - lava_callback - INFO - Starting async job submit for node 69cf0fe59e2394c38fe0189d 2026-04-03 00:56:28,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:28,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fe59e2394c38fe0189d HTTP/1.1" 200 None 2026-04-03 00:56:28,764 - lava_callback - DEBUG - Processing node 69cf0fe59e2394c38fe0189d: baseline-arm64-foundriesio 2026-04-03 00:56:28,804 - lava_callback - INFO - Received callback for node 69cf0fa79e2394c38fe01787 INFO: 172.21.0.1:33534 - "POST /node/69cf0fa79e2394c38fe01787 HTTP/1.0" 202 Accepted 2026-04-03 00:56:28,835 - lava_callback - INFO - Starting async job submit for node 69cf0fa79e2394c38fe01787 2026-04-03 00:56:28,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:28,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa79e2394c38fe01787 HTTP/1.1" 200 None 2026-04-03 00:56:29,000 - lava_callback - DEBUG - Processing node 69cf0fa79e2394c38fe01787: baseline-arm64 2026-04-03 00:56:29,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:29,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:29,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:29,754 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf0fe59e2394c38fe0189d/log.txt.gz 2026-04-03 00:56:29,820 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:29,853 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa79e2394c38fe01787/log.txt.gz 2026-04-03 00:56:30,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:30,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:30,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:30,478 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf0fe59e2394c38fe0189d/lava-logs.yaml 2026-04-03 00:56:30,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:30,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:30,556 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa79e2394c38fe01787/lava-logs.yaml 2026-04-03 00:56:30,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:30,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:30,679 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69cf0fe59e2394c38fe0189d/lava_callback.json.gz 2026-04-03 00:56:30,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:30,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fe59e2394c38fe0189d HTTP/1.1" 200 None 2026-04-03 00:56:30,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:30,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:30,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:30,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:30,754 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa79e2394c38fe01787/lava_callback.json.gz 2026-04-03 00:56:30,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:30,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fe59e2394c38fe0189d HTTP/1.1" 200 None 2026-04-03 00:56:30,812 - lava_callback - INFO - Completed processing callback for node 69cf0fe59e2394c38fe0189d 2026-04-03 00:56:30,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa79e2394c38fe01787 HTTP/1.1" 200 None 2026-04-03 00:56:30,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:30,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:30,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:30,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fa79e2394c38fe01787 HTTP/1.1" 200 None 2026-04-03 00:56:30,925 - lava_callback - INFO - Completed processing callback for node 69cf0fa79e2394c38fe01787 2026-04-03 00:56:31,269 - lava_callback - INFO - Received callback for node 69cf0fae9e2394c38fe0179a INFO: 172.21.0.1:54496 - "POST /node/69cf0fae9e2394c38fe0179a HTTP/1.0" 202 Accepted 2026-04-03 00:56:31,276 - lava_callback - INFO - Starting async job submit for node 69cf0fae9e2394c38fe0179a 2026-04-03 00:56:31,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:31,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fae9e2394c38fe0179a HTTP/1.1" 200 None 2026-04-03 00:56:31,313 - lava_callback - DEBUG - Processing node 69cf0fae9e2394c38fe0179a: baseline-arm64 2026-04-03 00:56:31,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:31,792 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:31,794 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fae9e2394c38fe0179a/log.txt.gz 2026-04-03 00:56:31,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:32,147 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:32,149 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fae9e2394c38fe0179a/lava-logs.yaml 2026-04-03 00:56:32,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:32,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:32,446 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fae9e2394c38fe0179a/lava_callback.json.gz 2026-04-03 00:56:32,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:32,485 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fae9e2394c38fe0179a HTTP/1.1" 200 None 2026-04-03 00:56:32,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:32,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:32,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:32,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fae9e2394c38fe0179a HTTP/1.1" 200 None 2026-04-03 00:56:32,526 - lava_callback - INFO - Completed processing callback for node 69cf0fae9e2394c38fe0179a INFO: 172.21.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54520 - "GET /node/693b0315fee8383e92b6f2de%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-03 00:56:37,572 - lava_callback - INFO - Received callback for node 69cf0fb09e2394c38fe0179d INFO: 172.21.0.1:54536 - "POST /node/69cf0fb09e2394c38fe0179d HTTP/1.0" 202 Accepted 2026-04-03 00:56:37,579 - lava_callback - INFO - Starting async job submit for node 69cf0fb09e2394c38fe0179d 2026-04-03 00:56:37,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:37,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb09e2394c38fe0179d HTTP/1.1" 200 None 2026-04-03 00:56:37,616 - lava_callback - DEBUG - Processing node 69cf0fb09e2394c38fe0179d: baseline-arm64 2026-04-03 00:56:37,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:38,026 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:38,028 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fb09e2394c38fe0179d/log.txt.gz 2026-04-03 00:56:38,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:38,357 - lava_callback - INFO - Received callback for node 69cf0fa69e2394c38fe01784 INFO: 172.21.0.1:54540 - "POST /node/69cf0fa69e2394c38fe01784 HTTP/1.0" 202 Accepted 2026-04-03 00:56:38,367 - lava_callback - INFO - Starting async job submit for node 69cf0fa69e2394c38fe01784 2026-04-03 00:56:38,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:38,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa69e2394c38fe01784 HTTP/1.1" 200 None 2026-04-03 00:56:38,421 - lava_callback - DEBUG - Processing node 69cf0fa69e2394c38fe01784: baseline-arm64 2026-04-03 00:56:38,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:38,527 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fb09e2394c38fe0179d/lava-logs.yaml 2026-04-03 00:56:38,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:38,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:38,922 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:38,924 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fb09e2394c38fe0179d/lava_callback.json.gz 2026-04-03 00:56:38,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:38,973 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa69e2394c38fe01784/log.txt.gz 2026-04-03 00:56:38,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:39,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb09e2394c38fe0179d HTTP/1.1" 200 None 2026-04-03 00:56:39,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:39,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:39,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:39,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fb09e2394c38fe0179d HTTP/1.1" 200 None 2026-04-03 00:56:39,242 - lava_callback - INFO - Completed processing callback for node 69cf0fb09e2394c38fe0179d 2026-04-03 00:56:39,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:39,755 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:39,757 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa69e2394c38fe01784/lava-logs.yaml 2026-04-03 00:56:39,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:39,979 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:39,981 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa69e2394c38fe01784/lava_callback.json.gz 2026-04-03 00:56:40,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:40,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa69e2394c38fe01784 HTTP/1.1" 200 None 2026-04-03 00:56:40,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:40,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:40,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:40,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fa69e2394c38fe01784 HTTP/1.1" 200 None 2026-04-03 00:56:40,064 - lava_callback - INFO - Completed processing callback for node 69cf0fa69e2394c38fe01784 2026-04-03 00:56:41,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:41,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-03 00:56:43,936 - lava_callback - INFO - Received callback for node 69cf0fa59e2394c38fe01781 INFO: 172.21.0.1:58194 - "POST /node/69cf0fa59e2394c38fe01781 HTTP/1.0" 202 Accepted 2026-04-03 00:56:43,943 - lava_callback - INFO - Starting async job submit for node 69cf0fa59e2394c38fe01781 2026-04-03 00:56:43,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:43,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa59e2394c38fe01781 HTTP/1.1" 200 None 2026-04-03 00:56:43,984 - lava_callback - DEBUG - Processing node 69cf0fa59e2394c38fe01781: baseline-arm64 2026-04-03 00:56:44,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:44,415 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:44,417 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa59e2394c38fe01781/log.txt.gz 2026-04-03 00:56:44,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:58198 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:56:45,070 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:45,072 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa59e2394c38fe01781/lava-logs.yaml 2026-04-03 00:56:45,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:45,222 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:45,223 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa59e2394c38fe01781/lava_callback.json.gz 2026-04-03 00:56:45,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:45,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa59e2394c38fe01781 HTTP/1.1" 200 None 2026-04-03 00:56:45,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:45,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:45,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:45,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fa59e2394c38fe01781 HTTP/1.1" 200 None 2026-04-03 00:56:45,330 - lava_callback - INFO - Completed processing callback for node 69cf0fa59e2394c38fe01781 INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:56:47,281 - lava_callback - INFO - Received callback for node 69cf0fb79e2394c38fe017b4 INFO: 172.21.0.1:58210 - "POST /node/69cf0fb79e2394c38fe017b4 HTTP/1.0" 202 Accepted 2026-04-03 00:56:47,288 - lava_callback - INFO - Starting async job submit for node 69cf0fb79e2394c38fe017b4 2026-04-03 00:56:47,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:47,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb79e2394c38fe017b4 HTTP/1.1" 200 None 2026-04-03 00:56:47,327 - lava_callback - DEBUG - Processing node 69cf0fb79e2394c38fe017b4: baseline-nfs-arm64 2026-04-03 00:56:47,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:47,951 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:47,953 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb79e2394c38fe017b4/log.txt.gz 2026-04-03 00:56:48,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:48,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:48,411 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb79e2394c38fe017b4/lava-logs.yaml 2026-04-03 00:56:48,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:48,564 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:48,567 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb79e2394c38fe017b4/lava_callback.json.gz 2026-04-03 00:56:48,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:48,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb79e2394c38fe017b4 HTTP/1.1" 200 None 2026-04-03 00:56:48,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:48,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:48,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:48,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fb79e2394c38fe017b4 HTTP/1.1" 200 None 2026-04-03 00:56:48,666 - lava_callback - INFO - Completed processing callback for node 69cf0fb79e2394c38fe017b4 2026-04-03 00:56:50,750 - lava_callback - INFO - Received callback for node 69cf0fa89e2394c38fe0178b INFO: 172.21.0.1:52272 - "POST /node/69cf0fa89e2394c38fe0178b HTTP/1.0" 202 Accepted 2026-04-03 00:56:50,756 - lava_callback - INFO - Starting async job submit for node 69cf0fa89e2394c38fe0178b 2026-04-03 00:56:50,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:50,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa89e2394c38fe0178b HTTP/1.1" 200 None 2026-04-03 00:56:50,793 - lava_callback - DEBUG - Processing node 69cf0fa89e2394c38fe0178b: baseline-arm64 2026-04-03 00:56:51,185 - lava_callback - INFO - Received callback for node 69cf0fab9e2394c38fe01791 INFO: 172.21.0.1:52288 - "POST /node/69cf0fab9e2394c38fe01791 HTTP/1.0" 202 Accepted 2026-04-03 00:56:51,212 - lava_callback - INFO - Starting async job submit for node 69cf0fab9e2394c38fe01791 2026-04-03 00:56:51,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:51,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:51,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fab9e2394c38fe01791 HTTP/1.1" 200 None 2026-04-03 00:56:51,284 - lava_callback - DEBUG - Processing node 69cf0fab9e2394c38fe01791: baseline-arm64 2026-04-03 00:56:51,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:51,493 - lava_callback - INFO - Received callback for node 69cf0fdc9e2394c38fe01848 2026-04-03 00:56:51,517 - lava_callback - INFO - Starting async job submit for node 69cf0fdc9e2394c38fe01848 INFO: 172.21.0.1:52302 - "POST /node/69cf0fdc9e2394c38fe01848 HTTP/1.0" 202 Accepted 2026-04-03 00:56:51,578 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa89e2394c38fe0178b/log.txt.gz 2026-04-03 00:56:51,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:51,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:51,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fdc9e2394c38fe01848 HTTP/1.1" 200 None 2026-04-03 00:56:51,701 - lava_callback - DEBUG - Processing node 69cf0fdc9e2394c38fe01848: baseline-arm64-clabbe 2026-04-03 00:56:51,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:51,999 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fab9e2394c38fe01791/log.txt.gz 2026-04-03 00:56:52,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:52,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:52,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:52,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:52,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cf0fdc9e2394c38fe01848/log.txt.gz 2026-04-03 00:56:52,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:52,694 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:52,732 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fab9e2394c38fe01791/lava-logs.yaml 2026-04-03 00:56:52,733 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa89e2394c38fe0178b/lava-logs.yaml 2026-04-03 00:56:52,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:52,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:52,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:52,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:52,972 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fab9e2394c38fe01791/lava_callback.json.gz 2026-04-03 00:56:52,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:53,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fab9e2394c38fe01791 HTTP/1.1" 200 None 2026-04-03 00:56:53,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:53,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:53,025 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cf0fdc9e2394c38fe01848/lava-logs.yaml 2026-04-03 00:56:53,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:53,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:53,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:53,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:53,055 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa89e2394c38fe0178b/lava_callback.json.gz 2026-04-03 00:56:53,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:53,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fab9e2394c38fe01791 HTTP/1.1" 200 None 2026-04-03 00:56:53,113 - lava_callback - INFO - Completed processing callback for node 69cf0fab9e2394c38fe01791 2026-04-03 00:56:53,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa89e2394c38fe0178b HTTP/1.1" 200 None 2026-04-03 00:56:53,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:53,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:53,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:53,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:53,211 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69cf0fdc9e2394c38fe01848/lava_callback.json.gz 2026-04-03 00:56:53,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fa89e2394c38fe0178b HTTP/1.1" 200 None 2026-04-03 00:56:53,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:53,312 - lava_callback - INFO - Completed processing callback for node 69cf0fa89e2394c38fe0178b 2026-04-03 00:56:53,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fdc9e2394c38fe01848 HTTP/1.1" 200 None 2026-04-03 00:56:53,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:53,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:53,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:53,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fdc9e2394c38fe01848 HTTP/1.1" 200 None 2026-04-03 00:56:53,397 - lava_callback - INFO - Completed processing callback for node 69cf0fdc9e2394c38fe01848 INFO: 172.21.0.1:52316 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:56:57,252 - lava_callback - INFO - Received callback for node 69cf0fad9e2394c38fe01797 INFO: 172.21.0.1:52324 - "POST /node/69cf0fad9e2394c38fe01797 HTTP/1.0" 202 Accepted 2026-04-03 00:56:57,262 - lava_callback - INFO - Starting async job submit for node 69cf0fad9e2394c38fe01797 2026-04-03 00:56:57,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:57,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fad9e2394c38fe01797 HTTP/1.1" 200 None 2026-04-03 00:56:57,301 - lava_callback - DEBUG - Processing node 69cf0fad9e2394c38fe01797: baseline-arm64 2026-04-03 00:56:57,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:57,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:57,761 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fad9e2394c38fe01797/log.txt.gz 2026-04-03 00:56:57,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:58,136 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:58,138 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fad9e2394c38fe01797/lava-logs.yaml 2026-04-03 00:56:58,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:56:58,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:56:58,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fad9e2394c38fe01797/lava_callback.json.gz 2026-04-03 00:56:58,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:58,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fad9e2394c38fe01797 HTTP/1.1" 200 None 2026-04-03 00:56:58,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:58,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:56:58,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:58,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fad9e2394c38fe01797 HTTP/1.1" 200 None 2026-04-03 00:56:58,356 - lava_callback - INFO - Completed processing callback for node 69cf0fad9e2394c38fe01797 2026-04-03 00:56:59,822 - lava_callback - INFO - Received callback for node 69cf0fbd9e2394c38fe017c7 INFO: 172.21.0.1:60774 - "POST /node/69cf0fbd9e2394c38fe017c7 HTTP/1.0" 202 Accepted 2026-04-03 00:56:59,832 - lava_callback - INFO - Starting async job submit for node 69cf0fbd9e2394c38fe017c7 2026-04-03 00:56:59,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:56:59,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fbd9e2394c38fe017c7 HTTP/1.1" 200 None 2026-04-03 00:56:59,871 - lava_callback - DEBUG - Processing node 69cf0fbd9e2394c38fe017c7: baseline-nfs-arm64 2026-04-03 00:57:00,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:00,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:00,480 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbd9e2394c38fe017c7/log.txt.gz 2026-04-03 00:57:00,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:00,861 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:00,863 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbd9e2394c38fe017c7/lava-logs.yaml 2026-04-03 00:57:00,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:01,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:01,014 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbd9e2394c38fe017c7/lava_callback.json.gz 2026-04-03 00:57:01,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:01,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fbd9e2394c38fe017c7 HTTP/1.1" 200 None 2026-04-03 00:57:01,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:01,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:57:01,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:01,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fbd9e2394c38fe017c7 HTTP/1.1" 200 None 2026-04-03 00:57:01,112 - lava_callback - INFO - Completed processing callback for node 69cf0fbd9e2394c38fe017c7 2026-04-03 00:57:04,340 - lava_callback - INFO - Received callback for node 69cf0fb99e2394c38fe017b7 INFO: 172.21.0.1:60778 - "POST /node/69cf0fb99e2394c38fe017b7 HTTP/1.0" 202 Accepted 2026-04-03 00:57:04,348 - lava_callback - INFO - Starting async job submit for node 69cf0fb99e2394c38fe017b7 2026-04-03 00:57:04,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:04,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb99e2394c38fe017b7 HTTP/1.1" 200 None 2026-04-03 00:57:04,407 - lava_callback - DEBUG - Processing node 69cf0fb99e2394c38fe017b7: baseline-nfs-arm64 2026-04-03 00:57:04,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:05,097 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:05,099 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb99e2394c38fe017b7/log.txt.gz 2026-04-03 00:57:05,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:05,640 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:05,642 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb99e2394c38fe017b7/lava-logs.yaml 2026-04-03 00:57:05,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:05,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:05,883 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb99e2394c38fe017b7/lava_callback.json.gz 2026-04-03 00:57:05,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:05,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb99e2394c38fe017b7 HTTP/1.1" 200 None 2026-04-03 00:57:05,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:05,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:57:05,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:05,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fb99e2394c38fe017b7 HTTP/1.1" 200 None 2026-04-03 00:57:05,968 - lava_callback - INFO - Completed processing callback for node 69cf0fb99e2394c38fe017b7 INFO: 172.21.0.1:60780 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:57:08,154 - lava_callback - INFO - Received callback for node 69cf0fa09e2394c38fe0174f INFO: 172.21.0.1:60784 - "POST /node/69cf0fa09e2394c38fe0174f HTTP/1.0" 202 Accepted 2026-04-03 00:57:08,160 - lava_callback - INFO - Starting async job submit for node 69cf0fa09e2394c38fe0174f 2026-04-03 00:57:08,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:08,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa09e2394c38fe0174f HTTP/1.1" 200 None 2026-04-03 00:57:08,196 - lava_callback - DEBUG - Processing node 69cf0fa09e2394c38fe0174f: baseline-arm64 2026-04-03 00:57:08,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:08,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:08,576 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa09e2394c38fe0174f/log.txt.gz 2026-04-03 00:57:08,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:08,901 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:08,903 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa09e2394c38fe0174f/lava-logs.yaml 2026-04-03 00:57:08,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:09,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:09,077 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa09e2394c38fe0174f/lava_callback.json.gz 2026-04-03 00:57:09,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:09,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa09e2394c38fe0174f HTTP/1.1" 200 None 2026-04-03 00:57:09,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:09,133 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:57:09,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:09,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fa09e2394c38fe0174f HTTP/1.1" 200 None 2026-04-03 00:57:09,177 - lava_callback - INFO - Completed processing callback for node 69cf0fa09e2394c38fe0174f 2026-04-03 00:57:11,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:11,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-03 00:57:14,890 - lava_callback - INFO - Received callback for node 69cf0fb29e2394c38fe017a3 INFO: 172.21.0.1:53254 - "POST /node/69cf0fb29e2394c38fe017a3 HTTP/1.0" 202 Accepted 2026-04-03 00:57:14,898 - lava_callback - INFO - Starting async job submit for node 69cf0fb29e2394c38fe017a3 2026-04-03 00:57:14,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:14,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb29e2394c38fe017a3 HTTP/1.1" 200 None 2026-04-03 00:57:14,951 - lava_callback - DEBUG - Processing node 69cf0fb29e2394c38fe017a3: baseline-nfs-arm64 INFO: 172.21.0.1:53262 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 00:57:15,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:15,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:15,413 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb29e2394c38fe017a3/log.txt.gz 2026-04-03 00:57:15,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:53266 - "GET /node/698b6b730a16aea5a0dc7b8b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:53282 - "GET /node/693b0f10fee8383e92b72a20%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-03 00:57:15,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:15,804 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb29e2394c38fe017a3/lava-logs.yaml 2026-04-03 00:57:15,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:15,949 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:15,950 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb29e2394c38fe017a3/lava_callback.json.gz 2026-04-03 00:57:15,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:16,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb29e2394c38fe017a3 HTTP/1.1" 200 None 2026-04-03 00:57:16,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:16,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:57:16,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:16,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fb29e2394c38fe017a3 HTTP/1.1" 200 None 2026-04-03 00:57:16,042 - lava_callback - INFO - Completed processing callback for node 69cf0fb29e2394c38fe017a3 INFO: 172.21.0.1:53294 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:57:22,137 - lava_callback - INFO - Received callback for node 69cf0fc09e2394c38fe017cd INFO: 172.21.0.1:58096 - "POST /node/69cf0fc09e2394c38fe017cd HTTP/1.0" 202 Accepted 2026-04-03 00:57:22,145 - lava_callback - INFO - Starting async job submit for node 69cf0fc09e2394c38fe017cd 2026-04-03 00:57:22,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:22,184 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fc09e2394c38fe017cd HTTP/1.1" 200 None 2026-04-03 00:57:22,185 - lava_callback - DEBUG - Processing node 69cf0fc09e2394c38fe017cd: baseline-nfs-arm64 2026-04-03 00:57:22,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:22,785 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:22,787 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fc09e2394c38fe017cd/log.txt.gz 2026-04-03 00:57:23,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:23,240 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:23,242 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fc09e2394c38fe017cd/lava-logs.yaml 2026-04-03 00:57:23,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:23,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:23,479 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fc09e2394c38fe017cd/lava_callback.json.gz 2026-04-03 00:57:23,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:23,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fc09e2394c38fe017cd HTTP/1.1" 200 None 2026-04-03 00:57:23,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:23,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:57:23,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:23,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fc09e2394c38fe017cd HTTP/1.1" 200 None 2026-04-03 00:57:23,563 - lava_callback - INFO - Completed processing callback for node 69cf0fc09e2394c38fe017cd 2026-04-03 00:57:23,790 - lava_callback - INFO - Received callback for node 69cf0fbb9e2394c38fe017c1 INFO: 172.21.0.1:58104 - "POST /node/69cf0fbb9e2394c38fe017c1 HTTP/1.0" 202 Accepted 2026-04-03 00:57:23,797 - lava_callback - INFO - Starting async job submit for node 69cf0fbb9e2394c38fe017c1 2026-04-03 00:57:23,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:23,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fbb9e2394c38fe017c1 HTTP/1.1" 200 None 2026-04-03 00:57:23,838 - lava_callback - DEBUG - Processing node 69cf0fbb9e2394c38fe017c1: baseline-nfs-arm64 2026-04-03 00:57:24,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:24,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:24,280 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbb9e2394c38fe017c1/log.txt.gz 2026-04-03 00:57:24,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:24,740 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:24,742 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbb9e2394c38fe017c1/lava-logs.yaml 2026-04-03 00:57:24,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:24,937 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:24,939 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbb9e2394c38fe017c1/lava_callback.json.gz 2026-04-03 00:57:24,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:24,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fbb9e2394c38fe017c1 HTTP/1.1" 200 None 2026-04-03 00:57:24,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:24,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:57:25,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:25,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fbb9e2394c38fe017c1 HTTP/1.1" 200 None 2026-04-03 00:57:25,038 - lava_callback - INFO - Completed processing callback for node 69cf0fbb9e2394c38fe017c1 INFO: 172.21.0.1:58108 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:57:26,622 - lava_callback - INFO - Received callback for node 69cf0fb59e2394c38fe017ae INFO: 172.21.0.1:58124 - "POST /node/69cf0fb59e2394c38fe017ae HTTP/1.0" 202 Accepted 2026-04-03 00:57:26,629 - lava_callback - INFO - Starting async job submit for node 69cf0fb59e2394c38fe017ae 2026-04-03 00:57:26,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:26,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb59e2394c38fe017ae HTTP/1.1" 200 None 2026-04-03 00:57:26,666 - lava_callback - DEBUG - Processing node 69cf0fb59e2394c38fe017ae: baseline-nfs-arm64 2026-04-03 00:57:27,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:27,140 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:27,142 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb59e2394c38fe017ae/log.txt.gz 2026-04-03 00:57:27,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:27,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:27,607 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb59e2394c38fe017ae/lava-logs.yaml 2026-04-03 00:57:27,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:27,816 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:27,818 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb59e2394c38fe017ae/lava_callback.json.gz 2026-04-03 00:57:27,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:27,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb59e2394c38fe017ae HTTP/1.1" 200 None 2026-04-03 00:57:27,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:27,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:57:27,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:27,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fb59e2394c38fe017ae HTTP/1.1" 200 None 2026-04-03 00:57:27,934 - lava_callback - INFO - Completed processing callback for node 69cf0fb59e2394c38fe017ae INFO: 172.21.0.1:37446 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:57:39,405 - lava_callback - INFO - Received callback for node 69cf0fbe9e2394c38fe017ca INFO: 172.21.0.1:37448 - "POST /node/69cf0fbe9e2394c38fe017ca HTTP/1.0" 202 Accepted 2026-04-03 00:57:39,415 - lava_callback - INFO - Starting async job submit for node 69cf0fbe9e2394c38fe017ca 2026-04-03 00:57:39,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:39,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fbe9e2394c38fe017ca HTTP/1.1" 200 None 2026-04-03 00:57:39,466 - lava_callback - DEBUG - Processing node 69cf0fbe9e2394c38fe017ca: baseline-nfs-arm64 2026-04-03 00:57:39,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:39,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:39,987 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbe9e2394c38fe017ca/log.txt.gz 2026-04-03 00:57:40,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:40,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:40,437 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbe9e2394c38fe017ca/lava-logs.yaml 2026-04-03 00:57:40,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:57:40,582 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:57:40,584 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbe9e2394c38fe017ca/lava_callback.json.gz 2026-04-03 00:57:40,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:40,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fbe9e2394c38fe017ca HTTP/1.1" 200 None 2026-04-03 00:57:40,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:40,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:57:40,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:40,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fbe9e2394c38fe017ca HTTP/1.1" 200 None 2026-04-03 00:57:40,669 - lava_callback - INFO - Completed processing callback for node 69cf0fbe9e2394c38fe017ca 2026-04-03 00:57:41,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:57:41,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:59070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41580 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:59:08,871 - lava_callback - INFO - Received callback for node 69cf0fa49e2394c38fe0177c INFO: 172.21.0.1:41586 - "POST /node/69cf0fa49e2394c38fe0177c HTTP/1.0" 202 Accepted 2026-04-03 00:59:08,877 - lava_callback - INFO - Starting async job submit for node 69cf0fa49e2394c38fe0177c 2026-04-03 00:59:08,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:59:08,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa49e2394c38fe0177c HTTP/1.1" 200 None 2026-04-03 00:59:08,911 - lava_callback - DEBUG - Processing node 69cf0fa49e2394c38fe0177c: baseline-arm64 2026-04-03 00:59:09,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:59:09,370 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:59:09,372 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa49e2394c38fe0177c/log.txt.gz 2026-04-03 00:59:09,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:59:09,938 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:59:09,940 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa49e2394c38fe0177c/lava-logs.yaml 2026-04-03 00:59:09,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:59:10,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:59:10,132 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fa49e2394c38fe0177c/lava_callback.json.gz 2026-04-03 00:59:10,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:59:10,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa49e2394c38fe0177c HTTP/1.1" 200 None 2026-04-03 00:59:10,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:59:10,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:59:10,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:59:10,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fa49e2394c38fe0177c HTTP/1.1" 200 None 2026-04-03 00:59:10,220 - lava_callback - INFO - Completed processing callback for node 69cf0fa49e2394c38fe0177c 2026-04-03 00:59:11,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:59:11,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39532 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:59:25,471 - lava_callback - INFO - Received callback for node 69cf0fb39e2394c38fe017a9 INFO: 172.21.0.1:55448 - "POST /node/69cf0fb39e2394c38fe017a9 HTTP/1.0" 202 Accepted 2026-04-03 00:59:25,477 - lava_callback - INFO - Starting async job submit for node 69cf0fb39e2394c38fe017a9 2026-04-03 00:59:25,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:59:25,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb39e2394c38fe017a9 HTTP/1.1" 200 None 2026-04-03 00:59:25,528 - lava_callback - DEBUG - Processing node 69cf0fb39e2394c38fe017a9: baseline-nfs-arm64 2026-04-03 00:59:25,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:59:26,142 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:59:26,144 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb39e2394c38fe017a9/log.txt.gz 2026-04-03 00:59:26,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:55464 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:59:26,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:59:26,730 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb39e2394c38fe017a9/lava-logs.yaml 2026-04-03 00:59:26,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 00:59:27,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 00:59:27,021 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb39e2394c38fe017a9/lava_callback.json.gz 2026-04-03 00:59:27,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:59:27,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb39e2394c38fe017a9 HTTP/1.1" 200 None 2026-04-03 00:59:27,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:59:27,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 00:59:27,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:59:27,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fb39e2394c38fe017a9 HTTP/1.1" 200 None 2026-04-03 00:59:27,128 - lava_callback - INFO - Completed processing callback for node 69cf0fb39e2394c38fe017a9 INFO: 172.21.0.1:55476 - "GET /node/693b19cb15f407b7629df7ad%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:55488 - "GET /node/698a40cb22b4cc0f227b05d4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK 2026-04-03 00:59:41,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 00:59:41,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43794 - "GET /node/69910ab5c45febfe44529331%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:43810 - "GET /node/694bd952c19e09af58d49c31%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59188 - "GET /node/693ff10c575bc060ce14ea40%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:59190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42418 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:01:34,738 - lava_callback - INFO - Received callback for node 69cc6e5a8cd9cbcdc9459e0f INFO: 172.21.0.1:46388 - "POST /node/69cc6e5a8cd9cbcdc9459e0f HTTP/1.0" 202 Accepted 2026-04-03 01:01:34,743 - lava_callback - INFO - Starting async job submit for node 69cc6e5a8cd9cbcdc9459e0f 2026-04-03 01:01:34,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:01:34,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6e5a8cd9cbcdc9459e0f HTTP/1.1" 200 None 2026-04-03 01:01:34,778 - lava_callback - DEBUG - Processing node 69cc6e5a8cd9cbcdc9459e0f: baseline-arm-baylibre 2026-04-03 01:01:34,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6e5a8cd9cbcdc9459e0f 2026-04-03 01:01:34,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:01:34,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6e5a8cd9cbcdc9459e0f HTTP/1.1" 200 None 2026-04-03 01:01:34,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:01:34,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-03 01:01:34,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:01:34,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6e5a8cd9cbcdc9459e0f HTTP/1.1" 200 None 2026-04-03 01:01:34,844 - lava_callback - ERROR - Error processing callback for node 69cc6e5a8cd9cbcdc9459e0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45370 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:02:14,462 - lava_callback - INFO - Received callback for node 69cc6e828cd9cbcdc9459e23 INFO: 172.21.0.1:46364 - "POST /node/69cc6e828cd9cbcdc9459e23 HTTP/1.0" 202 Accepted 2026-04-03 01:02:14,467 - lava_callback - INFO - Starting async job submit for node 69cc6e828cd9cbcdc9459e23 2026-04-03 01:02:14,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:02:14,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6e828cd9cbcdc9459e23 HTTP/1.1" 200 None 2026-04-03 01:02:14,483 - lava_callback - DEBUG - Processing node 69cc6e828cd9cbcdc9459e23: baseline-arm-baylibre 2026-04-03 01:02:14,483 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6e828cd9cbcdc9459e23 2026-04-03 01:02:14,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:02:14,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6e828cd9cbcdc9459e23 HTTP/1.1" 200 None 2026-04-03 01:02:14,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:02:14,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:02:14,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:02:14,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6e828cd9cbcdc9459e23 HTTP/1.1" 200 None 2026-04-03 01:02:14,536 - lava_callback - ERROR - Error processing callback for node 69cc6e828cd9cbcdc9459e23: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46380 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56436 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:02:54,562 - lava_callback - INFO - Received callback for node 69cc6eab8cd9cbcdc9459e30 INFO: 172.21.0.1:45540 - "POST /node/69cc6eab8cd9cbcdc9459e30 HTTP/1.0" 202 Accepted 2026-04-03 01:02:54,567 - lava_callback - INFO - Starting async job submit for node 69cc6eab8cd9cbcdc9459e30 2026-04-03 01:02:54,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:02:54,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6eab8cd9cbcdc9459e30 HTTP/1.1" 200 None 2026-04-03 01:02:54,630 - lava_callback - DEBUG - Processing node 69cc6eab8cd9cbcdc9459e30: baseline-arm-baylibre 2026-04-03 01:02:54,631 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6eab8cd9cbcdc9459e30 2026-04-03 01:02:54,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:02:54,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6eab8cd9cbcdc9459e30 HTTP/1.1" 200 None 2026-04-03 01:02:54,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:02:54,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:02:54,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:02:54,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6eab8cd9cbcdc9459e30 HTTP/1.1" 200 None 2026-04-03 01:02:54,733 - lava_callback - ERROR - Error processing callback for node 69cc6eab8cd9cbcdc9459e30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45550 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:03:04,937 - lava_callback - INFO - Received callback for node 69cf0fb19e2394c38fe017a0 INFO: 172.21.0.1:46334 - "POST /node/69cf0fb19e2394c38fe017a0 HTTP/1.0" 202 Accepted 2026-04-03 01:03:04,944 - lava_callback - INFO - Starting async job submit for node 69cf0fb19e2394c38fe017a0 2026-04-03 01:03:04,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:03:04,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb19e2394c38fe017a0 HTTP/1.1" 200 None 2026-04-03 01:03:04,996 - lava_callback - DEBUG - Processing node 69cf0fb19e2394c38fe017a0: baseline-arm64 2026-04-03 01:03:05,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:03:05,554 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:03:05,556 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fb19e2394c38fe017a0/log.txt.gz 2026-04-03 01:03:05,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:03:06,055 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:03:06,057 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fb19e2394c38fe017a0/lava-logs.yaml 2026-04-03 01:03:06,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:03:06,257 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:03:06,259 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fb19e2394c38fe017a0/lava_callback.json.gz DEBUG: Setup failed for 69cf0fb19e2394c38fe017a0. Transitting job node result: incomplete -> fail 2026-04-03 01:03:06,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:03:06,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb19e2394c38fe017a0 HTTP/1.1" 200 None 2026-04-03 01:03:06,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:03:06,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:03:06,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:03:06,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fb19e2394c38fe017a0 HTTP/1.1" 200 None 2026-04-03 01:03:06,499 - lava_callback - INFO - Completed processing callback for node 69cf0fb19e2394c38fe017a0 INFO: 172.21.0.1:46350 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:03:11,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:03:11,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42176 - "GET /node/69885bdce408e9ea60faee44%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:42182 - "GET /node/69935091523507f3806a69ce%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:42194 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:03:34,588 - lava_callback - INFO - Received callback for node 69cc6ed68cd9cbcdc9459e3d INFO: 172.21.0.1:56980 - "POST /node/69cc6ed68cd9cbcdc9459e3d HTTP/1.0" 202 Accepted 2026-04-03 01:03:34,594 - lava_callback - INFO - Starting async job submit for node 69cc6ed68cd9cbcdc9459e3d 2026-04-03 01:03:34,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:03:34,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ed68cd9cbcdc9459e3d HTTP/1.1" 200 None 2026-04-03 01:03:34,609 - lava_callback - DEBUG - Processing node 69cc6ed68cd9cbcdc9459e3d: baseline-arm-baylibre 2026-04-03 01:03:34,609 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6ed68cd9cbcdc9459e3d 2026-04-03 01:03:34,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:03:34,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6ed68cd9cbcdc9459e3d HTTP/1.1" 200 None 2026-04-03 01:03:34,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:03:34,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:03:34,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:03:34,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6ed68cd9cbcdc9459e3d HTTP/1.1" 200 None 2026-04-03 01:03:34,665 - lava_callback - ERROR - Error processing callback for node 69cc6ed68cd9cbcdc9459e3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39714 - "GET /node/6992ed0531166481c8dd9b0d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:39722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45912 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:05:15,111 - lava_callback - INFO - Received callback for node 69cc6f418cd9cbcdc9459e54 INFO: 172.21.0.1:36674 - "POST /node/69cc6f418cd9cbcdc9459e54 HTTP/1.0" 202 Accepted 2026-04-03 01:05:15,117 - lava_callback - INFO - Starting async job submit for node 69cc6f418cd9cbcdc9459e54 2026-04-03 01:05:15,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:15,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6f418cd9cbcdc9459e54 HTTP/1.1" 200 None 2026-04-03 01:05:15,133 - lava_callback - DEBUG - Processing node 69cc6f418cd9cbcdc9459e54: baseline-arm-baylibre 2026-04-03 01:05:15,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6f418cd9cbcdc9459e54 2026-04-03 01:05:15,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:36690 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 01:05:15,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6f418cd9cbcdc9459e54 HTTP/1.1" 200 None 2026-04-03 01:05:15,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:15,174 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:05:15,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:15,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6f418cd9cbcdc9459e54 HTTP/1.1" 200 None 2026-04-03 01:05:15,206 - lava_callback - ERROR - Error processing callback for node 69cc6f418cd9cbcdc9459e54: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60322 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:05:52,618 - lava_callback - INFO - Received callback for node 69cc6f6e8cd9cbcdc9459e67 INFO: 172.21.0.1:58024 - "POST /node/69cc6f6e8cd9cbcdc9459e67 HTTP/1.0" 202 Accepted 2026-04-03 01:05:52,627 - lava_callback - INFO - Starting async job submit for node 69cc6f6e8cd9cbcdc9459e67 2026-04-03 01:05:52,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:52,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6f6e8cd9cbcdc9459e67 HTTP/1.1" 200 None 2026-04-03 01:05:52,654 - lava_callback - DEBUG - Processing node 69cc6f6e8cd9cbcdc9459e67: baseline-arm-clabbe 2026-04-03 01:05:52,654 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6f6e8cd9cbcdc9459e67 2026-04-03 01:05:52,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:52,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6f6e8cd9cbcdc9459e67 HTTP/1.1" 200 None 2026-04-03 01:05:52,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:52,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:05:52,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:52,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6f6e8cd9cbcdc9459e67 HTTP/1.1" 200 None 2026-04-03 01:05:52,725 - lava_callback - ERROR - Error processing callback for node 69cc6f6e8cd9cbcdc9459e67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:05:53,352 - lava_callback - INFO - Received callback for node 69cf0e739e2394c38fe01524 INFO: 172.21.0.1:58038 - "POST /node/69cf0e739e2394c38fe01524 HTTP/1.0" 202 Accepted 2026-04-03 01:05:53,360 - lava_callback - INFO - Starting async job submit for node 69cf0e739e2394c38fe01524 2026-04-03 01:05:53,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:53,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e739e2394c38fe01524 HTTP/1.1" 200 None 2026-04-03 01:05:53,448 - lava_callback - DEBUG - Processing node 69cf0e739e2394c38fe01524: baseline-arm 2026-04-03 01:05:54,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:05:54,254 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:05:54,256 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e739e2394c38fe01524/log.txt.gz 2026-04-03 01:05:54,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:05:54,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:05:54,934 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e739e2394c38fe01524/lava-logs.yaml 2026-04-03 01:05:54,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:05:55,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:05:55,112 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf0e739e2394c38fe01524/lava_callback.json.gz DEBUG: Setup failed for 69cf0e739e2394c38fe01524. Transitting job node result: incomplete -> fail 2026-04-03 01:05:55,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:55,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e739e2394c38fe01524 HTTP/1.1" 200 None 2026-04-03 01:05:55,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:55,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:05:55,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:05:55,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0e739e2394c38fe01524 HTTP/1.1" 200 None 2026-04-03 01:05:55,438 - lava_callback - INFO - Completed processing callback for node 69cf0e739e2394c38fe01524 INFO: 172.21.0.1:58052 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:06:02,420 - lava_callback - INFO - Received callback for node 69cf0fac9e2394c38fe01794 INFO: 172.21.0.1:40120 - "POST /node/69cf0fac9e2394c38fe01794 HTTP/1.0" 202 Accepted 2026-04-03 01:06:02,427 - lava_callback - INFO - Starting async job submit for node 69cf0fac9e2394c38fe01794 2026-04-03 01:06:02,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:02,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fac9e2394c38fe01794 HTTP/1.1" 200 None 2026-04-03 01:06:02,467 - lava_callback - DEBUG - Processing node 69cf0fac9e2394c38fe01794: baseline-arm64 2026-04-03 01:06:02,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:06:03,088 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:06:03,090 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fac9e2394c38fe01794/log.txt.gz 2026-04-03 01:06:03,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:06:03,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:06:03,638 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fac9e2394c38fe01794/lava-logs.yaml 2026-04-03 01:06:03,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:06:03,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:06:03,827 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf0fac9e2394c38fe01794/lava_callback.json.gz 2026-04-03 01:06:03,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:03,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fac9e2394c38fe01794 HTTP/1.1" 200 None 2026-04-03 01:06:03,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:03,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:06:03,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:03,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fac9e2394c38fe01794 HTTP/1.1" 200 None 2026-04-03 01:06:03,980 - lava_callback - INFO - Completed processing callback for node 69cf0fac9e2394c38fe01794 INFO: 172.21.0.1:40128 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:06:11,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:11,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46384 - "GET /node/698b680b0a16aea5a0dc7648%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-03 01:06:18,142 - lava_callback - INFO - Received callback for node 69cf0fbc9e2394c38fe017c4 INFO: 172.21.0.1:46386 - "POST /node/69cf0fbc9e2394c38fe017c4 HTTP/1.0" 202 Accepted 2026-04-03 01:06:18,149 - lava_callback - INFO - Starting async job submit for node 69cf0fbc9e2394c38fe017c4 2026-04-03 01:06:18,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:18,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fbc9e2394c38fe017c4 HTTP/1.1" 200 None 2026-04-03 01:06:18,192 - lava_callback - DEBUG - Processing node 69cf0fbc9e2394c38fe017c4: baseline-nfs-arm64 2026-04-03 01:06:18,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:06:18,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:06:18,831 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbc9e2394c38fe017c4/log.txt.gz 2026-04-03 01:06:19,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:06:19,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:06:19,334 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbc9e2394c38fe017c4/lava-logs.yaml 2026-04-03 01:06:19,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:06:19,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:06:19,546 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fbc9e2394c38fe017c4/lava_callback.json.gz 2026-04-03 01:06:19,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:19,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fbc9e2394c38fe017c4 HTTP/1.1" 200 None 2026-04-03 01:06:19,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:19,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:06:19,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:19,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fbc9e2394c38fe017c4 HTTP/1.1" 200 None 2026-04-03 01:06:19,703 - lava_callback - INFO - Completed processing callback for node 69cf0fbc9e2394c38fe017c4 INFO: 172.21.0.1:35598 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:06:34,773 - lava_callback - INFO - Received callback for node 69cc6f968cd9cbcdc9459e72 INFO: 172.21.0.1:55118 - "POST /node/69cc6f968cd9cbcdc9459e72 HTTP/1.0" 202 Accepted 2026-04-03 01:06:34,778 - lava_callback - INFO - Starting async job submit for node 69cc6f968cd9cbcdc9459e72 2026-04-03 01:06:34,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:34,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6f968cd9cbcdc9459e72 HTTP/1.1" 200 None 2026-04-03 01:06:34,794 - lava_callback - DEBUG - Processing node 69cc6f968cd9cbcdc9459e72: baseline-arm-baylibre 2026-04-03 01:06:34,794 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6f968cd9cbcdc9459e72 2026-04-03 01:06:34,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:34,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6f968cd9cbcdc9459e72 HTTP/1.1" 200 None 2026-04-03 01:06:34,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:34,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:06:34,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:34,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6f968cd9cbcdc9459e72 HTTP/1.1" 200 None 2026-04-03 01:06:34,850 - lava_callback - ERROR - Error processing callback for node 69cc6f968cd9cbcdc9459e72: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55122 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:06:41,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:06:41,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53516 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:07:34,939 - lava_callback - INFO - Received callback for node 69cc6fc18cd9cbcdc9459e7f INFO: 172.21.0.1:57426 - "POST /node/69cc6fc18cd9cbcdc9459e7f HTTP/1.0" 202 Accepted 2026-04-03 01:07:34,945 - lava_callback - INFO - Starting async job submit for node 69cc6fc18cd9cbcdc9459e7f 2026-04-03 01:07:34,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:07:34,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fc18cd9cbcdc9459e7f HTTP/1.1" 200 None 2026-04-03 01:07:34,961 - lava_callback - DEBUG - Processing node 69cc6fc18cd9cbcdc9459e7f: baseline-arm-baylibre 2026-04-03 01:07:34,961 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6fc18cd9cbcdc9459e7f 2026-04-03 01:07:34,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:07:34,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fc18cd9cbcdc9459e7f HTTP/1.1" 200 None 2026-04-03 01:07:34,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:07:34,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:07:34,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:07:35,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fc18cd9cbcdc9459e7f HTTP/1.1" 200 None 2026-04-03 01:07:35,018 - lava_callback - ERROR - Error processing callback for node 69cc6fc18cd9cbcdc9459e7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57442 - "GET /node/68826ba987dcced189497f35%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57458 - "GET /node/69280d9bb1b5e06bf6c04d24%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38082 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:08:14,850 - lava_callback - INFO - Received callback for node 69cc6fe98cd9cbcdc9459ebf INFO: 172.21.0.1:59564 - "POST /node/69cc6fe98cd9cbcdc9459ebf HTTP/1.0" 202 Accepted 2026-04-03 01:08:14,858 - lava_callback - INFO - Starting async job submit for node 69cc6fe98cd9cbcdc9459ebf 2026-04-03 01:08:14,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:14,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe98cd9cbcdc9459ebf HTTP/1.1" 200 None 2026-04-03 01:08:14,874 - lava_callback - DEBUG - Processing node 69cc6fe98cd9cbcdc9459ebf: baseline-arm-baylibre 2026-04-03 01:08:14,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc6fe98cd9cbcdc9459ebf 2026-04-03 01:08:14,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:14,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc6fe98cd9cbcdc9459ebf HTTP/1.1" 200 None 2026-04-03 01:08:14,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:14,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-03 01:08:14,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:14,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc6fe98cd9cbcdc9459ebf HTTP/1.1" 200 None 2026-04-03 01:08:14,946 - lava_callback - ERROR - Error processing callback for node 69cc6fe98cd9cbcdc9459ebf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58212 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:08:34,848 - lava_callback - INFO - Received callback for node 69cc70058cd9cbcdc9459f15 INFO: 172.21.0.1:47496 - "POST /node/69cc70058cd9cbcdc9459f15 HTTP/1.0" 202 Accepted 2026-04-03 01:08:34,854 - lava_callback - INFO - Starting async job submit for node 69cc70058cd9cbcdc9459f15 2026-04-03 01:08:34,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:34,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70058cd9cbcdc9459f15 HTTP/1.1" 200 None 2026-04-03 01:08:34,870 - lava_callback - DEBUG - Processing node 69cc70058cd9cbcdc9459f15: baseline-arm64-baylibre 2026-04-03 01:08:34,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc70058cd9cbcdc9459f15 2026-04-03 01:08:34,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:34,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70058cd9cbcdc9459f15 HTTP/1.1" 200 None 2026-04-03 01:08:34,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:34,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:34,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:34,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70058cd9cbcdc9459f15 HTTP/1.1" 200 None 2026-04-03 01:08:34,928 - lava_callback - ERROR - Error processing callback for node 69cc70058cd9cbcdc9459f15: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:08:35,532 - lava_callback - INFO - Received callback for node 69cc70078cd9cbcdc9459f18 INFO: 172.21.0.1:47512 - "POST /node/69cc70078cd9cbcdc9459f18 HTTP/1.0" 202 Accepted 2026-04-03 01:08:35,538 - lava_callback - INFO - Starting async job submit for node 69cc70078cd9cbcdc9459f18 2026-04-03 01:08:35,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:35,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70078cd9cbcdc9459f18 HTTP/1.1" 200 None 2026-04-03 01:08:35,554 - lava_callback - DEBUG - Processing node 69cc70078cd9cbcdc9459f18: baseline-arm64-baylibre 2026-04-03 01:08:35,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc70078cd9cbcdc9459f18 2026-04-03 01:08:35,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:35,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70078cd9cbcdc9459f18 HTTP/1.1" 200 None 2026-04-03 01:08:35,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:35,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:35,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:35,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70078cd9cbcdc9459f18 HTTP/1.1" 200 None 2026-04-03 01:08:35,610 - lava_callback - ERROR - Error processing callback for node 69cc70078cd9cbcdc9459f18: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:08:36,188 - lava_callback - INFO - Received callback for node 69cc700a8cd9cbcdc9459f1f INFO: 172.21.0.1:47518 - "POST /node/69cc700a8cd9cbcdc9459f1f HTTP/1.0" 202 Accepted 2026-04-03 01:08:36,196 - lava_callback - INFO - Starting async job submit for node 69cc700a8cd9cbcdc9459f1f 2026-04-03 01:08:36,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:36,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc700a8cd9cbcdc9459f1f HTTP/1.1" 200 None 2026-04-03 01:08:36,214 - lava_callback - DEBUG - Processing node 69cc700a8cd9cbcdc9459f1f: baseline-arm64-baylibre 2026-04-03 01:08:36,214 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc700a8cd9cbcdc9459f1f 2026-04-03 01:08:36,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:36,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc700a8cd9cbcdc9459f1f HTTP/1.1" 200 None 2026-04-03 01:08:36,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:36,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:36,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:36,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc700a8cd9cbcdc9459f1f HTTP/1.1" 200 None 2026-04-03 01:08:36,290 - lava_callback - ERROR - Error processing callback for node 69cc700a8cd9cbcdc9459f1f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47520 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:08:36,861 - lava_callback - INFO - Received callback for node 69cc700b8cd9cbcdc9459f3e INFO: 172.21.0.1:47534 - "POST /node/69cc700b8cd9cbcdc9459f3e HTTP/1.0" 202 Accepted 2026-04-03 01:08:36,870 - lava_callback - INFO - Starting async job submit for node 69cc700b8cd9cbcdc9459f3e 2026-04-03 01:08:36,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:36,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc700b8cd9cbcdc9459f3e HTTP/1.1" 200 None 2026-04-03 01:08:36,891 - lava_callback - DEBUG - Processing node 69cc700b8cd9cbcdc9459f3e: baseline-arm64-baylibre 2026-04-03 01:08:36,891 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc700b8cd9cbcdc9459f3e 2026-04-03 01:08:36,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:36,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc700b8cd9cbcdc9459f3e HTTP/1.1" 200 None 2026-04-03 01:08:36,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:36,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:36,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:36,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc700b8cd9cbcdc9459f3e HTTP/1.1" 200 None 2026-04-03 01:08:36,947 - lava_callback - ERROR - Error processing callback for node 69cc700b8cd9cbcdc9459f3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:08:37,525 - lava_callback - INFO - Received callback for node 69cc700d8cd9cbcdc9459f43 INFO: 172.21.0.1:47542 - "POST /node/69cc700d8cd9cbcdc9459f43 HTTP/1.0" 202 Accepted 2026-04-03 01:08:37,531 - lava_callback - INFO - Starting async job submit for node 69cc700d8cd9cbcdc9459f43 2026-04-03 01:08:37,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:37,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc700d8cd9cbcdc9459f43 HTTP/1.1" 200 None 2026-04-03 01:08:37,547 - lava_callback - DEBUG - Processing node 69cc700d8cd9cbcdc9459f43: baseline-arm64-baylibre 2026-04-03 01:08:37,547 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc700d8cd9cbcdc9459f43 2026-04-03 01:08:37,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:37,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc700d8cd9cbcdc9459f43 HTTP/1.1" 200 None 2026-04-03 01:08:37,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:37,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:37,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:37,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc700d8cd9cbcdc9459f43 HTTP/1.1" 200 None 2026-04-03 01:08:37,616 - lava_callback - ERROR - Error processing callback for node 69cc700d8cd9cbcdc9459f43: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49956 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:08:52,824 - lava_callback - INFO - Received callback for node 69cc70198cd9cbcdc9459f4e INFO: 172.21.0.1:33392 - "POST /node/69cc70198cd9cbcdc9459f4e HTTP/1.0" 202 Accepted 2026-04-03 01:08:52,830 - lava_callback - INFO - Starting async job submit for node 69cc70198cd9cbcdc9459f4e 2026-04-03 01:08:52,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:52,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70198cd9cbcdc9459f4e HTTP/1.1" 200 None 2026-04-03 01:08:52,845 - lava_callback - DEBUG - Processing node 69cc70198cd9cbcdc9459f4e: baseline-arm64-clabbe 2026-04-03 01:08:52,846 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc70198cd9cbcdc9459f4e 2026-04-03 01:08:52,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:52,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70198cd9cbcdc9459f4e HTTP/1.1" 200 None 2026-04-03 01:08:52,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:52,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:52,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:52,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70198cd9cbcdc9459f4e HTTP/1.1" 200 None 2026-04-03 01:08:52,905 - lava_callback - ERROR - Error processing callback for node 69cc70198cd9cbcdc9459f4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:08:53,499 - lava_callback - INFO - Received callback for node 69cc701a8cd9cbcdc9459f55 INFO: 172.21.0.1:33394 - "POST /node/69cc701a8cd9cbcdc9459f55 HTTP/1.0" 202 Accepted 2026-04-03 01:08:53,508 - lava_callback - INFO - Starting async job submit for node 69cc701a8cd9cbcdc9459f55 2026-04-03 01:08:53,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:53,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701a8cd9cbcdc9459f55 HTTP/1.1" 200 None 2026-04-03 01:08:53,525 - lava_callback - DEBUG - Processing node 69cc701a8cd9cbcdc9459f55: baseline-arm64-clabbe 2026-04-03 01:08:53,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc701a8cd9cbcdc9459f55 2026-04-03 01:08:53,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:53,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701a8cd9cbcdc9459f55 HTTP/1.1" 200 None 2026-04-03 01:08:53,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:53,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:53,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:53,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc701a8cd9cbcdc9459f55 HTTP/1.1" 200 None 2026-04-03 01:08:53,594 - lava_callback - ERROR - Error processing callback for node 69cc701a8cd9cbcdc9459f55: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:08:54,207 - lava_callback - INFO - Received callback for node 69cc701b8cd9cbcdc9459f58 INFO: 172.21.0.1:33396 - "POST /node/69cc701b8cd9cbcdc9459f58 HTTP/1.0" 202 Accepted 2026-04-03 01:08:54,213 - lava_callback - INFO - Starting async job submit for node 69cc701b8cd9cbcdc9459f58 2026-04-03 01:08:54,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:54,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701b8cd9cbcdc9459f58 HTTP/1.1" 200 None 2026-04-03 01:08:54,228 - lava_callback - DEBUG - Processing node 69cc701b8cd9cbcdc9459f58: baseline-arm64-clabbe 2026-04-03 01:08:54,228 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc701b8cd9cbcdc9459f58 2026-04-03 01:08:54,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:54,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701b8cd9cbcdc9459f58 HTTP/1.1" 200 None 2026-04-03 01:08:54,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:54,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:54,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:54,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc701b8cd9cbcdc9459f58 HTTP/1.1" 200 None 2026-04-03 01:08:54,284 - lava_callback - ERROR - Error processing callback for node 69cc701b8cd9cbcdc9459f58: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:08:54,901 - lava_callback - INFO - Received callback for node 69cc701c8cd9cbcdc9459f61 INFO: 172.21.0.1:33402 - "POST /node/69cc701c8cd9cbcdc9459f61 HTTP/1.0" 202 Accepted 2026-04-03 01:08:54,907 - lava_callback - INFO - Starting async job submit for node 69cc701c8cd9cbcdc9459f61 2026-04-03 01:08:54,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:54,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701c8cd9cbcdc9459f61 HTTP/1.1" 200 None 2026-04-03 01:08:54,924 - lava_callback - DEBUG - Processing node 69cc701c8cd9cbcdc9459f61: baseline-arm64-clabbe 2026-04-03 01:08:54,924 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc701c8cd9cbcdc9459f61 2026-04-03 01:08:54,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:54,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701c8cd9cbcdc9459f61 HTTP/1.1" 200 None 2026-04-03 01:08:54,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:54,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:54,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:54,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc701c8cd9cbcdc9459f61 HTTP/1.1" 200 None 2026-04-03 01:08:54,974 - lava_callback - ERROR - Error processing callback for node 69cc701c8cd9cbcdc9459f61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:08:55,585 - lava_callback - INFO - Received callback for node 69cc701f8cd9cbcdc9459f6a INFO: 172.21.0.1:33406 - "POST /node/69cc701f8cd9cbcdc9459f6a HTTP/1.0" 202 Accepted 2026-04-03 01:08:55,592 - lava_callback - INFO - Starting async job submit for node 69cc701f8cd9cbcdc9459f6a 2026-04-03 01:08:55,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:55,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701f8cd9cbcdc9459f6a HTTP/1.1" 200 None 2026-04-03 01:08:55,608 - lava_callback - DEBUG - Processing node 69cc701f8cd9cbcdc9459f6a: baseline-arm64-clabbe 2026-04-03 01:08:55,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc701f8cd9cbcdc9459f6a 2026-04-03 01:08:55,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:55,627 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701f8cd9cbcdc9459f6a HTTP/1.1" 200 None 2026-04-03 01:08:55,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:55,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:55,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:55,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc701f8cd9cbcdc9459f6a HTTP/1.1" 200 None 2026-04-03 01:08:55,666 - lava_callback - ERROR - Error processing callback for node 69cc701f8cd9cbcdc9459f6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:08:56,265 - lava_callback - INFO - Received callback for node 69cc701e8cd9cbcdc9459f67 INFO: 172.21.0.1:33412 - "POST /node/69cc701e8cd9cbcdc9459f67 HTTP/1.0" 202 Accepted 2026-04-03 01:08:56,273 - lava_callback - INFO - Starting async job submit for node 69cc701e8cd9cbcdc9459f67 2026-04-03 01:08:56,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:56,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701e8cd9cbcdc9459f67 HTTP/1.1" 200 None 2026-04-03 01:08:56,289 - lava_callback - DEBUG - Processing node 69cc701e8cd9cbcdc9459f67: baseline-arm64-clabbe 2026-04-03 01:08:56,289 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc701e8cd9cbcdc9459f67 2026-04-03 01:08:56,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:56,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701e8cd9cbcdc9459f67 HTTP/1.1" 200 None 2026-04-03 01:08:56,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:56,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 01:08:56,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:56,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc701e8cd9cbcdc9459f67 HTTP/1.1" 200 None 2026-04-03 01:08:56,350 - lava_callback - ERROR - Error processing callback for node 69cc701e8cd9cbcdc9459f67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33428 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:08:56,956 - lava_callback - INFO - Received callback for node 69cc701a8cd9cbcdc9459f52 INFO: 172.21.0.1:33434 - "POST /node/69cc701a8cd9cbcdc9459f52 HTTP/1.0" 202 Accepted 2026-04-03 01:08:56,962 - lava_callback - INFO - Starting async job submit for node 69cc701a8cd9cbcdc9459f52 2026-04-03 01:08:56,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:56,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701a8cd9cbcdc9459f52 HTTP/1.1" 200 None 2026-04-03 01:08:56,978 - lava_callback - DEBUG - Processing node 69cc701a8cd9cbcdc9459f52: baseline-arm-clabbe 2026-04-03 01:08:56,978 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc701a8cd9cbcdc9459f52 2026-04-03 01:08:56,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:56,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc701a8cd9cbcdc9459f52 HTTP/1.1" 200 None 2026-04-03 01:08:56,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:57,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:08:57,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:08:57,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc701a8cd9cbcdc9459f52 HTTP/1.1" 200 None 2026-04-03 01:08:57,039 - lava_callback - ERROR - Error processing callback for node 69cc701a8cd9cbcdc9459f52: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34216 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:09:13,213 - lava_callback - INFO - Received callback for node 69cf0ead9e2394c38fe015de INFO: 172.21.0.1:37560 - "POST /node/69cf0ead9e2394c38fe015de HTTP/1.0" 202 Accepted 2026-04-03 01:09:13,266 - lava_callback - INFO - Starting async job submit for node 69cf0ead9e2394c38fe015de 2026-04-03 01:09:13,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:13,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ead9e2394c38fe015de HTTP/1.1" 200 None 2026-04-03 01:09:13,328 - lava_callback - DEBUG - Processing node 69cf0ead9e2394c38fe015de: baseline-arm-pengutronix 2026-04-03 01:09:13,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:09:13,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:09:13,608 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0ead9e2394c38fe015de/log.txt.gz 2026-04-03 01:09:13,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:09:13,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:09:13,865 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0ead9e2394c38fe015de/lava-logs.yaml 2026-04-03 01:09:13,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:09:13,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:09:13,986 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf0ead9e2394c38fe015de/lava_callback.json.gz DEBUG: Setup failed for 69cf0ead9e2394c38fe015de. Transitting job node result: incomplete -> fail 2026-04-03 01:09:14,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:14,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ead9e2394c38fe015de HTTP/1.1" 200 None 2026-04-03 01:09:14,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:14,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:09:14,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:14,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ead9e2394c38fe015de HTTP/1.1" 200 None 2026-04-03 01:09:14,153 - lava_callback - INFO - Completed processing callback for node 69cf0ead9e2394c38fe015de INFO: 172.21.0.1:37576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50466 - "GET /node/69376d4f0c55536e7ddd9f6b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:50480 - "GET /node/69892e0585773790cd34b603%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-03 01:09:32,754 - lava_callback - INFO - Received callback for node 69cc70448cd9cbcdc9459fb7 INFO: 172.21.0.1:54648 - "POST /node/69cc70448cd9cbcdc9459fb7 HTTP/1.0" 202 Accepted 2026-04-03 01:09:32,761 - lava_callback - INFO - Starting async job submit for node 69cc70448cd9cbcdc9459fb7 2026-04-03 01:09:32,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:32,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70448cd9cbcdc9459fb7 HTTP/1.1" 200 None 2026-04-03 01:09:32,777 - lava_callback - DEBUG - Processing node 69cc70448cd9cbcdc9459fb7: baseline-arm-clabbe 2026-04-03 01:09:32,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc70448cd9cbcdc9459fb7 2026-04-03 01:09:32,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:32,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70448cd9cbcdc9459fb7 HTTP/1.1" 200 None 2026-04-03 01:09:32,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:32,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:09:32,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:32,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70448cd9cbcdc9459fb7 HTTP/1.1" 200 None 2026-04-03 01:09:32,852 - lava_callback - ERROR - Error processing callback for node 69cc70448cd9cbcdc9459fb7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:09:35,622 - lava_callback - INFO - Received callback for node 69cf13409e2394c38fe01bd1 INFO: 172.21.0.1:54660 - "POST /node/69cf13409e2394c38fe01bd1 HTTP/1.0" 202 Accepted 2026-04-03 01:09:35,628 - lava_callback - INFO - Starting async job submit for node 69cf13409e2394c38fe01bd1 2026-04-03 01:09:35,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:35,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf13409e2394c38fe01bd1 HTTP/1.1" 200 None 2026-04-03 01:09:35,654 - lava_callback - DEBUG - Processing node 69cf13409e2394c38fe01bd1: baseline-arm 2026-04-03 01:09:35,662 - lava_callback - WARNING - Failed to upload log 2026-04-03 01:09:35,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:09:35,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:09:35,813 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf13409e2394c38fe01bd1/lava-logs.yaml 2026-04-03 01:09:35,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:09:35,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:09:35,958 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf13409e2394c38fe01bd1/lava_callback.json.gz 2026-04-03 01:09:35,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:35,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf13409e2394c38fe01bd1 HTTP/1.1" 200 None 2026-04-03 01:09:35,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:35,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:09:35,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:36,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf13409e2394c38fe01bd1 HTTP/1.1" 200 None 2026-04-03 01:09:36,038 - lava_callback - INFO - Completed processing callback for node 69cf13409e2394c38fe01bd1 INFO: 172.21.0.1:54666 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:09:41,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:09:41,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36582 - "GET /node/693f36df24745582e399e111%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:36592 - "GET /node/6940ac811e3c24e95681a98e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:36596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36746 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:10:13,159 - lava_callback - INFO - Received callback for node 69cc706d8cd9cbcdc945a03b INFO: 172.21.0.1:54870 - "POST /node/69cc706d8cd9cbcdc945a03b HTTP/1.0" 202 Accepted 2026-04-03 01:10:13,165 - lava_callback - INFO - Starting async job submit for node 69cc706d8cd9cbcdc945a03b 2026-04-03 01:10:13,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:10:13,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc706d8cd9cbcdc945a03b HTTP/1.1" 200 None 2026-04-03 01:10:13,180 - lava_callback - DEBUG - Processing node 69cc706d8cd9cbcdc945a03b: baseline-arm-clabbe 2026-04-03 01:10:13,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc706d8cd9cbcdc945a03b 2026-04-03 01:10:13,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:10:13,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc706d8cd9cbcdc945a03b HTTP/1.1" 200 None 2026-04-03 01:10:13,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:10:13,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:10:13,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:10:13,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc706d8cd9cbcdc945a03b HTTP/1.1" 200 None 2026-04-03 01:10:13,241 - lava_callback - ERROR - Error processing callback for node 69cc706d8cd9cbcdc945a03b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49136 - "GET /node/69c16cc72b2b51e9ad97a0f2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49142 - "GET /node/693f354c24745582e399de95%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52904 - "GET /node/693b854e41fb4c9509765d7d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:50512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46954 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:12:12,908 - lava_callback - INFO - Received callback for node 69cc70da8cd9cbcdc945a0b2 INFO: 172.21.0.1:55966 - "POST /node/69cc70da8cd9cbcdc945a0b2 HTTP/1.0" 202 Accepted 2026-04-03 01:12:12,916 - lava_callback - INFO - Starting async job submit for node 69cc70da8cd9cbcdc945a0b2 2026-04-03 01:12:12,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:12,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70da8cd9cbcdc945a0b2 HTTP/1.1" 200 None 2026-04-03 01:12:12,941 - lava_callback - DEBUG - Processing node 69cc70da8cd9cbcdc945a0b2: baseline-arm-clabbe 2026-04-03 01:12:12,941 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc70da8cd9cbcdc945a0b2 2026-04-03 01:12:12,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:12,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc70da8cd9cbcdc945a0b2 HTTP/1.1" 200 None 2026-04-03 01:12:12,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:12,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 01:12:12,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:13,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc70da8cd9cbcdc945a0b2 HTTP/1.1" 200 None 2026-04-03 01:12:13,014 - lava_callback - ERROR - Error processing callback for node 69cc70da8cd9cbcdc945a0b2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59018 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:12:52,848 - lava_callback - INFO - Received callback for node 69cc71068cd9cbcdc945a0c8 INFO: 172.21.0.1:38990 - "POST /node/69cc71068cd9cbcdc945a0c8 HTTP/1.0" 202 Accepted 2026-04-03 01:12:52,857 - lava_callback - INFO - Starting async job submit for node 69cc71068cd9cbcdc945a0c8 2026-04-03 01:12:52,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:52,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc71068cd9cbcdc945a0c8 HTTP/1.1" 200 None 2026-04-03 01:12:52,884 - lava_callback - DEBUG - Processing node 69cc71068cd9cbcdc945a0c8: baseline-arm-clabbe 2026-04-03 01:12:52,885 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc71068cd9cbcdc945a0c8 2026-04-03 01:12:52,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:52,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc71068cd9cbcdc945a0c8 HTTP/1.1" 200 None 2026-04-03 01:12:52,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:52,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-03 01:12:52,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:52,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc71068cd9cbcdc945a0c8 HTTP/1.1" 200 None 2026-04-03 01:12:52,992 - lava_callback - ERROR - Error processing callback for node 69cc71068cd9cbcdc945a0c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38992 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:12:56,778 - lava_callback - INFO - Received callback for node 69cf13b89e2394c38fe01c02 INFO: 172.21.0.1:38998 - "POST /node/69cf13b89e2394c38fe01c02 HTTP/1.0" 202 Accepted 2026-04-03 01:12:56,784 - lava_callback - INFO - Starting async job submit for node 69cf13b89e2394c38fe01c02 2026-04-03 01:12:56,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:56,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf13b89e2394c38fe01c02 HTTP/1.1" 200 None 2026-04-03 01:12:56,826 - lava_callback - DEBUG - Processing node 69cf13b89e2394c38fe01c02: baseline-x86 2026-04-03 01:12:57,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:12:57,435 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:12:57,437 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69cf13b89e2394c38fe01c02/log.txt.gz 2026-04-03 01:12:57,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:12:58,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:12:58,061 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69cf13b89e2394c38fe01c02/lava-logs.yaml 2026-04-03 01:12:58,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:12:58,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:12:58,210 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69cf13b89e2394c38fe01c02/lava_callback.json.gz 2026-04-03 01:12:58,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:58,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf13b89e2394c38fe01c02 HTTP/1.1" 200 None 2026-04-03 01:12:58,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:58,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf071c9e2394c38fe00f6d HTTP/1.1" 200 None 2026-04-03 01:12:58,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:12:58,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf13b89e2394c38fe01c02 HTTP/1.1" 200 None 2026-04-03 01:12:58,323 - lava_callback - INFO - Completed processing callback for node 69cf13b89e2394c38fe01c02 INFO: 172.21.0.1:50954 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:13:11,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:13:11,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55862 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 01:13:15,223 - lava_callback - INFO - Received callback for node 69cc71288cd9cbcdc945a0dd INFO: 172.21.0.1:55870 - "POST /node/69cc71288cd9cbcdc945a0dd HTTP/1.0" 202 Accepted 2026-04-03 01:13:15,231 - lava_callback - INFO - Starting async job submit for node 69cc71288cd9cbcdc945a0dd 2026-04-03 01:13:15,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:13:15,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc71288cd9cbcdc945a0dd HTTP/1.1" 200 None 2026-04-03 01:13:15,267 - lava_callback - DEBUG - Processing node 69cc71288cd9cbcdc945a0dd: baseline-arm-baylibre 2026-04-03 01:13:15,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc71288cd9cbcdc945a0dd 2026-04-03 01:13:15,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:13:15,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc71288cd9cbcdc945a0dd HTTP/1.1" 200 None 2026-04-03 01:13:15,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:13:15,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-03 01:13:15,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:13:15,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc71288cd9cbcdc945a0dd HTTP/1.1" 200 None 2026-04-03 01:13:15,382 - lava_callback - ERROR - Error processing callback for node 69cc71288cd9cbcdc945a0dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44334 - "GET /node/698af4963b3beb19e7616a63%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:44344 - "GET /node/694bd92bc19e09af58d49b02%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-03 01:13:55,167 - lava_callback - INFO - Received callback for node 69cc71518cd9cbcdc945a0ec INFO: 172.21.0.1:44358 - "POST /node/69cc71518cd9cbcdc945a0ec HTTP/1.0" 202 Accepted 2026-04-03 01:13:55,173 - lava_callback - INFO - Starting async job submit for node 69cc71518cd9cbcdc945a0ec 2026-04-03 01:13:55,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:13:55,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc71518cd9cbcdc945a0ec HTTP/1.1" 200 None 2026-04-03 01:13:55,217 - lava_callback - DEBUG - Processing node 69cc71518cd9cbcdc945a0ec: baseline-arm-baylibre 2026-04-03 01:13:55,217 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc71518cd9cbcdc945a0ec 2026-04-03 01:13:55,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:13:55,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc71518cd9cbcdc945a0ec HTTP/1.1" 200 None 2026-04-03 01:13:55,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:13:55,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-03 01:13:55,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:13:55,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc71518cd9cbcdc945a0ec HTTP/1.1" 200 None 2026-04-03 01:13:55,341 - lava_callback - ERROR - Error processing callback for node 69cc71518cd9cbcdc945a0ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43872 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:14:49,725 - lava_callback - INFO - Received callback for node 69cf0fd59e2394c38fe0182f INFO: 172.21.0.1:56670 - "POST /node/69cf0fd59e2394c38fe0182f HTTP/1.0" 202 Accepted 2026-04-03 01:14:49,730 - lava_callback - INFO - Starting async job submit for node 69cf0fd59e2394c38fe0182f 2026-04-03 01:14:49,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:14:49,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fd59e2394c38fe0182f HTTP/1.1" 200 None 2026-04-03 01:14:49,768 - lava_callback - DEBUG - Processing node 69cf0fd59e2394c38fe0182f: baseline-arm-broonie 2026-04-03 01:14:49,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:14:50,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:14:50,319 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0fd59e2394c38fe0182f/log.txt.gz 2026-04-03 01:14:50,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:14:50,984 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:14:50,986 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0fd59e2394c38fe0182f/lava-logs.yaml 2026-04-03 01:14:50,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:14:51,223 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:14:51,225 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0fd59e2394c38fe0182f/lava_callback.json.gz 2026-04-03 01:14:51,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:14:51,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fd59e2394c38fe0182f HTTP/1.1" 200 None 2026-04-03 01:14:51,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:14:51,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:14:51,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:14:51,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fd59e2394c38fe0182f HTTP/1.1" 200 None 2026-04-03 01:14:51,331 - lava_callback - INFO - Completed processing callback for node 69cf0fd59e2394c38fe0182f INFO: 172.21.0.1:56672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40292 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:15:11,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:15:11,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46470 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:15:27,255 - lava_callback - INFO - Received callback for node 69cf0fb69e2394c38fe017b1 INFO: 172.21.0.1:46480 - "POST /node/69cf0fb69e2394c38fe017b1 HTTP/1.0" 202 Accepted 2026-04-03 01:15:27,264 - lava_callback - INFO - Starting async job submit for node 69cf0fb69e2394c38fe017b1 2026-04-03 01:15:27,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:15:27,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb69e2394c38fe017b1 HTTP/1.1" 200 None 2026-04-03 01:15:27,333 - lava_callback - DEBUG - Processing node 69cf0fb69e2394c38fe017b1: baseline-nfs-arm64 2026-04-03 01:15:28,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:15:28,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:15:28,661 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb69e2394c38fe017b1/log.txt.gz 2026-04-03 01:15:29,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:15:29,591 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:15:29,594 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb69e2394c38fe017b1/lava-logs.yaml 2026-04-03 01:15:29,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:15:29,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:15:29,817 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf0fb69e2394c38fe017b1/lava_callback.json.gz DEBUG: Setup failed for 69cf0fb69e2394c38fe017b1. Transitting job node result: incomplete -> fail 2026-04-03 01:15:29,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:15:29,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fb69e2394c38fe017b1 HTTP/1.1" 200 None 2026-04-03 01:15:29,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:15:29,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:15:29,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:15:29,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fb69e2394c38fe017b1 HTTP/1.1" 200 None 2026-04-03 01:15:30,051 - lava_callback - INFO - Completed processing callback for node 69cf0fb69e2394c38fe017b1 INFO: 172.21.0.1:52772 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:15:41,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:15:41,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48350 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48356 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:17:20,051 - lava_callback - INFO - Received callback for node 69cf0e8f9e2394c38fe0156e INFO: 172.21.0.1:58554 - "POST /node/69cf0e8f9e2394c38fe0156e HTTP/1.0" 202 Accepted 2026-04-03 01:17:20,057 - lava_callback - INFO - Starting async job submit for node 69cf0e8f9e2394c38fe0156e 2026-04-03 01:17:20,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:17:20,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e8f9e2394c38fe0156e HTTP/1.1" 200 None 2026-04-03 01:17:20,095 - lava_callback - DEBUG - Processing node 69cf0e8f9e2394c38fe0156e: baseline-arm-broonie 2026-04-03 01:17:20,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:17:20,495 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:17:20,497 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0e8f9e2394c38fe0156e/log.txt.gz 2026-04-03 01:17:20,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:17:20,842 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:17:20,844 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0e8f9e2394c38fe0156e/lava-logs.yaml 2026-04-03 01:17:20,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:17:21,335 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:17:21,337 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0e8f9e2394c38fe0156e/lava_callback.json.gz 2026-04-03 01:17:21,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:17:21,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e8f9e2394c38fe0156e HTTP/1.1" 200 None 2026-04-03 01:17:21,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:17:21,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:17:21,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:17:21,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0e8f9e2394c38fe0156e HTTP/1.1" 200 None 2026-04-03 01:17:21,422 - lava_callback - INFO - Completed processing callback for node 69cf0e8f9e2394c38fe0156e INFO: 172.21.0.1:58566 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:17:34,895 - lava_callback - INFO - Received callback for node 69cf0e929e2394c38fe01571 INFO: 172.21.0.1:50738 - "POST /node/69cf0e929e2394c38fe01571 HTTP/1.0" 202 Accepted 2026-04-03 01:17:34,902 - lava_callback - INFO - Starting async job submit for node 69cf0e929e2394c38fe01571 2026-04-03 01:17:34,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:17:34,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e929e2394c38fe01571 HTTP/1.1" 200 None 2026-04-03 01:17:34,952 - lava_callback - DEBUG - Processing node 69cf0e929e2394c38fe01571: baseline-arm-broonie 2026-04-03 01:17:35,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:17:35,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:17:35,761 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0e929e2394c38fe01571/log.txt.gz 2026-04-03 01:17:35,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:17:36,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:17:36,203 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0e929e2394c38fe01571/lava-logs.yaml 2026-04-03 01:17:36,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:17:36,362 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:17:36,364 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0e929e2394c38fe01571/lava_callback.json.gz 2026-04-03 01:17:36,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:17:36,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0e929e2394c38fe01571 HTTP/1.1" 200 None 2026-04-03 01:17:36,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:17:36,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:17:36,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:17:36,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0e929e2394c38fe01571 HTTP/1.1" 200 None 2026-04-03 01:17:36,470 - lava_callback - INFO - Completed processing callback for node 69cf0e929e2394c38fe01571 INFO: 172.21.0.1:50752 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:17:41,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:17:41,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44782 - "GET /node/69b1362aa870a1dc3343c485%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:47190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35522 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:20:01,062 - lava_callback - INFO - Received callback for node 69cf15ac9e2394c38fe01cdf INFO: 172.21.0.1:51066 - "POST /node/69cf15ac9e2394c38fe01cdf HTTP/1.0" 202 Accepted 2026-04-03 01:20:01,070 - lava_callback - INFO - Starting async job submit for node 69cf15ac9e2394c38fe01cdf 2026-04-03 01:20:01,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:20:01,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf15ac9e2394c38fe01cdf HTTP/1.1" 200 None 2026-04-03 01:20:01,093 - lava_callback - DEBUG - Processing node 69cf15ac9e2394c38fe01cdf: baseline-nfs-arm64 2026-04-03 01:20:01,094 - lava_callback - WARNING - Failed to upload log 2026-04-03 01:20:01,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:20:01,247 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:20:01,249 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf15ac9e2394c38fe01cdf/lava-logs.yaml 2026-04-03 01:20:01,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:20:01,418 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:20:01,420 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf15ac9e2394c38fe01cdf/lava_callback.json.gz 2026-04-03 01:20:01,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:20:01,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf15ac9e2394c38fe01cdf HTTP/1.1" 200 None 2026-04-03 01:20:01,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:20:01,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:20:01,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:20:01,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf15ac9e2394c38fe01cdf HTTP/1.1" 200 None 2026-04-03 01:20:01,493 - lava_callback - INFO - Completed processing callback for node 69cf15ac9e2394c38fe01cdf INFO: 172.21.0.1:51072 - "GET /node/693934766e7b0b7f22e51eea%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51074 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:20:11,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:20:11,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49842 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:20:53,329 - lava_callback - INFO - Received callback for node 69cc72e58cd9cbcdc945a26c INFO: 172.21.0.1:40462 - "POST /node/69cc72e58cd9cbcdc945a26c HTTP/1.0" 202 Accepted 2026-04-03 01:20:53,335 - lava_callback - INFO - Starting async job submit for node 69cc72e58cd9cbcdc945a26c 2026-04-03 01:20:53,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:20:53,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc72e58cd9cbcdc945a26c HTTP/1.1" 200 None 2026-04-03 01:20:53,357 - lava_callback - DEBUG - Processing node 69cc72e58cd9cbcdc945a26c: baseline-arm-clabbe 2026-04-03 01:20:53,358 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc72e58cd9cbcdc945a26c 2026-04-03 01:20:53,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:20:53,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc72e58cd9cbcdc945a26c HTTP/1.1" 200 None 2026-04-03 01:20:53,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:20:53,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-03 01:20:53,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:20:53,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc72e58cd9cbcdc945a26c HTTP/1.1" 200 None 2026-04-03 01:20:53,422 - lava_callback - ERROR - Error processing callback for node 69cc72e58cd9cbcdc945a26c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET /node/6953201331522df6df79a4cf%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57988 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 01:22:16,125 - lava_callback - INFO - Received callback for node 69cf0ef79e2394c38fe0168d INFO: 172.21.0.1:58002 - "POST /node/69cf0ef79e2394c38fe0168d HTTP/1.0" 202 Accepted 2026-04-03 01:22:16,133 - lava_callback - INFO - Starting async job submit for node 69cf0ef79e2394c38fe0168d 2026-04-03 01:22:16,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:22:16,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ef79e2394c38fe0168d HTTP/1.1" 200 None 2026-04-03 01:22:16,177 - lava_callback - DEBUG - Processing node 69cf0ef79e2394c38fe0168d: ltp-syscalls INFO: 172.21.0.1:58006 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:22:17,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:22:17,224 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:22:17,226 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cf0ef79e2394c38fe0168d/log.txt.gz 2026-04-03 01:22:17,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:22:18,143 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:22:18,146 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cf0ef79e2394c38fe0168d/lava-logs.yaml 2026-04-03 01:22:18,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:22:18,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:22:18,329 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69cf0ef79e2394c38fe0168d/lava_callback.json.gz 2026-04-03 01:22:18,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:22:18,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ef79e2394c38fe0168d HTTP/1.1" 200 None 2026-04-03 01:22:18,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:22:18,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:22:18,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:22:18,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ef79e2394c38fe0168d HTTP/1.1" 200 None 2026-04-03 01:22:18,417 - lava_callback - INFO - Completed processing callback for node 69cf0ef79e2394c38fe0168d INFO: 172.21.0.1:44338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56754 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:22:41,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:22:41,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50920 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:24:33,375 - lava_callback - INFO - Received callback for node 69cc73c48cd9cbcdc945a31a INFO: 172.21.0.1:53134 - "POST /node/69cc73c48cd9cbcdc945a31a HTTP/1.0" 202 Accepted 2026-04-03 01:24:33,384 - lava_callback - INFO - Starting async job submit for node 69cc73c48cd9cbcdc945a31a 2026-04-03 01:24:33,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:24:33,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc73c48cd9cbcdc945a31a HTTP/1.1" 200 None 2026-04-03 01:24:33,414 - lava_callback - DEBUG - Processing node 69cc73c48cd9cbcdc945a31a: baseline-arm-clabbe 2026-04-03 01:24:33,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc73c48cd9cbcdc945a31a 2026-04-03 01:24:33,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:24:33,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc73c48cd9cbcdc945a31a HTTP/1.1" 200 None 2026-04-03 01:24:33,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:24:33,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-03 01:24:33,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:24:33,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc73c48cd9cbcdc945a31a HTTP/1.1" 200 None 2026-04-03 01:24:33,495 - lava_callback - ERROR - Error processing callback for node 69cc73c48cd9cbcdc945a31a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55192 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:25:13,351 - lava_callback - INFO - Received callback for node 69cc73ec8cd9cbcdc945a33f INFO: 172.21.0.1:44624 - "POST /node/69cc73ec8cd9cbcdc945a33f HTTP/1.0" 202 Accepted 2026-04-03 01:25:13,360 - lava_callback - INFO - Starting async job submit for node 69cc73ec8cd9cbcdc945a33f 2026-04-03 01:25:13,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:25:13,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc73ec8cd9cbcdc945a33f HTTP/1.1" 200 None 2026-04-03 01:25:13,392 - lava_callback - DEBUG - Processing node 69cc73ec8cd9cbcdc945a33f: baseline-arm-clabbe 2026-04-03 01:25:13,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc73ec8cd9cbcdc945a33f 2026-04-03 01:25:13,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:25:13,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc73ec8cd9cbcdc945a33f HTTP/1.1" 200 None 2026-04-03 01:25:13,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:25:13,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-03 01:25:13,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:25:13,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc73ec8cd9cbcdc945a33f HTTP/1.1" 200 None 2026-04-03 01:25:13,482 - lava_callback - ERROR - Error processing callback for node 69cc73ec8cd9cbcdc945a33f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48404 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:25:53,436 - lava_callback - INFO - Received callback for node 69cc74158cd9cbcdc945a346 INFO: 172.21.0.1:48318 - "POST /node/69cc74158cd9cbcdc945a346 HTTP/1.0" 202 Accepted 2026-04-03 01:25:53,445 - lava_callback - INFO - Starting async job submit for node 69cc74158cd9cbcdc945a346 2026-04-03 01:25:53,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:25:53,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc74158cd9cbcdc945a346 HTTP/1.1" 200 None 2026-04-03 01:25:53,475 - lava_callback - DEBUG - Processing node 69cc74158cd9cbcdc945a346: baseline-arm-clabbe 2026-04-03 01:25:53,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc74158cd9cbcdc945a346 2026-04-03 01:25:53,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:25:53,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc74158cd9cbcdc945a346 HTTP/1.1" 200 None 2026-04-03 01:25:53,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:25:53,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-04-03 01:25:53,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:25:53,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc74158cd9cbcdc945a346 HTTP/1.1" 200 None 2026-04-03 01:25:53,576 - lava_callback - ERROR - Error processing callback for node 69cc74158cd9cbcdc945a346: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43738 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:29:25,844 - lava_callback - INFO - Received callback for node 69cf0fca9e2394c38fe0180b INFO: 172.21.0.1:60078 - "POST /node/69cf0fca9e2394c38fe0180b HTTP/1.0" 202 Accepted 2026-04-03 01:29:25,850 - lava_callback - INFO - Starting async job submit for node 69cf0fca9e2394c38fe0180b 2026-04-03 01:29:25,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:29:25,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fca9e2394c38fe0180b HTTP/1.1" 200 None 2026-04-03 01:29:25,889 - lava_callback - DEBUG - Processing node 69cf0fca9e2394c38fe0180b: baseline-arm64-broonie 2026-04-03 01:29:26,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:29:26,250 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:29:26,252 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fca9e2394c38fe0180b/log.txt.gz 2026-04-03 01:29:26,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:60080 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:29:26,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:29:26,623 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fca9e2394c38fe0180b/lava-logs.yaml 2026-04-03 01:29:26,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:29:26,772 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:29:26,774 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fca9e2394c38fe0180b/lava_callback.json.gz 2026-04-03 01:29:26,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:29:26,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fca9e2394c38fe0180b HTTP/1.1" 200 None 2026-04-03 01:29:26,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:29:26,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:29:26,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:29:26,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fca9e2394c38fe0180b HTTP/1.1" 200 None 2026-04-03 01:29:26,871 - lava_callback - INFO - Completed processing callback for node 69cf0fca9e2394c38fe0180b INFO: 172.21.0.1:50464 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:29:41,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:29:41,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55016 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 01:31:16,139 - lava_callback - INFO - Received callback for node 69cc754f8cd9cbcdc945a442 INFO: 172.21.0.1:55022 - "POST /node/69cc754f8cd9cbcdc945a442 HTTP/1.0" 202 Accepted 2026-04-03 01:31:16,145 - lava_callback - INFO - Starting async job submit for node 69cc754f8cd9cbcdc945a442 2026-04-03 01:31:16,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:31:16,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc754f8cd9cbcdc945a442 HTTP/1.1" 200 None 2026-04-03 01:31:16,161 - lava_callback - DEBUG - Processing node 69cc754f8cd9cbcdc945a442: baseline-arm64-baylibre 2026-04-03 01:31:16,161 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc754f8cd9cbcdc945a442 2026-04-03 01:31:16,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:31:16,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc754f8cd9cbcdc945a442 HTTP/1.1" 200 None 2026-04-03 01:31:16,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:31:16,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:31:16,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:31:16,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc754f8cd9cbcdc945a442 HTTP/1.1" 200 None 2026-04-03 01:31:16,231 - lava_callback - ERROR - Error processing callback for node 69cc754f8cd9cbcdc945a442: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36992 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:32:20,718 - lava_callback - INFO - Received callback for node 69cf18629e2394c38fe01e15 INFO: 172.21.0.1:46932 - "POST /node/69cf18629e2394c38fe01e15 HTTP/1.0" 202 Accepted 2026-04-03 01:32:20,729 - lava_callback - INFO - Starting async job submit for node 69cf18629e2394c38fe01e15 2026-04-03 01:32:20,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:20,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf18629e2394c38fe01e15 HTTP/1.1" 200 None 2026-04-03 01:32:20,794 - lava_callback - DEBUG - Processing node 69cf18629e2394c38fe01e15: baseline-arm64 2026-04-03 01:32:20,935 - lava_callback - WARNING - Failed to upload log 2026-04-03 01:32:21,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:32:21,218 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:32:21,220 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf18629e2394c38fe01e15/lava-logs.yaml 2026-04-03 01:32:21,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:32:21,361 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:32:21,363 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf18629e2394c38fe01e15/lava_callback.json.gz 2026-04-03 01:32:21,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:21,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf18629e2394c38fe01e15 HTTP/1.1" 200 None 2026-04-03 01:32:21,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:21,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:32:21,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:21,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf18629e2394c38fe01e15 HTTP/1.1" 200 None 2026-04-03 01:32:21,570 - lava_callback - INFO - Completed processing callback for node 69cf18629e2394c38fe01e15 INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:32:36,210 - lava_callback - INFO - Received callback for node 69cc75ab8cd9cbcdc945a4ba INFO: 172.21.0.1:36154 - "POST /node/69cc75ab8cd9cbcdc945a4ba HTTP/1.0" 202 Accepted 2026-04-03 01:32:36,216 - lava_callback - INFO - Starting async job submit for node 69cc75ab8cd9cbcdc945a4ba 2026-04-03 01:32:36,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:36,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc75ab8cd9cbcdc945a4ba HTTP/1.1" 200 None 2026-04-03 01:32:36,236 - lava_callback - DEBUG - Processing node 69cc75ab8cd9cbcdc945a4ba: baseline-arm64-baylibre 2026-04-03 01:32:36,236 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc75ab8cd9cbcdc945a4ba 2026-04-03 01:32:36,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:36,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc75ab8cd9cbcdc945a4ba HTTP/1.1" 200 None 2026-04-03 01:32:36,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:36,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:32:36,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:36,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc75ab8cd9cbcdc945a4ba HTTP/1.1" 200 None 2026-04-03 01:32:36,294 - lava_callback - ERROR - Error processing callback for node 69cc75ab8cd9cbcdc945a4ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36170 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:32:41,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:41,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 01:32:43,150 - lava_callback - INFO - Received callback for node 69cf17669e2394c38fe01d99 INFO: 172.21.0.1:40716 - "POST /node/69cf17669e2394c38fe01d99 HTTP/1.0" 202 Accepted 2026-04-03 01:32:43,156 - lava_callback - INFO - Starting async job submit for node 69cf17669e2394c38fe01d99 2026-04-03 01:32:43,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:43,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf17669e2394c38fe01d99 HTTP/1.1" 200 None 2026-04-03 01:32:43,195 - lava_callback - DEBUG - Processing node 69cf17669e2394c38fe01d99: baseline-arm64-broonie 2026-04-03 01:32:43,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:32:43,557 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:32:43,559 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf17669e2394c38fe01d99/log.txt.gz 2026-04-03 01:32:43,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:32:43,900 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:32:43,902 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf17669e2394c38fe01d99/lava-logs.yaml 2026-04-03 01:32:43,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:32:44,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:32:44,041 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf17669e2394c38fe01d99/lava_callback.json.gz 2026-04-03 01:32:44,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:44,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf17669e2394c38fe01d99 HTTP/1.1" 200 None 2026-04-03 01:32:44,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:44,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:32:44,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:32:44,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf17669e2394c38fe01d99 HTTP/1.1" 200 None 2026-04-03 01:32:44,124 - lava_callback - INFO - Completed processing callback for node 69cf17669e2394c38fe01d99 INFO: 172.21.0.1:40730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48062 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:33:11,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:33:11,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32838 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32850 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:33:26,321 - lava_callback - INFO - Received callback for node 69cf0fcf9e2394c38fe01814 INFO: 172.21.0.1:57098 - "POST /node/69cf0fcf9e2394c38fe01814 HTTP/1.0" 202 Accepted 2026-04-03 01:33:26,327 - lava_callback - INFO - Starting async job submit for node 69cf0fcf9e2394c38fe01814 2026-04-03 01:33:26,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:33:26,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fcf9e2394c38fe01814 HTTP/1.1" 200 None 2026-04-03 01:33:26,379 - lava_callback - DEBUG - Processing node 69cf0fcf9e2394c38fe01814: baseline-arm64-broonie INFO: 172.21.0.1:57106 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:33:26,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:33:26,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:33:26,825 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fcf9e2394c38fe01814/log.txt.gz 2026-04-03 01:33:26,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:33:27,203 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:33:27,205 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fcf9e2394c38fe01814/lava-logs.yaml 2026-04-03 01:33:27,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:33:27,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:33:27,346 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fcf9e2394c38fe01814/lava_callback.json.gz 2026-04-03 01:33:27,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:33:27,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fcf9e2394c38fe01814 HTTP/1.1" 200 None 2026-04-03 01:33:27,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:33:27,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:33:27,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:33:27,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fcf9e2394c38fe01814 HTTP/1.1" 200 None 2026-04-03 01:33:27,435 - lava_callback - INFO - Completed processing callback for node 69cf0fcf9e2394c38fe01814 INFO: 172.21.0.1:49434 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:33:41,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:33:41,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42020 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 01:34:16,293 - lava_callback - INFO - Received callback for node 69cc76038cd9cbcdc945a4fb INFO: 172.21.0.1:42024 - "POST /node/69cc76038cd9cbcdc945a4fb HTTP/1.0" 202 Accepted 2026-04-03 01:34:16,305 - lava_callback - INFO - Starting async job submit for node 69cc76038cd9cbcdc945a4fb 2026-04-03 01:34:16,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:34:16,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc76038cd9cbcdc945a4fb HTTP/1.1" 200 None 2026-04-03 01:34:16,335 - lava_callback - DEBUG - Processing node 69cc76038cd9cbcdc945a4fb: baseline-arm64-baylibre 2026-04-03 01:34:16,335 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc76038cd9cbcdc945a4fb 2026-04-03 01:34:16,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:34:16,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc76038cd9cbcdc945a4fb HTTP/1.1" 200 None 2026-04-03 01:34:16,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:34:16,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:34:16,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:34:16,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc76038cd9cbcdc945a4fb HTTP/1.1" 200 None 2026-04-03 01:34:16,425 - lava_callback - ERROR - Error processing callback for node 69cc76038cd9cbcdc945a4fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55630 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:35:12,688 - lava_callback - INFO - Received callback for node 69cf190a9e2394c38fe01e77 INFO: 172.21.0.1:50728 - "POST /node/69cf190a9e2394c38fe01e77 HTTP/1.0" 202 Accepted 2026-04-03 01:35:12,699 - lava_callback - INFO - Starting async job submit for node 69cf190a9e2394c38fe01e77 2026-04-03 01:35:12,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:35:12,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf190a9e2394c38fe01e77 HTTP/1.1" 200 None 2026-04-03 01:35:12,771 - lava_callback - DEBUG - Processing node 69cf190a9e2394c38fe01e77: baseline-arm64 2026-04-03 01:35:13,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:35:13,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:35:13,211 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf190a9e2394c38fe01e77/log.txt.gz 2026-04-03 01:35:13,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:35:13,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:35:13,522 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf190a9e2394c38fe01e77/lava-logs.yaml 2026-04-03 01:35:13,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:35:13,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:35:13,726 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf190a9e2394c38fe01e77/lava_callback.json.gz 2026-04-03 01:35:13,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:35:13,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf190a9e2394c38fe01e77 HTTP/1.1" 200 None 2026-04-03 01:35:13,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:35:13,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:35:13,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:35:13,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf190a9e2394c38fe01e77 HTTP/1.1" 200 None 2026-04-03 01:35:13,933 - lava_callback - INFO - Completed processing callback for node 69cf190a9e2394c38fe01e77 INFO: 172.21.0.1:50738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57702 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:35:36,331 - lava_callback - INFO - Received callback for node 69cc765c8cd9cbcdc945a558 INFO: 172.21.0.1:46678 - "POST /node/69cc765c8cd9cbcdc945a558 HTTP/1.0" 202 Accepted 2026-04-03 01:35:36,340 - lava_callback - INFO - Starting async job submit for node 69cc765c8cd9cbcdc945a558 2026-04-03 01:35:36,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:35:36,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc765c8cd9cbcdc945a558 HTTP/1.1" 200 None 2026-04-03 01:35:36,368 - lava_callback - DEBUG - Processing node 69cc765c8cd9cbcdc945a558: baseline-arm64-baylibre 2026-04-03 01:35:36,368 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc765c8cd9cbcdc945a558 2026-04-03 01:35:36,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:35:36,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc765c8cd9cbcdc945a558 HTTP/1.1" 200 None 2026-04-03 01:35:36,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:35:36,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:35:36,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:35:36,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc765c8cd9cbcdc945a558 HTTP/1.1" 200 None 2026-04-03 01:35:36,442 - lava_callback - ERROR - Error processing callback for node 69cc765c8cd9cbcdc945a558: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:35:41,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:35:41,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47678 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:36:32,429 - lava_callback - INFO - Received callback for node 69cf0fd19e2394c38fe01824 INFO: 172.21.0.1:41018 - "POST /node/69cf0fd19e2394c38fe01824 HTTP/1.0" 202 Accepted 2026-04-03 01:36:32,438 - lava_callback - INFO - Starting async job submit for node 69cf0fd19e2394c38fe01824 2026-04-03 01:36:32,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:36:32,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fd19e2394c38fe01824 HTTP/1.1" 200 None 2026-04-03 01:36:32,489 - lava_callback - DEBUG - Processing node 69cf0fd19e2394c38fe01824: baseline-arm64-broonie 2026-04-03 01:36:32,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:36:33,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:36:33,002 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd19e2394c38fe01824/log.txt.gz 2026-04-03 01:36:33,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:36:33,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:36:33,498 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd19e2394c38fe01824/lava-logs.yaml 2026-04-03 01:36:33,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:36:33,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:36:33,658 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd19e2394c38fe01824/lava_callback.json.gz 2026-04-03 01:36:33,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:36:33,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fd19e2394c38fe01824 HTTP/1.1" 200 None 2026-04-03 01:36:33,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:36:33,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:36:33,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:36:33,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fd19e2394c38fe01824 HTTP/1.1" 200 None 2026-04-03 01:36:33,766 - lava_callback - INFO - Completed processing callback for node 69cf0fd19e2394c38fe01824 INFO: 172.21.0.1:41020 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:36:41,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:36:41,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54590 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:36:56,431 - lava_callback - INFO - Received callback for node 69cc76b58cd9cbcdc945a5b5 INFO: 172.21.0.1:57798 - "POST /node/69cc76b58cd9cbcdc945a5b5 HTTP/1.0" 202 Accepted 2026-04-03 01:36:56,437 - lava_callback - INFO - Starting async job submit for node 69cc76b58cd9cbcdc945a5b5 2026-04-03 01:36:56,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:36:56,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc76b58cd9cbcdc945a5b5 HTTP/1.1" 200 None 2026-04-03 01:36:56,453 - lava_callback - DEBUG - Processing node 69cc76b58cd9cbcdc945a5b5: baseline-arm64-baylibre 2026-04-03 01:36:56,453 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc76b58cd9cbcdc945a5b5 2026-04-03 01:36:56,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:36:56,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc76b58cd9cbcdc945a5b5 HTTP/1.1" 200 None 2026-04-03 01:36:56,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:36:56,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:36:56,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:57802 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:36:56,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc76b58cd9cbcdc945a5b5 HTTP/1.1" 200 None 2026-04-03 01:36:56,510 - lava_callback - ERROR - Error processing callback for node 69cc76b58cd9cbcdc945a5b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50198 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:38:09,589 - lava_callback - INFO - Received callback for node 69cf198a9e2394c38fe01ebc INFO: 172.21.0.1:50208 - "POST /node/69cf198a9e2394c38fe01ebc HTTP/1.0" 202 Accepted 2026-04-03 01:38:09,596 - lava_callback - INFO - Starting async job submit for node 69cf198a9e2394c38fe01ebc 2026-04-03 01:38:09,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:38:09,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf198a9e2394c38fe01ebc HTTP/1.1" 200 None 2026-04-03 01:38:09,634 - lava_callback - DEBUG - Processing node 69cf198a9e2394c38fe01ebc: baseline-arm64 2026-04-03 01:38:10,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:38:10,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:38:10,350 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf198a9e2394c38fe01ebc/log.txt.gz 2026-04-03 01:38:10,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:38:11,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:38:11,015 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf198a9e2394c38fe01ebc/lava-logs.yaml 2026-04-03 01:38:11,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:38:11,344 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:38:11,346 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf198a9e2394c38fe01ebc/lava_callback.json.gz 2026-04-03 01:38:11,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:38:11,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf198a9e2394c38fe01ebc HTTP/1.1" 200 None 2026-04-03 01:38:11,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:38:11,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:38:11,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:38:11,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf198a9e2394c38fe01ebc HTTP/1.1" 200 None 2026-04-03 01:38:11,511 - lava_callback - INFO - Completed processing callback for node 69cf198a9e2394c38fe01ebc INFO: 172.21.0.1:38510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37010 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:38:36,513 - lava_callback - INFO - Received callback for node 69cc77118cd9cbcdc945a609 INFO: 172.21.0.1:37020 - "POST /node/69cc77118cd9cbcdc945a609 HTTP/1.0" 202 Accepted 2026-04-03 01:38:36,519 - lava_callback - INFO - Starting async job submit for node 69cc77118cd9cbcdc945a609 2026-04-03 01:38:36,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:38:36,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc77118cd9cbcdc945a609 HTTP/1.1" 200 None 2026-04-03 01:38:36,534 - lava_callback - DEBUG - Processing node 69cc77118cd9cbcdc945a609: baseline-arm64-baylibre 2026-04-03 01:38:36,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc77118cd9cbcdc945a609 2026-04-03 01:38:36,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:38:36,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc77118cd9cbcdc945a609 HTTP/1.1" 200 None 2026-04-03 01:38:36,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:38:36,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:38:36,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:38:36,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc77118cd9cbcdc945a609 HTTP/1.1" 200 None 2026-04-03 01:38:36,586 - lava_callback - ERROR - Error processing callback for node 69cc77118cd9cbcdc945a609: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 01:38:41,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:38:41,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57662 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41100 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:40:13,996 - lava_callback - INFO - Received callback for node 69cc77728cd9cbcdc945a671 INFO: 172.21.0.1:52854 - "POST /node/69cc77728cd9cbcdc945a671 HTTP/1.0" 202 Accepted 2026-04-03 01:40:14,002 - lava_callback - INFO - Starting async job submit for node 69cc77728cd9cbcdc945a671 2026-04-03 01:40:14,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:40:14,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc77728cd9cbcdc945a671 HTTP/1.1" 200 None 2026-04-03 01:40:14,018 - lava_callback - DEBUG - Processing node 69cc77728cd9cbcdc945a671: baseline-arm64-clabbe 2026-04-03 01:40:14,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc77728cd9cbcdc945a671 2026-04-03 01:40:14,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:40:14,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc77728cd9cbcdc945a671 HTTP/1.1" 200 None 2026-04-03 01:40:14,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:40:14,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:40:14,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:40:14,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc77728cd9cbcdc945a671 HTTP/1.1" 200 None 2026-04-03 01:40:14,069 - lava_callback - ERROR - Error processing callback for node 69cc77728cd9cbcdc945a671: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52860 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39788 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:41:54,051 - lava_callback - INFO - Received callback for node 69cc77cc8cd9cbcdc945a6c8 INFO: 172.21.0.1:45726 - "POST /node/69cc77cc8cd9cbcdc945a6c8 HTTP/1.0" 202 Accepted 2026-04-03 01:41:54,056 - lava_callback - INFO - Starting async job submit for node 69cc77cc8cd9cbcdc945a6c8 2026-04-03 01:41:54,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:41:54,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc77cc8cd9cbcdc945a6c8 HTTP/1.1" 200 None 2026-04-03 01:41:54,078 - lava_callback - DEBUG - Processing node 69cc77cc8cd9cbcdc945a6c8: baseline-arm64-clabbe 2026-04-03 01:41:54,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc77cc8cd9cbcdc945a6c8 2026-04-03 01:41:54,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:41:54,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc77cc8cd9cbcdc945a6c8 HTTP/1.1" 200 None 2026-04-03 01:41:54,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:41:54,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:41:54,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:41:54,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc77cc8cd9cbcdc945a6c8 HTTP/1.1" 200 None 2026-04-03 01:41:54,138 - lava_callback - ERROR - Error processing callback for node 69cc77cc8cd9cbcdc945a6c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57448 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:43:14,153 - lava_callback - INFO - Received callback for node 69cc78268cd9cbcdc945a721 INFO: 172.21.0.1:52156 - "POST /node/69cc78268cd9cbcdc945a721 HTTP/1.0" 202 Accepted 2026-04-03 01:43:14,158 - lava_callback - INFO - Starting async job submit for node 69cc78268cd9cbcdc945a721 2026-04-03 01:43:14,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:43:14,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc78268cd9cbcdc945a721 HTTP/1.1" 200 None 2026-04-03 01:43:14,174 - lava_callback - DEBUG - Processing node 69cc78268cd9cbcdc945a721: baseline-arm64-clabbe 2026-04-03 01:43:14,174 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc78268cd9cbcdc945a721 2026-04-03 01:43:14,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:43:14,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc78268cd9cbcdc945a721 HTTP/1.1" 200 None 2026-04-03 01:43:14,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:43:14,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:43:14,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:43:14,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc78268cd9cbcdc945a721 HTTP/1.1" 200 None 2026-04-03 01:43:14,230 - lava_callback - ERROR - Error processing callback for node 69cc78268cd9cbcdc945a721: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52168 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:43:25,890 - lava_callback - INFO - Received callback for node 69cf0a569e2394c38fe01341 INFO: 172.21.0.1:35826 - "POST /node/69cf0a569e2394c38fe01341 HTTP/1.0" 202 Accepted 2026-04-03 01:43:25,900 - lava_callback - INFO - Starting async job submit for node 69cf0a569e2394c38fe01341 2026-04-03 01:43:25,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:43:25,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0a569e2394c38fe01341 HTTP/1.1" 200 None 2026-04-03 01:43:25,973 - lava_callback - DEBUG - Processing node 69cf0a569e2394c38fe01341: ltp-smoke INFO: 172.21.0.1:35828 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:43:26,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:43:26,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:43:26,826 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf0a569e2394c38fe01341/log.txt.gz 2026-04-03 01:43:27,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:43:27,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:43:27,426 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf0a569e2394c38fe01341/lava-logs.yaml 2026-04-03 01:43:27,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:43:27,627 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:43:27,629 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf0a569e2394c38fe01341/lava_callback.json.gz 2026-04-03 01:43:27,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:43:27,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0a569e2394c38fe01341 HTTP/1.1" 200 None 2026-04-03 01:43:27,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:43:27,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf07039e2394c38fe00f45 HTTP/1.1" 200 None 2026-04-03 01:43:27,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:43:27,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0a569e2394c38fe01341 HTTP/1.1" 200 None 2026-04-03 01:43:27,926 - lava_callback - INFO - Completed processing callback for node 69cf0a569e2394c38fe01341 INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:43:41,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:43:41,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46550 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:44:54,225 - lava_callback - INFO - Received callback for node 69cc78808cd9cbcdc945a778 INFO: 172.21.0.1:41100 - "POST /node/69cc78808cd9cbcdc945a778 HTTP/1.0" 202 Accepted 2026-04-03 01:44:54,231 - lava_callback - INFO - Starting async job submit for node 69cc78808cd9cbcdc945a778 2026-04-03 01:44:54,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:44:54,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc78808cd9cbcdc945a778 HTTP/1.1" 200 None 2026-04-03 01:44:54,249 - lava_callback - DEBUG - Processing node 69cc78808cd9cbcdc945a778: baseline-arm64-clabbe 2026-04-03 01:44:54,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc78808cd9cbcdc945a778 2026-04-03 01:44:54,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:44:54,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc78808cd9cbcdc945a778 HTTP/1.1" 200 None 2026-04-03 01:44:54,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:44:54,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:44:54,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:44:54,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc78808cd9cbcdc945a778 HTTP/1.1" 200 None 2026-04-03 01:44:54,302 - lava_callback - ERROR - Error processing callback for node 69cc78808cd9cbcdc945a778: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55068 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:46:06,563 - lava_callback - INFO - Received callback for node 69cf0fd39e2394c38fe01827 INFO: 172.21.0.1:55078 - "POST /node/69cf0fd39e2394c38fe01827 HTTP/1.0" 202 Accepted 2026-04-03 01:46:06,573 - lava_callback - INFO - Starting async job submit for node 69cf0fd39e2394c38fe01827 2026-04-03 01:46:06,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:06,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fd39e2394c38fe01827 HTTP/1.1" 200 None 2026-04-03 01:46:06,629 - lava_callback - DEBUG - Processing node 69cf0fd39e2394c38fe01827: baseline-arm64-broonie 2026-04-03 01:46:06,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:46:07,120 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:46:07,123 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd39e2394c38fe01827/log.txt.gz 2026-04-03 01:46:07,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:46:07,562 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:46:07,564 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd39e2394c38fe01827/lava-logs.yaml 2026-04-03 01:46:07,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:46:07,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:46:07,893 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd39e2394c38fe01827/lava_callback.json.gz 2026-04-03 01:46:07,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:07,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fd39e2394c38fe01827 HTTP/1.1" 200 None 2026-04-03 01:46:07,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:07,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:46:07,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:07,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fd39e2394c38fe01827 HTTP/1.1" 200 None 2026-04-03 01:46:07,996 - lava_callback - INFO - Completed processing callback for node 69cf0fd39e2394c38fe01827 2026-04-03 01:46:11,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:11,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-03 01:46:14,294 - lava_callback - INFO - Received callback for node 69cc78d98cd9cbcdc945a7d6 INFO: 172.21.0.1:58232 - "POST /node/69cc78d98cd9cbcdc945a7d6 HTTP/1.0" 202 Accepted 2026-04-03 01:46:14,301 - lava_callback - INFO - Starting async job submit for node 69cc78d98cd9cbcdc945a7d6 2026-04-03 01:46:14,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:14,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc78d98cd9cbcdc945a7d6 HTTP/1.1" 200 None 2026-04-03 01:46:14,320 - lava_callback - DEBUG - Processing node 69cc78d98cd9cbcdc945a7d6: baseline-arm64-clabbe 2026-04-03 01:46:14,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc78d98cd9cbcdc945a7d6 2026-04-03 01:46:14,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:14,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc78d98cd9cbcdc945a7d6 HTTP/1.1" 200 None 2026-04-03 01:46:14,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:14,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:46:14,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:14,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc78d98cd9cbcdc945a7d6 HTTP/1.1" 200 None 2026-04-03 01:46:14,384 - lava_callback - ERROR - Error processing callback for node 69cc78d98cd9cbcdc945a7d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60800 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:46:53,270 - lava_callback - INFO - Received callback for node 69cf1a469e2394c38fe01f78 INFO: 172.21.0.1:43270 - "POST /node/69cf1a469e2394c38fe01f78 HTTP/1.0" 202 Accepted 2026-04-03 01:46:53,278 - lava_callback - INFO - Starting async job submit for node 69cf1a469e2394c38fe01f78 2026-04-03 01:46:53,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:53,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1a469e2394c38fe01f78 HTTP/1.1" 200 None 2026-04-03 01:46:53,349 - lava_callback - DEBUG - Processing node 69cf1a469e2394c38fe01f78: baseline-arm64 2026-04-03 01:46:53,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:46:53,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:46:53,911 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf1a469e2394c38fe01f78/log.txt.gz 2026-04-03 01:46:54,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:46:54,606 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:46:54,608 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf1a469e2394c38fe01f78/lava-logs.yaml 2026-04-03 01:46:54,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:46:54,776 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:46:54,778 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf1a469e2394c38fe01f78/lava_callback.json.gz DEBUG: Setup failed for 69cf1a469e2394c38fe01f78. Transitting job node result: incomplete -> fail 2026-04-03 01:46:54,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:54,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1a469e2394c38fe01f78 HTTP/1.1" 200 None 2026-04-03 01:46:54,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:54,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:46:54,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:46:54,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf1a469e2394c38fe01f78 HTTP/1.1" 200 None 2026-04-03 01:46:54,996 - lava_callback - INFO - Completed processing callback for node 69cf1a469e2394c38fe01f78 INFO: 172.21.0.1:43286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55530 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:47:11,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:47:11,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33110 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:48:34,364 - lava_callback - INFO - Received callback for node 69cc79618cd9cbcdc945a81e INFO: 172.21.0.1:44498 - "POST /node/69cc79618cd9cbcdc945a81e HTTP/1.0" 202 Accepted 2026-04-03 01:48:34,370 - lava_callback - INFO - Starting async job submit for node 69cc79618cd9cbcdc945a81e 2026-04-03 01:48:34,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:48:34,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc79618cd9cbcdc945a81e HTTP/1.1" 200 None 2026-04-03 01:48:34,386 - lava_callback - DEBUG - Processing node 69cc79618cd9cbcdc945a81e: baseline-arm64-clabbe 2026-04-03 01:48:34,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc79618cd9cbcdc945a81e 2026-04-03 01:48:34,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:48:34,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc79618cd9cbcdc945a81e HTTP/1.1" 200 None 2026-04-03 01:48:34,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:48:34,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 01:48:34,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:48:34,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc79618cd9cbcdc945a81e HTTP/1.1" 200 None 2026-04-03 01:48:34,445 - lava_callback - ERROR - Error processing callback for node 69cc79618cd9cbcdc945a81e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57582 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46714 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:50:46,067 - lava_callback - INFO - Received callback for node 69cf0ecf9e2394c38fe0161e INFO: 172.21.0.1:46716 - "POST /node/69cf0ecf9e2394c38fe0161e HTTP/1.0" 202 Accepted 2026-04-03 01:50:46,074 - lava_callback - INFO - Starting async job submit for node 69cf0ecf9e2394c38fe0161e 2026-04-03 01:50:46,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:50:46,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ecf9e2394c38fe0161e HTTP/1.1" 200 None 2026-04-03 01:50:46,130 - lava_callback - DEBUG - Processing node 69cf0ecf9e2394c38fe0161e: ltp-cap_bounds 2026-04-03 01:50:46,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46724 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:50:46,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:50:46,656 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cf0ecf9e2394c38fe0161e/log.txt.gz 2026-04-03 01:50:46,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:50:47,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:50:47,123 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cf0ecf9e2394c38fe0161e/lava-logs.yaml 2026-04-03 01:50:47,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:50:47,524 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:50:47,525 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69cf0ecf9e2394c38fe0161e/lava_callback.json.gz 2026-04-03 01:50:47,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:50:47,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ecf9e2394c38fe0161e HTTP/1.1" 200 None 2026-04-03 01:50:47,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:50:47,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:50:47,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:50:47,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ecf9e2394c38fe0161e HTTP/1.1" 200 None 2026-04-03 01:50:47,618 - lava_callback - INFO - Completed processing callback for node 69cf0ecf9e2394c38fe0161e INFO: 172.21.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39652 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:51:11,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:51:11,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49986 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:52:06,721 - lava_callback - INFO - Received callback for node 69cf0ed09e2394c38fe01621 INFO: 172.21.0.1:49996 - "POST /node/69cf0ed09e2394c38fe01621 HTTP/1.0" 202 Accepted 2026-04-03 01:52:06,727 - lava_callback - INFO - Starting async job submit for node 69cf0ed09e2394c38fe01621 2026-04-03 01:52:06,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:52:06,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ed09e2394c38fe01621 HTTP/1.1" 200 None 2026-04-03 01:52:06,788 - lava_callback - DEBUG - Processing node 69cf0ed09e2394c38fe01621: ltp-containers 2026-04-03 01:52:07,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:52:07,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:52:07,554 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69cf0ed09e2394c38fe01621/log.txt.gz 2026-04-03 01:52:07,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:52:08,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:52:08,314 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69cf0ed09e2394c38fe01621/lava-logs.yaml 2026-04-03 01:52:08,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:52:08,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:52:08,487 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69cf0ed09e2394c38fe01621/lava_callback.json.gz 2026-04-03 01:52:08,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:52:08,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ed09e2394c38fe01621 HTTP/1.1" 200 None 2026-04-03 01:52:08,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:52:08,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:52:08,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:52:08,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ed09e2394c38fe01621 HTTP/1.1" 200 None 2026-04-03 01:52:08,604 - lava_callback - INFO - Completed processing callback for node 69cf0ed09e2394c38fe01621 2026-04-03 01:52:11,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:52:11,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56064 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:54:38,087 - lava_callback - INFO - Received callback for node 69cf1b669e2394c38fe0207d INFO: 172.21.0.1:56066 - "POST /node/69cf1b669e2394c38fe0207d HTTP/1.0" 202 Accepted 2026-04-03 01:54:38,093 - lava_callback - INFO - Starting async job submit for node 69cf1b669e2394c38fe0207d 2026-04-03 01:54:38,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:54:38,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1b669e2394c38fe0207d HTTP/1.1" 200 None 2026-04-03 01:54:38,137 - lava_callback - DEBUG - Processing node 69cf1b669e2394c38fe0207d: baseline-arm64 2026-04-03 01:54:38,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:54:38,839 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:54:38,841 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf1b669e2394c38fe0207d/log.txt.gz 2026-04-03 01:54:39,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:54:39,334 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:54:39,336 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf1b669e2394c38fe0207d/lava-logs.yaml 2026-04-03 01:54:39,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:54:39,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:54:39,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf1b669e2394c38fe0207d/lava_callback.json.gz 2026-04-03 01:54:39,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:54:39,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1b669e2394c38fe0207d HTTP/1.1" 200 None 2026-04-03 01:54:39,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:54:39,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:54:39,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:54:39,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf1b669e2394c38fe0207d HTTP/1.1" 200 None 2026-04-03 01:54:39,651 - lava_callback - INFO - Completed processing callback for node 69cf1b669e2394c38fe0207d 2026-04-03 01:54:41,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:54:41,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44458 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:54:47,097 - lava_callback - INFO - Received callback for node 69cf0ed29e2394c38fe0163e INFO: 172.21.0.1:44460 - "POST /node/69cf0ed29e2394c38fe0163e HTTP/1.0" 202 Accepted 2026-04-03 01:54:47,106 - lava_callback - INFO - Starting async job submit for node 69cf0ed29e2394c38fe0163e 2026-04-03 01:54:47,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:54:47,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ed29e2394c38fe0163e HTTP/1.1" 200 None 2026-04-03 01:54:47,155 - lava_callback - DEBUG - Processing node 69cf0ed29e2394c38fe0163e: ltp-controllers 2026-04-03 01:54:47,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:54:47,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:54:47,861 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69cf0ed29e2394c38fe0163e/log.txt.gz 2026-04-03 01:54:48,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:54:48,422 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:54:48,424 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69cf0ed29e2394c38fe0163e/lava-logs.yaml 2026-04-03 01:54:48,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:54:48,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:54:48,610 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69cf0ed29e2394c38fe0163e/lava_callback.json.gz 2026-04-03 01:54:48,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:54:48,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ed29e2394c38fe0163e HTTP/1.1" 200 None 2026-04-03 01:54:48,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:54:48,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:54:48,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:54:48,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ed29e2394c38fe0163e HTTP/1.1" 200 None 2026-04-03 01:54:48,698 - lava_callback - INFO - Completed processing callback for node 69cf0ed29e2394c38fe0163e INFO: 172.21.0.1:45672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45860 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:55:11,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:55:11,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51596 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37820 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:56:20,460 - lava_callback - INFO - Received callback for node 69cf1bb49e2394c38fe020b5 INFO: 172.21.0.1:36332 - "POST /node/69cf1bb49e2394c38fe020b5 HTTP/1.0" 202 Accepted 2026-04-03 01:56:20,468 - lava_callback - INFO - Starting async job submit for node 69cf1bb49e2394c38fe020b5 2026-04-03 01:56:20,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:20,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1bb49e2394c38fe020b5 HTTP/1.1" 200 None 2026-04-03 01:56:20,514 - lava_callback - DEBUG - Processing node 69cf1bb49e2394c38fe020b5: baseline-nfs-arm64 2026-04-03 01:56:21,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:56:21,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:56:21,338 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf1bb49e2394c38fe020b5/log.txt.gz 2026-04-03 01:56:21,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:56:22,162 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:56:22,164 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf1bb49e2394c38fe020b5/lava-logs.yaml 2026-04-03 01:56:22,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:56:22,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:56:22,470 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf1bb49e2394c38fe020b5/lava_callback.json.gz 2026-04-03 01:56:22,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:22,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1bb49e2394c38fe020b5 HTTP/1.1" 200 None 2026-04-03 01:56:22,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:22,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 01:56:22,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:22,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf1bb49e2394c38fe020b5 HTTP/1.1" 200 None 2026-04-03 01:56:22,647 - lava_callback - INFO - Completed processing callback for node 69cf1bb49e2394c38fe020b5 INFO: 172.21.0.1:36338 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:56:26,912 - lava_callback - INFO - Received callback for node 69cf0ed89e2394c38fe0164b INFO: 172.21.0.1:36350 - "POST /node/69cf0ed89e2394c38fe0164b HTTP/1.0" 202 Accepted 2026-04-03 01:56:26,918 - lava_callback - INFO - Starting async job submit for node 69cf0ed89e2394c38fe0164b 2026-04-03 01:56:26,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:26,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ed89e2394c38fe0164b HTTP/1.1" 200 None 2026-04-03 01:56:26,961 - lava_callback - DEBUG - Processing node 69cf0ed89e2394c38fe0164b: ltp-fs_bind 2026-04-03 01:56:27,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:56:27,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:56:27,498 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cf0ed89e2394c38fe0164b/log.txt.gz 2026-04-03 01:56:27,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:56:27,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:56:27,984 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cf0ed89e2394c38fe0164b/lava-logs.yaml 2026-04-03 01:56:27,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:56:28,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:56:28,174 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69cf0ed89e2394c38fe0164b/lava_callback.json.gz 2026-04-03 01:56:28,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:28,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ed89e2394c38fe0164b HTTP/1.1" 200 None 2026-04-03 01:56:28,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:28,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:56:28,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:28,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ed89e2394c38fe0164b HTTP/1.1" 200 None 2026-04-03 01:56:28,266 - lava_callback - INFO - Completed processing callback for node 69cf0ed89e2394c38fe0164b INFO: 172.21.0.1:40322 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:56:41,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:41,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43600 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:56:48,627 - lava_callback - INFO - Received callback for node 69cf0ed59e2394c38fe01643 INFO: 172.21.0.1:43608 - "POST /node/69cf0ed59e2394c38fe01643 HTTP/1.0" 202 Accepted 2026-04-03 01:56:48,634 - lava_callback - INFO - Starting async job submit for node 69cf0ed59e2394c38fe01643 2026-04-03 01:56:48,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:48,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ed59e2394c38fe01643 HTTP/1.1" 200 None 2026-04-03 01:56:48,673 - lava_callback - DEBUG - Processing node 69cf0ed59e2394c38fe01643: ltp-cve 2026-04-03 01:56:49,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:56:49,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:56:49,222 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69cf0ed59e2394c38fe01643/log.txt.gz 2026-04-03 01:56:49,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:56:49,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:56:49,662 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69cf0ed59e2394c38fe01643/lava-logs.yaml 2026-04-03 01:56:49,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:56:50,071 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:56:50,073 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69cf0ed59e2394c38fe01643/lava_callback.json.gz 2026-04-03 01:56:50,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:50,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ed59e2394c38fe01643 HTTP/1.1" 200 None 2026-04-03 01:56:50,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:50,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:56:50,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:56:50,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ed59e2394c38fe01643 HTTP/1.1" 200 None 2026-04-03 01:56:50,189 - lava_callback - INFO - Completed processing callback for node 69cf0ed59e2394c38fe01643 INFO: 172.21.0.1:51024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39532 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:57:11,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:57:11,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35598 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:40598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35686 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:58:47,363 - lava_callback - INFO - Received callback for node 69cf0edb9e2394c38fe01652 INFO: 172.21.0.1:35698 - "POST /node/69cf0edb9e2394c38fe01652 HTTP/1.0" 202 Accepted 2026-04-03 01:58:47,370 - lava_callback - INFO - Starting async job submit for node 69cf0edb9e2394c38fe01652 2026-04-03 01:58:47,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:58:47,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0edb9e2394c38fe01652 HTTP/1.1" 200 None 2026-04-03 01:58:47,430 - lava_callback - DEBUG - Processing node 69cf0edb9e2394c38fe01652: ltp-input 2026-04-03 01:58:47,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:58:48,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:58:48,041 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69cf0edb9e2394c38fe01652/log.txt.gz 2026-04-03 01:58:48,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:58:48,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:58:48,559 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69cf0edb9e2394c38fe01652/lava-logs.yaml 2026-04-03 01:58:48,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 01:58:48,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 01:58:48,817 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69cf0edb9e2394c38fe01652/lava_callback.json.gz 2026-04-03 01:58:48,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:58:48,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0edb9e2394c38fe01652 HTTP/1.1" 200 None 2026-04-03 01:58:48,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:58:48,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 01:58:48,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:58:48,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0edb9e2394c38fe01652 HTTP/1.1" 200 None 2026-04-03 01:58:48,905 - lava_callback - INFO - Completed processing callback for node 69cf0edb9e2394c38fe01652 INFO: 172.21.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38242 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:59:11,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:59:11,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35508 - "GET / HTTP/1.0" 200 OK 2026-04-03 01:59:57,730 - lava_callback - INFO - Received callback for node 69cc7c198cd9cbcdc945aa62 INFO: 172.21.0.1:35520 - "POST /node/69cc7c198cd9cbcdc945aa62 HTTP/1.0" 202 Accepted 2026-04-03 01:59:57,736 - lava_callback - INFO - Starting async job submit for node 69cc7c198cd9cbcdc945aa62 2026-04-03 01:59:57,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:59:57,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7c198cd9cbcdc945aa62 HTTP/1.1" 200 None 2026-04-03 01:59:57,752 - lava_callback - DEBUG - Processing node 69cc7c198cd9cbcdc945aa62: baseline-arm64-baylibre 2026-04-03 01:59:57,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc7c198cd9cbcdc945aa62 2026-04-03 01:59:57,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:59:57,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7c198cd9cbcdc945aa62 HTTP/1.1" 200 None 2026-04-03 01:59:57,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:59:57,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 01:59:57,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 01:59:57,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7c198cd9cbcdc945aa62 HTTP/1.1" 200 None 2026-04-03 01:59:57,807 - lava_callback - ERROR - Error processing callback for node 69cc7c198cd9cbcdc945aa62: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39948 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:00:29,535 - lava_callback - INFO - Received callback for node 69cf0ede9e2394c38fe01659 INFO: 172.21.0.1:39960 - "POST /node/69cf0ede9e2394c38fe01659 HTTP/1.0" 202 Accepted 2026-04-03 02:00:29,541 - lava_callback - INFO - Starting async job submit for node 69cf0ede9e2394c38fe01659 2026-04-03 02:00:29,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:00:29,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ede9e2394c38fe01659 HTTP/1.1" 200 None 2026-04-03 02:00:29,584 - lava_callback - DEBUG - Processing node 69cf0ede9e2394c38fe01659: ltp-ipc 2026-04-03 02:00:29,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:00:30,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:00:30,129 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69cf0ede9e2394c38fe01659/log.txt.gz 2026-04-03 02:00:30,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:00:30,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:00:30,966 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69cf0ede9e2394c38fe01659/lava-logs.yaml 2026-04-03 02:00:30,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:00:31,219 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:00:31,222 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69cf0ede9e2394c38fe01659/lava_callback.json.gz 2026-04-03 02:00:31,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:00:31,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ede9e2394c38fe01659 HTTP/1.1" 200 None 2026-04-03 02:00:31,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:00:31,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:00:31,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:00:31,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ede9e2394c38fe01659 HTTP/1.1" 200 None 2026-04-03 02:00:31,309 - lava_callback - INFO - Completed processing callback for node 69cf0ede9e2394c38fe01659 INFO: 172.21.0.1:34908 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:00:41,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:00:41,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60654 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:00:48,106 - lava_callback - INFO - Received callback for node 69cf0ee59e2394c38fe0165f INFO: 172.21.0.1:60662 - "POST /node/69cf0ee59e2394c38fe0165f HTTP/1.0" 202 Accepted 2026-04-03 02:00:48,112 - lava_callback - INFO - Starting async job submit for node 69cf0ee59e2394c38fe0165f 2026-04-03 02:00:48,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:00:48,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ee59e2394c38fe0165f HTTP/1.1" 200 None 2026-04-03 02:00:48,152 - lava_callback - DEBUG - Processing node 69cf0ee59e2394c38fe0165f: ltp-pty 2026-04-03 02:00:48,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:00:48,642 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:00:48,643 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69cf0ee59e2394c38fe0165f/log.txt.gz 2026-04-03 02:00:48,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:00:49,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:00:49,085 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69cf0ee59e2394c38fe0165f/lava-logs.yaml 2026-04-03 02:00:49,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:00:49,258 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:00:49,260 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69cf0ee59e2394c38fe0165f/lava_callback.json.gz 2026-04-03 02:00:49,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:00:49,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ee59e2394c38fe0165f HTTP/1.1" 200 None 2026-04-03 02:00:49,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:00:49,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:00:49,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:00:49,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ee59e2394c38fe0165f HTTP/1.1" 200 None 2026-04-03 02:00:49,344 - lava_callback - INFO - Completed processing callback for node 69cf0ee59e2394c38fe0165f INFO: 172.21.0.1:45494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45506 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:01:11,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:01:11,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54684 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:01:20,560 - lava_callback - INFO - Received callback for node 69cf1b3e9e2394c38fe0204d INFO: 172.21.0.1:47666 - "POST /node/69cf1b3e9e2394c38fe0204d HTTP/1.0" 202 Accepted 2026-04-03 02:01:20,567 - lava_callback - INFO - Starting async job submit for node 69cf1b3e9e2394c38fe0204d 2026-04-03 02:01:20,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:01:20,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1b3e9e2394c38fe0204d HTTP/1.1" 200 None 2026-04-03 02:01:20,635 - lava_callback - DEBUG - Processing node 69cf1b3e9e2394c38fe0204d: baseline-arm 2026-04-03 02:01:21,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:01:21,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:01:21,341 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf1b3e9e2394c38fe0204d/log.txt.gz 2026-04-03 02:01:21,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:01:22,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:01:22,131 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf1b3e9e2394c38fe0204d/lava-logs.yaml 2026-04-03 02:01:22,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:01:22,297 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:01:22,299 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf1b3e9e2394c38fe0204d/lava_callback.json.gz DEBUG: Setup failed for 69cf1b3e9e2394c38fe0204d. Transitting job node result: incomplete -> fail 2026-04-03 02:01:22,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:01:22,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1b3e9e2394c38fe0204d HTTP/1.1" 200 None 2026-04-03 02:01:22,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:01:22,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:01:22,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:01:22,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf1b3e9e2394c38fe0204d HTTP/1.1" 200 None 2026-04-03 02:01:22,587 - lava_callback - INFO - Completed processing callback for node 69cf1b3e9e2394c38fe0204d INFO: 172.21.0.1:47670 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:01:27,393 - lava_callback - INFO - Received callback for node 69cf0eec9e2394c38fe01666 INFO: 172.21.0.1:47672 - "POST /node/69cf0eec9e2394c38fe01666 HTTP/1.0" 202 Accepted 2026-04-03 02:01:27,400 - lava_callback - INFO - Starting async job submit for node 69cf0eec9e2394c38fe01666 2026-04-03 02:01:27,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:01:27,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0eec9e2394c38fe01666 HTTP/1.1" 200 None 2026-04-03 02:01:27,453 - lava_callback - DEBUG - Processing node 69cf0eec9e2394c38fe01666: ltp-sched 2026-04-03 02:01:27,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:01:27,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:01:27,961 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69cf0eec9e2394c38fe01666/log.txt.gz 2026-04-03 02:01:28,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:01:28,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:01:28,407 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69cf0eec9e2394c38fe01666/lava-logs.yaml 2026-04-03 02:01:28,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:01:28,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:01:28,567 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69cf0eec9e2394c38fe01666/lava_callback.json.gz 2026-04-03 02:01:28,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:01:28,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0eec9e2394c38fe01666 HTTP/1.1" 200 None 2026-04-03 02:01:28,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:01:28,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:01:28,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:01:28,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0eec9e2394c38fe01666 HTTP/1.1" 200 None 2026-04-03 02:01:28,681 - lava_callback - INFO - Completed processing callback for node 69cf0eec9e2394c38fe01666 INFO: 172.21.0.1:38180 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:01:41,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:01:41,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52994 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:02:57,395 - lava_callback - INFO - Received callback for node 69cf1fc49e2394c38fe02434 INFO: 172.21.0.1:53004 - "POST /node/69cf1fc49e2394c38fe02434 HTTP/1.0" 202 Accepted 2026-04-03 02:02:57,400 - lava_callback - INFO - Starting async job submit for node 69cf1fc49e2394c38fe02434 2026-04-03 02:02:57,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:02:57,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1fc49e2394c38fe02434 HTTP/1.1" 200 None 2026-04-03 02:02:57,418 - lava_callback - DEBUG - Processing node 69cf1fc49e2394c38fe02434: baseline-arm 2026-04-03 02:02:57,423 - lava_callback - WARNING - Failed to upload log 2026-04-03 02:02:57,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:02:57,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:02:57,615 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf1fc49e2394c38fe02434/lava-logs.yaml 2026-04-03 02:02:57,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:02:57,941 - lava_callback - INFO - Received callback for node 69cc7ccb8cd9cbcdc945aaf1 INFO: 172.21.0.1:53012 - "POST /node/69cc7ccb8cd9cbcdc945aaf1 HTTP/1.0" 202 Accepted 2026-04-03 02:02:57,947 - lava_callback - INFO - Starting async job submit for node 69cc7ccb8cd9cbcdc945aaf1 2026-04-03 02:02:57,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:02:57,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7ccb8cd9cbcdc945aaf1 HTTP/1.1" 200 None 2026-04-03 02:02:57,970 - lava_callback - DEBUG - Processing node 69cc7ccb8cd9cbcdc945aaf1: baseline-arm64-baylibre 2026-04-03 02:02:57,970 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc7ccb8cd9cbcdc945aaf1 2026-04-03 02:02:57,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:02:57,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7ccb8cd9cbcdc945aaf1 HTTP/1.1" 200 None 2026-04-03 02:02:57,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:02:57,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:02:58,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:02:58,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7ccb8cd9cbcdc945aaf1 HTTP/1.1" 200 None 2026-04-03 02:02:58,026 - lava_callback - ERROR - Error processing callback for node 69cc7ccb8cd9cbcdc945aaf1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 02:02:58,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:02:58,081 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf1fc49e2394c38fe02434/lava_callback.json.gz 2026-04-03 02:02:58,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:02:58,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1fc49e2394c38fe02434 HTTP/1.1" 200 None 2026-04-03 02:02:58,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:02:58,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:02:58,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:02:58,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf1fc49e2394c38fe02434 HTTP/1.1" 200 None 2026-04-03 02:02:58,198 - lava_callback - INFO - Completed processing callback for node 69cf1fc49e2394c38fe02434 INFO: 172.21.0.1:53914 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:03:07,138 - lava_callback - INFO - Received callback for node 69cf0ef09e2394c38fe0168a INFO: 172.21.0.1:53930 - "POST /node/69cf0ef09e2394c38fe0168a HTTP/1.0" 202 Accepted 2026-04-03 02:03:07,145 - lava_callback - INFO - Starting async job submit for node 69cf0ef09e2394c38fe0168a 2026-04-03 02:03:07,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:03:07,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ef09e2394c38fe0168a HTTP/1.1" 200 None 2026-04-03 02:03:07,184 - lava_callback - DEBUG - Processing node 69cf0ef09e2394c38fe0168a: ltp-smoke 2026-04-03 02:03:07,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:03:07,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:03:07,894 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf0ef09e2394c38fe0168a/log.txt.gz 2026-04-03 02:03:08,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:03:08,396 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:03:08,398 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf0ef09e2394c38fe0168a/lava-logs.yaml 2026-04-03 02:03:08,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:03:08,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:03:08,613 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf0ef09e2394c38fe0168a/lava_callback.json.gz 2026-04-03 02:03:08,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:03:08,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0ef09e2394c38fe0168a HTTP/1.1" 200 None 2026-04-03 02:03:08,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:03:08,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:03:08,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:03:08,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0ef09e2394c38fe0168a HTTP/1.1" 200 None 2026-04-03 02:03:08,709 - lava_callback - INFO - Completed processing callback for node 69cf0ef09e2394c38fe0168a 2026-04-03 02:03:11,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:03:11,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43256 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:03:47,019 - lava_callback - INFO - Received callback for node 69cf0fa49e2394c38fe0177f INFO: 172.21.0.1:43270 - "POST /node/69cf0fa49e2394c38fe0177f HTTP/1.0" 202 Accepted 2026-04-03 02:03:47,025 - lava_callback - INFO - Starting async job submit for node 69cf0fa49e2394c38fe0177f 2026-04-03 02:03:47,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:03:47,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa49e2394c38fe0177f HTTP/1.1" 200 None 2026-04-03 02:03:47,065 - lava_callback - DEBUG - Processing node 69cf0fa49e2394c38fe0177f: baseline-arm-broonie 2026-04-03 02:03:47,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:03:47,392 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:03:47,394 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0fa49e2394c38fe0177f/log.txt.gz 2026-04-03 02:03:47,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:03:47,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:03:47,796 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0fa49e2394c38fe0177f/lava-logs.yaml 2026-04-03 02:03:47,802 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:03:47,935 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:03:47,937 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69cf0fa49e2394c38fe0177f/lava_callback.json.gz 2026-04-03 02:03:47,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:03:47,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fa49e2394c38fe0177f HTTP/1.1" 200 None 2026-04-03 02:03:47,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:03:48,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:03:48,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:03:48,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fa49e2394c38fe0177f HTTP/1.1" 200 None 2026-04-03 02:03:48,032 - lava_callback - INFO - Completed processing callback for node 69cf0fa49e2394c38fe0177f INFO: 172.21.0.1:55800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35174 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:04:11,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:11,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:34248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54392 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:04:27,261 - lava_callback - INFO - Received callback for node 69cf0efd9e2394c38fe01690 INFO: 172.21.0.1:54402 - "POST /node/69cf0efd9e2394c38fe01690 HTTP/1.0" 202 Accepted 2026-04-03 02:04:27,267 - lava_callback - INFO - Starting async job submit for node 69cf0efd9e2394c38fe01690 2026-04-03 02:04:27,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:27,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0efd9e2394c38fe01690 HTTP/1.1" 200 None 2026-04-03 02:04:27,306 - lava_callback - DEBUG - Processing node 69cf0efd9e2394c38fe01690: ltp-syscalls-ipc 2026-04-03 02:04:27,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:04:27,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:04:27,835 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cf0efd9e2394c38fe01690/log.txt.gz 2026-04-03 02:04:28,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:04:28,364 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:04:28,367 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cf0efd9e2394c38fe01690/lava-logs.yaml 2026-04-03 02:04:28,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:04:28,747 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:04:28,749 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69cf0efd9e2394c38fe01690/lava_callback.json.gz 2026-04-03 02:04:28,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:28,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0efd9e2394c38fe01690 HTTP/1.1" 200 None 2026-04-03 02:04:28,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:28,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:04:28,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:28,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0efd9e2394c38fe01690 HTTP/1.1" 200 None 2026-04-03 02:04:28,863 - lava_callback - INFO - Completed processing callback for node 69cf0efd9e2394c38fe01690 INFO: 172.21.0.1:34350 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:04:38,024 - lava_callback - INFO - Received callback for node 69cc7d288cd9cbcdc945ab38 INFO: 172.21.0.1:34360 - "POST /node/69cc7d288cd9cbcdc945ab38 HTTP/1.0" 202 Accepted 2026-04-03 02:04:38,029 - lava_callback - INFO - Starting async job submit for node 69cc7d288cd9cbcdc945ab38 2026-04-03 02:04:38,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:38,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7d288cd9cbcdc945ab38 HTTP/1.1" 200 None 2026-04-03 02:04:38,045 - lava_callback - DEBUG - Processing node 69cc7d288cd9cbcdc945ab38: baseline-arm64-baylibre 2026-04-03 02:04:38,045 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc7d288cd9cbcdc945ab38 2026-04-03 02:04:38,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:38,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7d288cd9cbcdc945ab38 HTTP/1.1" 200 None 2026-04-03 02:04:38,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:38,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:04:38,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:38,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7d288cd9cbcdc945ab38 HTTP/1.1" 200 None 2026-04-03 02:04:38,112 - lava_callback - ERROR - Error processing callback for node 69cc7d288cd9cbcdc945ab38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 02:04:41,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:41,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56374 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:04:48,696 - lava_callback - INFO - Received callback for node 69cf0f029e2394c38fe016a2 INFO: 172.21.0.1:56380 - "POST /node/69cf0f029e2394c38fe016a2 HTTP/1.0" 202 Accepted 2026-04-03 02:04:48,702 - lava_callback - INFO - Starting async job submit for node 69cf0f029e2394c38fe016a2 2026-04-03 02:04:48,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:48,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0f029e2394c38fe016a2 HTTP/1.1" 200 None 2026-04-03 02:04:48,765 - lava_callback - DEBUG - Processing node 69cf0f029e2394c38fe016a2: ltp-watchqueue 2026-04-03 02:04:49,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:04:49,253 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:04:49,255 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cf0f029e2394c38fe016a2/log.txt.gz 2026-04-03 02:04:49,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:04:49,882 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:04:49,884 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cf0f029e2394c38fe016a2/lava-logs.yaml 2026-04-03 02:04:49,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:04:50,061 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:04:50,062 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69cf0f029e2394c38fe016a2/lava_callback.json.gz 2026-04-03 02:04:50,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:50,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0f029e2394c38fe016a2 HTTP/1.1" 200 None 2026-04-03 02:04:50,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:50,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:04:50,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:04:50,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0f029e2394c38fe016a2 HTTP/1.1" 200 None 2026-04-03 02:04:50,150 - lava_callback - INFO - Completed processing callback for node 69cf0f029e2394c38fe016a2 INFO: 172.21.0.1:38784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59522 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:05:11,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:05:11,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37410 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:06:18,119 - lava_callback - INFO - Received callback for node 69cc7d848cd9cbcdc945ab8f INFO: 172.21.0.1:37412 - "POST /node/69cc7d848cd9cbcdc945ab8f HTTP/1.0" 202 Accepted 2026-04-03 02:06:18,125 - lava_callback - INFO - Starting async job submit for node 69cc7d848cd9cbcdc945ab8f 2026-04-03 02:06:18,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:06:18,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7d848cd9cbcdc945ab8f HTTP/1.1" 200 None 2026-04-03 02:06:18,142 - lava_callback - DEBUG - Processing node 69cc7d848cd9cbcdc945ab8f: baseline-arm64-baylibre 2026-04-03 02:06:18,142 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc7d848cd9cbcdc945ab8f 2026-04-03 02:06:18,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:06:18,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7d848cd9cbcdc945ab8f HTTP/1.1" 200 None 2026-04-03 02:06:18,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:06:18,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:06:18,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:06:18,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7d848cd9cbcdc945ab8f HTTP/1.1" 200 None 2026-04-03 02:06:18,212 - lava_callback - ERROR - Error processing callback for node 69cc7d848cd9cbcdc945ab8f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45410 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:07:06,287 - lava_callback - INFO - Received callback for node 69cf20be9e2394c38fe0252f INFO: 172.21.0.1:58212 - "POST /node/69cf20be9e2394c38fe0252f HTTP/1.0" 202 Accepted 2026-04-03 02:07:06,292 - lava_callback - INFO - Starting async job submit for node 69cf20be9e2394c38fe0252f 2026-04-03 02:07:06,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:07:06,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf20be9e2394c38fe0252f HTTP/1.1" 200 None 2026-04-03 02:07:06,309 - lava_callback - DEBUG - Processing node 69cf20be9e2394c38fe0252f: baseline-nfs-arm64 2026-04-03 02:07:06,311 - lava_callback - WARNING - Failed to upload log 2026-04-03 02:07:06,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:07:06,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:07:06,474 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf20be9e2394c38fe0252f/lava-logs.yaml 2026-04-03 02:07:06,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:58214 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:07:06,600 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:07:06,601 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf20be9e2394c38fe0252f/lava_callback.json.gz 2026-04-03 02:07:06,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:07:06,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf20be9e2394c38fe0252f HTTP/1.1" 200 None 2026-04-03 02:07:06,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:07:06,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:07:06,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:07:06,653 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf20be9e2394c38fe0252f HTTP/1.1" 200 None 2026-04-03 02:07:06,657 - lava_callback - INFO - Completed processing callback for node 69cf20be9e2394c38fe0252f 2026-04-03 02:07:11,862 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:07:11,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44972 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:07:27,218 - lava_callback - INFO - Received callback for node 69cf11489e2394c38fe01a95 INFO: 172.21.0.1:44984 - "POST /node/69cf11489e2394c38fe01a95 HTTP/1.0" 202 Accepted 2026-04-03 02:07:27,225 - lava_callback - INFO - Starting async job submit for node 69cf11489e2394c38fe01a95 2026-04-03 02:07:27,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:07:27,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf11489e2394c38fe01a95 HTTP/1.1" 200 None 2026-04-03 02:07:27,279 - lava_callback - DEBUG - Processing node 69cf11489e2394c38fe01a95: ltp-fsx 2026-04-03 02:07:27,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:07:27,843 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:07:27,844 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69cf11489e2394c38fe01a95/log.txt.gz 2026-04-03 02:07:28,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:07:28,315 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:07:28,317 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69cf11489e2394c38fe01a95/lava-logs.yaml 2026-04-03 02:07:28,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:07:28,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:07:28,507 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69cf11489e2394c38fe01a95/lava_callback.json.gz 2026-04-03 02:07:28,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:07:28,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf11489e2394c38fe01a95 HTTP/1.1" 200 None 2026-04-03 02:07:28,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:07:28,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:07:28,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:07:28,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf11489e2394c38fe01a95 HTTP/1.1" 200 None 2026-04-03 02:07:28,589 - lava_callback - INFO - Completed processing callback for node 69cf11489e2394c38fe01a95 INFO: 172.21.0.1:47846 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:07:41,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:07:41,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58904 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:08:07,329 - lava_callback - INFO - Received callback for node 69cf11da9e2394c38fe01af7 INFO: 172.21.0.1:58920 - "POST /node/69cf11da9e2394c38fe01af7 HTTP/1.0" 202 Accepted 2026-04-03 02:08:07,336 - lava_callback - INFO - Starting async job submit for node 69cf11da9e2394c38fe01af7 2026-04-03 02:08:07,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:08:07,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf11da9e2394c38fe01af7 HTTP/1.1" 200 None 2026-04-03 02:08:07,387 - lava_callback - DEBUG - Processing node 69cf11da9e2394c38fe01af7: ltp-timers 2026-04-03 02:08:07,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:08:08,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:08:08,098 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69cf11da9e2394c38fe01af7/log.txt.gz 2026-04-03 02:08:08,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:08:08,605 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:08:08,607 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69cf11da9e2394c38fe01af7/lava-logs.yaml 2026-04-03 02:08:08,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:08:08,818 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:08:08,820 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69cf11da9e2394c38fe01af7/lava_callback.json.gz 2026-04-03 02:08:08,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:08:08,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf11da9e2394c38fe01af7 HTTP/1.1" 200 None 2026-04-03 02:08:08,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:08:08,879 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:08:08,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:08:08,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf11da9e2394c38fe01af7 HTTP/1.1" 200 None 2026-04-03 02:08:08,963 - lava_callback - INFO - Completed processing callback for node 69cf11da9e2394c38fe01af7 2026-04-03 02:08:11,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:08:11,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54192 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51646 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:09:18,280 - lava_callback - INFO - Received callback for node 69cc7e398cd9cbcdc945ac41 INFO: 172.21.0.1:51660 - "POST /node/69cc7e398cd9cbcdc945ac41 HTTP/1.0" 202 Accepted 2026-04-03 02:09:18,286 - lava_callback - INFO - Starting async job submit for node 69cc7e398cd9cbcdc945ac41 2026-04-03 02:09:18,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:09:18,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7e398cd9cbcdc945ac41 HTTP/1.1" 200 None 2026-04-03 02:09:18,327 - lava_callback - DEBUG - Processing node 69cc7e398cd9cbcdc945ac41: baseline-arm64-baylibre 2026-04-03 02:09:18,327 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc7e398cd9cbcdc945ac41 2026-04-03 02:09:18,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:09:18,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7e398cd9cbcdc945ac41 HTTP/1.1" 200 None 2026-04-03 02:09:18,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:09:18,356 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:09:18,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:09:18,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7e398cd9cbcdc945ac41 HTTP/1.1" 200 None 2026-04-03 02:09:18,389 - lava_callback - ERROR - Error processing callback for node 69cc7e398cd9cbcdc945ac41: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38402 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:10:38,311 - lava_callback - INFO - Received callback for node 69cc7e978cd9cbcdc945ac92 INFO: 172.21.0.1:38412 - "POST /node/69cc7e978cd9cbcdc945ac92 HTTP/1.0" 202 Accepted 2026-04-03 02:10:38,319 - lava_callback - INFO - Starting async job submit for node 69cc7e978cd9cbcdc945ac92 2026-04-03 02:10:38,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:10:38,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7e978cd9cbcdc945ac92 HTTP/1.1" 200 None 2026-04-03 02:10:38,336 - lava_callback - DEBUG - Processing node 69cc7e978cd9cbcdc945ac92: baseline-arm64-baylibre 2026-04-03 02:10:38,336 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc7e978cd9cbcdc945ac92 2026-04-03 02:10:38,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:10:38,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7e978cd9cbcdc945ac92 HTTP/1.1" 200 None 2026-04-03 02:10:38,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:10:38,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:10:38,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:10:38,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7e978cd9cbcdc945ac92 HTTP/1.1" 200 None 2026-04-03 02:10:38,411 - lava_callback - ERROR - Error processing callback for node 69cc7e978cd9cbcdc945ac92: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38020 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:11:12,097 - lava_callback - INFO - Received callback for node 69cf218b9e2394c38fe025dc INFO: 172.21.0.1:43010 - "POST /node/69cf218b9e2394c38fe025dc HTTP/1.0" 202 Accepted 2026-04-03 02:11:12,107 - lava_callback - INFO - Starting async job submit for node 69cf218b9e2394c38fe025dc 2026-04-03 02:11:12,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:11:12,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf218b9e2394c38fe025dc HTTP/1.1" 200 None 2026-04-03 02:11:12,167 - lava_callback - DEBUG - Processing node 69cf218b9e2394c38fe025dc: baseline-arm64 2026-04-03 02:11:12,307 - lava_callback - WARNING - Failed to upload log 2026-04-03 02:11:12,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:11:12,586 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:11:12,588 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf218b9e2394c38fe025dc/lava-logs.yaml 2026-04-03 02:11:12,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:11:12,738 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:11:12,740 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf218b9e2394c38fe025dc/lava_callback.json.gz 2026-04-03 02:11:12,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:11:12,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf218b9e2394c38fe025dc HTTP/1.1" 200 None 2026-04-03 02:11:12,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:11:12,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:11:12,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:11:12,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf218b9e2394c38fe025dc HTTP/1.1" 200 None 2026-04-03 02:11:12,922 - lava_callback - INFO - Completed processing callback for node 69cf218b9e2394c38fe025dc INFO: 172.21.0.1:43016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53676 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:11:41,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:11:41,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46006 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:12:35,458 - lava_callback - INFO - Received callback for node 69cc7efd8cd9cbcdc945acd4 INFO: 172.21.0.1:60808 - "POST /node/69cc7efd8cd9cbcdc945acd4 HTTP/1.0" 202 Accepted 2026-04-03 02:12:35,463 - lava_callback - INFO - Starting async job submit for node 69cc7efd8cd9cbcdc945acd4 2026-04-03 02:12:35,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:12:35,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7efd8cd9cbcdc945acd4 HTTP/1.1" 200 None 2026-04-03 02:12:35,479 - lava_callback - DEBUG - Processing node 69cc7efd8cd9cbcdc945acd4: baseline-arm64-clabbe 2026-04-03 02:12:35,479 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc7efd8cd9cbcdc945acd4 2026-04-03 02:12:35,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:12:35,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7efd8cd9cbcdc945acd4 HTTP/1.1" 200 None 2026-04-03 02:12:35,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:12:35,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:12:35,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:12:35,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7efd8cd9cbcdc945acd4 HTTP/1.1" 200 None 2026-04-03 02:12:35,537 - lava_callback - ERROR - Error processing callback for node 69cc7efd8cd9cbcdc945acd4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38614 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:13:55,438 - lava_callback - INFO - Received callback for node 69cc7f558cd9cbcdc945ad09 INFO: 172.21.0.1:38982 - "POST /node/69cc7f558cd9cbcdc945ad09 HTTP/1.0" 202 Accepted 2026-04-03 02:13:55,443 - lava_callback - INFO - Starting async job submit for node 69cc7f558cd9cbcdc945ad09 2026-04-03 02:13:55,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:13:55,463 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7f558cd9cbcdc945ad09 HTTP/1.1" 200 None 2026-04-03 02:13:55,464 - lava_callback - DEBUG - Processing node 69cc7f558cd9cbcdc945ad09: baseline-arm64-clabbe 2026-04-03 02:13:55,464 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc7f558cd9cbcdc945ad09 2026-04-03 02:13:55,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:13:55,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7f558cd9cbcdc945ad09 HTTP/1.1" 200 None 2026-04-03 02:13:55,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:13:55,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:13:55,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:13:55,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7f558cd9cbcdc945ad09 HTTP/1.1" 200 None 2026-04-03 02:13:55,521 - lava_callback - ERROR - Error processing callback for node 69cc7f558cd9cbcdc945ad09: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41844 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:14:54,181 - lava_callback - INFO - Received callback for node 69cf22679e2394c38fe02697 INFO: 172.21.0.1:53416 - "POST /node/69cf22679e2394c38fe02697 HTTP/1.0" 202 Accepted 2026-04-03 02:14:54,187 - lava_callback - INFO - Starting async job submit for node 69cf22679e2394c38fe02697 2026-04-03 02:14:54,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:14:54,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf22679e2394c38fe02697 HTTP/1.1" 200 None 2026-04-03 02:14:54,232 - lava_callback - DEBUG - Processing node 69cf22679e2394c38fe02697: baseline-arm64 2026-04-03 02:14:54,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:14:54,661 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:14:54,663 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf22679e2394c38fe02697/log.txt.gz 2026-04-03 02:14:54,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:14:54,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:14:54,927 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf22679e2394c38fe02697/lava-logs.yaml 2026-04-03 02:14:54,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:14:55,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:14:55,110 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf22679e2394c38fe02697/lava_callback.json.gz 2026-04-03 02:14:55,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:14:55,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf22679e2394c38fe02697 HTTP/1.1" 200 None 2026-04-03 02:14:55,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:14:55,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:14:55,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:14:55,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf22679e2394c38fe02697 HTTP/1.1" 200 None 2026-04-03 02:14:55,291 - lava_callback - INFO - Completed processing callback for node 69cf22679e2394c38fe02697 INFO: 172.21.0.1:53418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52448 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:15:11,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:15:11,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37232 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 02:15:15,513 - lava_callback - INFO - Received callback for node 69cc7fad8cd9cbcdc945ad36 INFO: 172.21.0.1:37240 - "POST /node/69cc7fad8cd9cbcdc945ad36 HTTP/1.0" 202 Accepted 2026-04-03 02:15:15,519 - lava_callback - INFO - Starting async job submit for node 69cc7fad8cd9cbcdc945ad36 2026-04-03 02:15:15,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:15:15,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7fad8cd9cbcdc945ad36 HTTP/1.1" 200 None 2026-04-03 02:15:15,535 - lava_callback - DEBUG - Processing node 69cc7fad8cd9cbcdc945ad36: baseline-arm64-clabbe 2026-04-03 02:15:15,535 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc7fad8cd9cbcdc945ad36 2026-04-03 02:15:15,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:15:15,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc7fad8cd9cbcdc945ad36 HTTP/1.1" 200 None 2026-04-03 02:15:15,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:15:15,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:15:15,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:15:15,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc7fad8cd9cbcdc945ad36 HTTP/1.1" 200 None 2026-04-03 02:15:15,593 - lava_callback - ERROR - Error processing callback for node 69cc7fad8cd9cbcdc945ad36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33170 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:16:55,712 - lava_callback - INFO - Received callback for node 69cc80078cd9cbcdc945ad65 INFO: 172.21.0.1:52440 - "POST /node/69cc80078cd9cbcdc945ad65 HTTP/1.0" 202 Accepted 2026-04-03 02:16:55,718 - lava_callback - INFO - Starting async job submit for node 69cc80078cd9cbcdc945ad65 2026-04-03 02:16:55,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:16:55,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc80078cd9cbcdc945ad65 HTTP/1.1" 200 None 2026-04-03 02:16:55,733 - lava_callback - DEBUG - Processing node 69cc80078cd9cbcdc945ad65: baseline-arm64-clabbe 2026-04-03 02:16:55,733 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc80078cd9cbcdc945ad65 2026-04-03 02:16:55,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:16:55,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc80078cd9cbcdc945ad65 HTTP/1.1" 200 None 2026-04-03 02:16:55,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:16:55,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:16:55,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:16:55,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc80078cd9cbcdc945ad65 HTTP/1.1" 200 None 2026-04-03 02:16:55,797 - lava_callback - ERROR - Error processing callback for node 69cc80078cd9cbcdc945ad65: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36068 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 02:18:15,565 - lava_callback - INFO - Received callback for node 69cc805d8cd9cbcdc945ad9a INFO: 172.21.0.1:36078 - "POST /node/69cc805d8cd9cbcdc945ad9a HTTP/1.0" 202 Accepted 2026-04-03 02:18:15,572 - lava_callback - INFO - Starting async job submit for node 69cc805d8cd9cbcdc945ad9a 2026-04-03 02:18:15,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:18:15,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc805d8cd9cbcdc945ad9a HTTP/1.1" 200 None 2026-04-03 02:18:15,588 - lava_callback - DEBUG - Processing node 69cc805d8cd9cbcdc945ad9a: baseline-arm64-clabbe 2026-04-03 02:18:15,589 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc805d8cd9cbcdc945ad9a 2026-04-03 02:18:15,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:18:15,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc805d8cd9cbcdc945ad9a HTTP/1.1" 200 None 2026-04-03 02:18:15,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:18:15,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:18:15,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:18:15,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc805d8cd9cbcdc945ad9a HTTP/1.1" 200 None 2026-04-03 02:18:15,658 - lava_callback - ERROR - Error processing callback for node 69cc805d8cd9cbcdc945ad9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37024 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:19:55,691 - lava_callback - INFO - Received callback for node 69cc80b68cd9cbcdc945add2 INFO: 172.21.0.1:36754 - "POST /node/69cc80b68cd9cbcdc945add2 HTTP/1.0" 202 Accepted 2026-04-03 02:19:55,697 - lava_callback - INFO - Starting async job submit for node 69cc80b68cd9cbcdc945add2 2026-04-03 02:19:55,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:19:55,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc80b68cd9cbcdc945add2 HTTP/1.1" 200 None 2026-04-03 02:19:55,720 - lava_callback - DEBUG - Processing node 69cc80b68cd9cbcdc945add2: baseline-arm64-clabbe 2026-04-03 02:19:55,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc80b68cd9cbcdc945add2 2026-04-03 02:19:55,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:19:55,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc80b68cd9cbcdc945add2 HTTP/1.1" 200 None 2026-04-03 02:19:55,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:19:55,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 02:19:55,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:19:55,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc80b68cd9cbcdc945add2 HTTP/1.1" 200 None 2026-04-03 02:19:55,785 - lava_callback - ERROR - Error processing callback for node 69cc80b68cd9cbcdc945add2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37538 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:20:58,288 - lava_callback - INFO - Received callback for node 69cf23859e2394c38fe02762 INFO: 172.21.0.1:37552 - "POST /node/69cf23859e2394c38fe02762 HTTP/1.0" 202 Accepted 2026-04-03 02:20:58,296 - lava_callback - INFO - Starting async job submit for node 69cf23859e2394c38fe02762 2026-04-03 02:20:58,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:20:58,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf23859e2394c38fe02762 HTTP/1.1" 200 None 2026-04-03 02:20:58,337 - lava_callback - DEBUG - Processing node 69cf23859e2394c38fe02762: baseline-arm64 2026-04-03 02:20:58,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:20:59,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:20:59,016 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf23859e2394c38fe02762/log.txt.gz 2026-04-03 02:20:59,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:20:59,527 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:20:59,529 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf23859e2394c38fe02762/lava-logs.yaml 2026-04-03 02:20:59,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:20:59,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:20:59,697 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf23859e2394c38fe02762/lava_callback.json.gz 2026-04-03 02:20:59,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:20:59,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf23859e2394c38fe02762 HTTP/1.1" 200 None 2026-04-03 02:20:59,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:20:59,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:20:59,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:20:59,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf23859e2394c38fe02762 HTTP/1.1" 200 None 2026-04-03 02:20:59,850 - lava_callback - INFO - Completed processing callback for node 69cf23859e2394c38fe02762 INFO: 172.21.0.1:57282 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:21:08,490 - lava_callback - INFO - Received callback for node 69cf1f889e2394c38fe0240b INFO: 172.21.0.1:57294 - "POST /node/69cf1f889e2394c38fe0240b HTTP/1.0" 202 Accepted 2026-04-03 02:21:08,537 - lava_callback - INFO - Starting async job submit for node 69cf1f889e2394c38fe0240b 2026-04-03 02:21:08,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:21:08,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1f889e2394c38fe0240b HTTP/1.1" 200 None 2026-04-03 02:21:08,580 - lava_callback - DEBUG - Processing node 69cf1f889e2394c38fe0240b: baseline-arm-pengutronix 2026-04-03 02:21:08,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:21:09,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:21:09,011 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf1f889e2394c38fe0240b/log.txt.gz 2026-04-03 02:21:09,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:21:09,264 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:21:09,266 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf1f889e2394c38fe0240b/lava-logs.yaml 2026-04-03 02:21:09,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:21:09,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:21:09,409 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf1f889e2394c38fe0240b/lava_callback.json.gz DEBUG: Setup failed for 69cf1f889e2394c38fe0240b. Transitting job node result: incomplete -> fail 2026-04-03 02:21:09,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:21:09,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf1f889e2394c38fe0240b HTTP/1.1" 200 None 2026-04-03 02:21:09,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:21:09,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:21:09,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:21:09,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf1f889e2394c38fe0240b HTTP/1.1" 200 None 2026-04-03 02:21:09,575 - lava_callback - INFO - Completed processing callback for node 69cf1f889e2394c38fe0240b 2026-04-03 02:21:11,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:21:11,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40114 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:25:50,133 - lava_callback - INFO - Received callback for node 69cf20779e2394c38fe024fb INFO: 172.21.0.1:56280 - "POST /node/69cf20779e2394c38fe024fb HTTP/1.0" 202 Accepted 2026-04-03 02:25:50,142 - lava_callback - INFO - Starting async job submit for node 69cf20779e2394c38fe024fb 2026-04-03 02:25:50,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:25:50,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf20779e2394c38fe024fb HTTP/1.1" 200 None 2026-04-03 02:25:50,202 - lava_callback - DEBUG - Processing node 69cf20779e2394c38fe024fb: baseline-nfs-arm64 2026-04-03 02:25:51,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:25:51,511 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:25:51,512 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf20779e2394c38fe024fb/log.txt.gz 2026-04-03 02:25:52,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:25:52,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:25:52,565 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf20779e2394c38fe024fb/lava-logs.yaml 2026-04-03 02:25:52,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:25:52,764 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:25:52,766 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf20779e2394c38fe024fb/lava_callback.json.gz DEBUG: Setup failed for 69cf20779e2394c38fe024fb. Transitting job node result: incomplete -> fail 2026-04-03 02:25:52,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:25:52,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf20779e2394c38fe024fb HTTP/1.1" 200 None 2026-04-03 02:25:52,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:25:52,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:25:52,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:25:52,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf20779e2394c38fe024fb HTTP/1.1" 200 None 2026-04-03 02:25:53,011 - lava_callback - INFO - Completed processing callback for node 69cf20779e2394c38fe024fb INFO: 172.21.0.1:56288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60072 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:26:11,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:26:12,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60582 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:27:19,290 - lava_callback - INFO - Received callback for node 69cc82798cd9cbcdc945aeda INFO: 172.21.0.1:60586 - "POST /node/69cc82798cd9cbcdc945aeda HTTP/1.0" 202 Accepted 2026-04-03 02:27:19,295 - lava_callback - INFO - Starting async job submit for node 69cc82798cd9cbcdc945aeda 2026-04-03 02:27:19,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:19,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc82798cd9cbcdc945aeda HTTP/1.1" 200 None 2026-04-03 02:27:19,311 - lava_callback - DEBUG - Processing node 69cc82798cd9cbcdc945aeda: baseline-arm64-baylibre 2026-04-03 02:27:19,312 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc82798cd9cbcdc945aeda 2026-04-03 02:27:19,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:19,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc82798cd9cbcdc945aeda HTTP/1.1" 200 None 2026-04-03 02:27:19,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:19,339 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-03 02:27:19,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:19,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc82798cd9cbcdc945aeda HTTP/1.1" 200 None 2026-04-03 02:27:19,362 - lava_callback - ERROR - Error processing callback for node 69cc82798cd9cbcdc945aeda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54710 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:27:28,033 - lava_callback - INFO - Received callback for node 69cf16c59e2394c38fe01d48 INFO: 172.21.0.1:54722 - "POST /node/69cf16c59e2394c38fe01d48 HTTP/1.0" 202 Accepted 2026-04-03 02:27:28,039 - lava_callback - INFO - Starting async job submit for node 69cf16c59e2394c38fe01d48 2026-04-03 02:27:28,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:28,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf16c59e2394c38fe01d48 HTTP/1.1" 200 None 2026-04-03 02:27:28,079 - lava_callback - DEBUG - Processing node 69cf16c59e2394c38fe01d48: baseline-arm64-broonie 2026-04-03 02:27:28,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:27:28,496 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:27:28,498 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf16c59e2394c38fe01d48/log.txt.gz 2026-04-03 02:27:28,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:27:28,858 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:27:28,860 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf16c59e2394c38fe01d48/lava-logs.yaml 2026-04-03 02:27:28,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:27:29,039 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:27:29,040 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf16c59e2394c38fe01d48/lava_callback.json.gz 2026-04-03 02:27:29,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:29,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf16c59e2394c38fe01d48 HTTP/1.1" 200 None 2026-04-03 02:27:29,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:29,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:27:29,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:29,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf16c59e2394c38fe01d48 HTTP/1.1" 200 None 2026-04-03 02:27:29,120 - lava_callback - INFO - Completed processing callback for node 69cf16c59e2394c38fe01d48 INFO: 172.21.0.1:34270 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:27:42,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:42,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53068 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:27:47,807 - lava_callback - INFO - Received callback for node 69cf0fce9e2394c38fe01811 INFO: 172.21.0.1:53082 - "POST /node/69cf0fce9e2394c38fe01811 HTTP/1.0" 202 Accepted 2026-04-03 02:27:47,813 - lava_callback - INFO - Starting async job submit for node 69cf0fce9e2394c38fe01811 2026-04-03 02:27:47,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:47,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fce9e2394c38fe01811 HTTP/1.1" 200 None 2026-04-03 02:27:47,851 - lava_callback - DEBUG - Processing node 69cf0fce9e2394c38fe01811: baseline-arm64-broonie 2026-04-03 02:27:48,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:27:48,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:27:48,420 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fce9e2394c38fe01811/log.txt.gz 2026-04-03 02:27:48,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:27:48,819 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:27:48,821 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fce9e2394c38fe01811/lava-logs.yaml 2026-04-03 02:27:48,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:27:48,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:27:48,961 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fce9e2394c38fe01811/lava_callback.json.gz 2026-04-03 02:27:48,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:49,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fce9e2394c38fe01811 HTTP/1.1" 200 None 2026-04-03 02:27:49,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:49,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:27:49,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:27:49,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fce9e2394c38fe01811 HTTP/1.1" 200 None 2026-04-03 02:27:49,050 - lava_callback - INFO - Completed processing callback for node 69cf0fce9e2394c38fe01811 INFO: 172.21.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55202 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:28:12,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:28:12,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51472 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 02:30:16,173 - lava_callback - INFO - Received callback for node 69cc832f8cd9cbcdc945af2f INFO: 172.21.0.1:51478 - "POST /node/69cc832f8cd9cbcdc945af2f HTTP/1.0" 202 Accepted 2026-04-03 02:30:16,179 - lava_callback - INFO - Starting async job submit for node 69cc832f8cd9cbcdc945af2f 2026-04-03 02:30:16,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:30:16,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc832f8cd9cbcdc945af2f HTTP/1.1" 200 None 2026-04-03 02:30:16,201 - lava_callback - DEBUG - Processing node 69cc832f8cd9cbcdc945af2f: baseline-arm64-clabbe 2026-04-03 02:30:16,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc832f8cd9cbcdc945af2f 2026-04-03 02:30:16,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:30:16,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc832f8cd9cbcdc945af2f HTTP/1.1" 200 None 2026-04-03 02:30:16,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:30:16,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-03 02:30:16,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:30:16,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc832f8cd9cbcdc945af2f HTTP/1.1" 200 None 2026-04-03 02:30:16,265 - lava_callback - ERROR - Error processing callback for node 69cc832f8cd9cbcdc945af2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36122 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:34:36,302 - lava_callback - INFO - Received callback for node 69cc842d8cd9cbcdc945afa8 INFO: 172.21.0.1:48510 - "POST /node/69cc842d8cd9cbcdc945afa8 HTTP/1.0" 202 Accepted 2026-04-03 02:34:36,307 - lava_callback - INFO - Starting async job submit for node 69cc842d8cd9cbcdc945afa8 2026-04-03 02:34:36,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:34:36,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc842d8cd9cbcdc945afa8 HTTP/1.1" 200 None 2026-04-03 02:34:36,325 - lava_callback - DEBUG - Processing node 69cc842d8cd9cbcdc945afa8: baseline-arm64-clabbe 2026-04-03 02:34:36,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc842d8cd9cbcdc945afa8 2026-04-03 02:34:36,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:34:36,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc842d8cd9cbcdc945afa8 HTTP/1.1" 200 None 2026-04-03 02:34:36,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:34:36,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-03 02:34:36,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:34:36,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc842d8cd9cbcdc945afa8 HTTP/1.1" 200 None 2026-04-03 02:34:36,386 - lava_callback - ERROR - Error processing callback for node 69cc842d8cd9cbcdc945afa8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50248 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:34:47,144 - lava_callback - INFO - Received callback for node 69cf257f9e2394c38fe0292f INFO: 172.21.0.1:50262 - "POST /node/69cf257f9e2394c38fe0292f HTTP/1.0" 202 Accepted 2026-04-03 02:34:47,156 - lava_callback - INFO - Starting async job submit for node 69cf257f9e2394c38fe0292f 2026-04-03 02:34:47,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:34:47,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf257f9e2394c38fe0292f HTTP/1.1" 200 None 2026-04-03 02:34:47,225 - lava_callback - DEBUG - Processing node 69cf257f9e2394c38fe0292f: baseline-arm64 2026-04-03 02:34:47,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:34:47,846 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:34:47,848 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf257f9e2394c38fe0292f/log.txt.gz 2026-04-03 02:34:48,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:34:48,616 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:34:48,618 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf257f9e2394c38fe0292f/lava-logs.yaml 2026-04-03 02:34:48,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:34:48,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:34:48,797 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf257f9e2394c38fe0292f/lava_callback.json.gz DEBUG: Setup failed for 69cf257f9e2394c38fe0292f. Transitting job node result: incomplete -> fail 2026-04-03 02:34:48,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:34:48,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf257f9e2394c38fe0292f HTTP/1.1" 200 None 2026-04-03 02:34:48,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:34:48,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:34:48,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:34:48,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf257f9e2394c38fe0292f HTTP/1.1" 200 None 2026-04-03 02:34:49,024 - lava_callback - INFO - Completed processing callback for node 69cf257f9e2394c38fe0292f INFO: 172.21.0.1:41346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44504 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:35:12,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:35:12,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60262 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41672 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:36:36,385 - lava_callback - INFO - Received callback for node 69cc84ad8cd9cbcdc945afda INFO: 172.21.0.1:59398 - "POST /node/69cc84ad8cd9cbcdc945afda HTTP/1.0" 202 Accepted 2026-04-03 02:36:36,391 - lava_callback - INFO - Starting async job submit for node 69cc84ad8cd9cbcdc945afda 2026-04-03 02:36:36,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:36:36,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc84ad8cd9cbcdc945afda HTTP/1.1" 200 None 2026-04-03 02:36:36,414 - lava_callback - DEBUG - Processing node 69cc84ad8cd9cbcdc945afda: baseline-arm64-clabbe 2026-04-03 02:36:36,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc84ad8cd9cbcdc945afda 2026-04-03 02:36:36,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:36:36,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc84ad8cd9cbcdc945afda HTTP/1.1" 200 None 2026-04-03 02:36:36,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:36:36,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-03 02:36:36,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:36:36,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc84ad8cd9cbcdc945afda HTTP/1.1" 200 None 2026-04-03 02:36:36,474 - lava_callback - ERROR - Error processing callback for node 69cc84ad8cd9cbcdc945afda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56006 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:36:47,547 - lava_callback - INFO - Received callback for node 69cf0fd09e2394c38fe01819 INFO: 172.21.0.1:56018 - "POST /node/69cf0fd09e2394c38fe01819 HTTP/1.0" 202 Accepted 2026-04-03 02:36:47,556 - lava_callback - INFO - Starting async job submit for node 69cf0fd09e2394c38fe01819 2026-04-03 02:36:47,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:36:47,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fd09e2394c38fe01819 HTTP/1.1" 200 None 2026-04-03 02:36:47,596 - lava_callback - DEBUG - Processing node 69cf0fd09e2394c38fe01819: baseline-arm64-broonie 2026-04-03 02:36:47,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:36:47,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:36:47,993 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd09e2394c38fe01819/log.txt.gz 2026-04-03 02:36:48,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:36:48,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:36:48,390 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd09e2394c38fe01819/lava-logs.yaml 2026-04-03 02:36:48,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:36:48,544 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:36:48,546 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd09e2394c38fe01819/lava_callback.json.gz 2026-04-03 02:36:48,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:36:48,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fd09e2394c38fe01819 HTTP/1.1" 200 None 2026-04-03 02:36:48,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:36:48,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:36:48,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:36:48,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fd09e2394c38fe01819 HTTP/1.1" 200 None 2026-04-03 02:36:48,632 - lava_callback - INFO - Completed processing callback for node 69cf0fd09e2394c38fe01819 INFO: 172.21.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35980 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:37:12,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:37:12,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44818 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:37:56,455 - lava_callback - INFO - Received callback for node 69cc85008cd9cbcdc945b003 INFO: 172.21.0.1:41952 - "POST /node/69cc85008cd9cbcdc945b003 HTTP/1.0" 202 Accepted 2026-04-03 02:37:56,460 - lava_callback - INFO - Starting async job submit for node 69cc85008cd9cbcdc945b003 2026-04-03 02:37:56,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:37:56,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc85008cd9cbcdc945b003 HTTP/1.1" 200 None 2026-04-03 02:37:56,476 - lava_callback - DEBUG - Processing node 69cc85008cd9cbcdc945b003: baseline-arm64-clabbe 2026-04-03 02:37:56,476 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc85008cd9cbcdc945b003 2026-04-03 02:37:56,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:37:56,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc85008cd9cbcdc945b003 HTTP/1.1" 200 None 2026-04-03 02:37:56,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:41960 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:37:56,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-03 02:37:56,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:37:56,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc85008cd9cbcdc945b003 HTTP/1.1" 200 None 2026-04-03 02:37:56,536 - lava_callback - ERROR - Error processing callback for node 69cc85008cd9cbcdc945b003: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53448 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56106 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:38:27,735 - lava_callback - INFO - Received callback for node 69cf28199e2394c38fe02b58 INFO: 172.21.0.1:56108 - "POST /node/69cf28199e2394c38fe02b58 HTTP/1.0" 202 Accepted 2026-04-03 02:38:27,742 - lava_callback - INFO - Starting async job submit for node 69cf28199e2394c38fe02b58 2026-04-03 02:38:27,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:38:27,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf28199e2394c38fe02b58 HTTP/1.1" 200 None 2026-04-03 02:38:27,759 - lava_callback - DEBUG - Processing node 69cf28199e2394c38fe02b58: baseline-nfs-arm64 2026-04-03 02:38:27,760 - lava_callback - WARNING - Failed to upload log 2026-04-03 02:38:27,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:38:28,080 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:38:28,082 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf28199e2394c38fe02b58/lava-logs.yaml 2026-04-03 02:38:28,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:38:28,228 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:38:28,230 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf28199e2394c38fe02b58/lava_callback.json.gz 2026-04-03 02:38:28,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:38:28,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf28199e2394c38fe02b58 HTTP/1.1" 200 None 2026-04-03 02:38:28,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:38:28,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:38:28,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:38:28,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf28199e2394c38fe02b58 HTTP/1.1" 200 None 2026-04-03 02:38:28,296 - lava_callback - INFO - Completed processing callback for node 69cf28199e2394c38fe02b58 INFO: 172.21.0.1:48648 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:38:42,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:38:42,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43126 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:39:36,480 - lava_callback - INFO - Received callback for node 69cc85578cd9cbcdc945b02b INFO: 172.21.0.1:40020 - "POST /node/69cc85578cd9cbcdc945b02b HTTP/1.0" 202 Accepted 2026-04-03 02:39:36,489 - lava_callback - INFO - Starting async job submit for node 69cc85578cd9cbcdc945b02b 2026-04-03 02:39:36,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:39:36,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc85578cd9cbcdc945b02b HTTP/1.1" 200 None 2026-04-03 02:39:36,505 - lava_callback - DEBUG - Processing node 69cc85578cd9cbcdc945b02b: baseline-arm64-clabbe 2026-04-03 02:39:36,506 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc85578cd9cbcdc945b02b 2026-04-03 02:39:36,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:40028 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:39:36,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc85578cd9cbcdc945b02b HTTP/1.1" 200 None 2026-04-03 02:39:36,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:39:36,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-03 02:39:36,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:39:36,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc85578cd9cbcdc945b02b HTTP/1.1" 200 None 2026-04-03 02:39:36,579 - lava_callback - ERROR - Error processing callback for node 69cc85578cd9cbcdc945b02b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58386 - "GET /node/693f87b60d8ce4018da96cb4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:58394 - "GET /node/6938c680e767fbc36e03e605%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56308 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:41:10,475 - lava_callback - INFO - Received callback for node 69cf26599e2394c38fe029bf INFO: 172.21.0.1:59810 - "POST /node/69cf26599e2394c38fe029bf HTTP/1.0" 202 Accepted 2026-04-03 02:41:10,481 - lava_callback - INFO - Starting async job submit for node 69cf26599e2394c38fe029bf 2026-04-03 02:41:10,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:41:10,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf26599e2394c38fe029bf HTTP/1.1" 200 None 2026-04-03 02:41:10,523 - lava_callback - DEBUG - Processing node 69cf26599e2394c38fe029bf: baseline-nfs-arm64 2026-04-03 02:41:11,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:41:11,338 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:41:11,340 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf26599e2394c38fe029bf/log.txt.gz 2026-04-03 02:41:11,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:41:11,859 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:41:11,861 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf26599e2394c38fe029bf/lava-logs.yaml 2026-04-03 02:41:11,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:41:12,054 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:41:12,056 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf26599e2394c38fe029bf/lava_callback.json.gz 2026-04-03 02:41:12,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:41:12,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf26599e2394c38fe029bf HTTP/1.1" 200 None 2026-04-03 02:41:12,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:41:12,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:41:12,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:41:12,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf26599e2394c38fe029bf HTTP/1.1" 200 None 2026-04-03 02:41:12,208 - lava_callback - INFO - Completed processing callback for node 69cf26599e2394c38fe029bf INFO: 172.21.0.1:59826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34344 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:41:36,614 - lava_callback - INFO - Received callback for node 69cc85d68cd9cbcdc945b062 INFO: 172.21.0.1:34346 - "POST /node/69cc85d68cd9cbcdc945b062 HTTP/1.0" 202 Accepted 2026-04-03 02:41:36,619 - lava_callback - INFO - Starting async job submit for node 69cc85d68cd9cbcdc945b062 2026-04-03 02:41:36,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:41:36,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc85d68cd9cbcdc945b062 HTTP/1.1" 200 None 2026-04-03 02:41:36,636 - lava_callback - DEBUG - Processing node 69cc85d68cd9cbcdc945b062: baseline-arm64-clabbe 2026-04-03 02:41:36,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc85d68cd9cbcdc945b062 2026-04-03 02:41:36,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:41:36,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc85d68cd9cbcdc945b062 HTTP/1.1" 200 None 2026-04-03 02:41:36,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:41:36,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-04-03 02:41:36,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:41:36,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc85d68cd9cbcdc945b062 HTTP/1.1" 200 None 2026-04-03 02:41:36,692 - lava_callback - ERROR - Error processing callback for node 69cc85d68cd9cbcdc945b062: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-03 02:41:42,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:41:42,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51684 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41256 - "GET /node/6992eebe31166481c8dda093%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:41260 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:42:28,006 - lava_callback - INFO - Received callback for node 69cf24ec9e2394c38fe0289c INFO: 172.21.0.1:41262 - "POST /node/69cf24ec9e2394c38fe0289c HTTP/1.0" 202 Accepted 2026-04-03 02:42:28,013 - lava_callback - INFO - Starting async job submit for node 69cf24ec9e2394c38fe0289c 2026-04-03 02:42:28,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:42:28,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf24ec9e2394c38fe0289c HTTP/1.1" 200 None 2026-04-03 02:42:28,092 - lava_callback - DEBUG - Processing node 69cf24ec9e2394c38fe0289c: ltp-smoke 2026-04-03 02:42:28,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:42:28,737 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:42:28,738 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf24ec9e2394c38fe0289c/log.txt.gz 2026-04-03 02:42:29,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:42:29,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:42:29,304 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf24ec9e2394c38fe0289c/lava-logs.yaml 2026-04-03 02:42:29,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:42:29,555 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:42:29,557 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69cf24ec9e2394c38fe0289c/lava_callback.json.gz 2026-04-03 02:42:29,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:42:29,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf24ec9e2394c38fe0289c HTTP/1.1" 200 None 2026-04-03 02:42:29,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:42:29,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf07039e2394c38fe00f45 HTTP/1.1" 200 None 2026-04-03 02:42:29,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:42:29,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf24ec9e2394c38fe0289c HTTP/1.1" 200 None 2026-04-03 02:42:29,874 - lava_callback - INFO - Completed processing callback for node 69cf24ec9e2394c38fe0289c 2026-04-03 02:42:33,243 - lava_callback - INFO - Received callback for node 69cf28df9e2394c38fe02c16 INFO: 172.21.0.1:34284 - "POST /node/69cf28df9e2394c38fe02c16 HTTP/1.0" 202 Accepted 2026-04-03 02:42:33,250 - lava_callback - INFO - Starting async job submit for node 69cf28df9e2394c38fe02c16 2026-04-03 02:42:33,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:42:33,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf28df9e2394c38fe02c16 HTTP/1.1" 200 None 2026-04-03 02:42:33,297 - lava_callback - DEBUG - Processing node 69cf28df9e2394c38fe02c16: baseline-arm64 2026-04-03 02:42:33,428 - lava_callback - WARNING - Failed to upload log 2026-04-03 02:42:33,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:42:33,673 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:42:33,676 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf28df9e2394c38fe02c16/lava-logs.yaml 2026-04-03 02:42:33,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:42:33,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:42:33,886 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf28df9e2394c38fe02c16/lava_callback.json.gz 2026-04-03 02:42:33,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:42:33,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf28df9e2394c38fe02c16 HTTP/1.1" 200 None 2026-04-03 02:42:33,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:42:33,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:42:33,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:42:33,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf28df9e2394c38fe02c16 HTTP/1.1" 200 None 2026-04-03 02:42:34,059 - lava_callback - INFO - Completed processing callback for node 69cf28df9e2394c38fe02c16 INFO: 172.21.0.1:34286 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:42:42,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:42:42,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40728 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:43:40,216 - lava_callback - INFO - Received callback for node 69cf260c9e2394c38fe0299a INFO: 172.21.0.1:53684 - "POST /node/69cf260c9e2394c38fe0299a HTTP/1.0" 202 Accepted 2026-04-03 02:43:40,222 - lava_callback - INFO - Starting async job submit for node 69cf260c9e2394c38fe0299a 2026-04-03 02:43:40,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:43:40,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf260c9e2394c38fe0299a HTTP/1.1" 200 None 2026-04-03 02:43:40,276 - lava_callback - DEBUG - Processing node 69cf260c9e2394c38fe0299a: baseline-arm64 2026-04-03 02:43:40,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:43:40,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:43:40,805 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf260c9e2394c38fe0299a/log.txt.gz 2026-04-03 02:43:40,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:43:41,245 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:43:41,247 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf260c9e2394c38fe0299a/lava-logs.yaml 2026-04-03 02:43:41,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:43:41,915 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:43:41,916 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf260c9e2394c38fe0299a/lava_callback.json.gz 2026-04-03 02:43:41,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:43:41,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf260c9e2394c38fe0299a HTTP/1.1" 200 None 2026-04-03 02:43:41,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:43:41,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:43:41,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:43:42,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf260c9e2394c38fe0299a HTTP/1.1" 200 None 2026-04-03 02:43:42,084 - lava_callback - INFO - Completed processing callback for node 69cf260c9e2394c38fe0299a 2026-04-03 02:43:42,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:43:42,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48342 - "GET /node/6941898cecc59770562e6ebd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48358 - "GET /node/69488df448abaf7e255845a9%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46590 - "GET /node/69523a695318368af0c49bc6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:53806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43066 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:45:54,746 - lava_callback - INFO - Received callback for node 69cf29ac9e2394c38fe02cbb INFO: 172.21.0.1:59838 - "POST /node/69cf29ac9e2394c38fe02cbb HTTP/1.0" 202 Accepted 2026-04-03 02:45:54,753 - lava_callback - INFO - Starting async job submit for node 69cf29ac9e2394c38fe02cbb 2026-04-03 02:45:54,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:45:54,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf29ac9e2394c38fe02cbb HTTP/1.1" 200 None 2026-04-03 02:45:54,797 - lava_callback - DEBUG - Processing node 69cf29ac9e2394c38fe02cbb: baseline-arm64 2026-04-03 02:45:54,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:45:55,349 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:45:55,351 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf29ac9e2394c38fe02cbb/log.txt.gz 2026-04-03 02:45:55,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:45:55,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:45:55,623 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf29ac9e2394c38fe02cbb/lava-logs.yaml 2026-04-03 02:45:55,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:45:55,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:45:55,754 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf29ac9e2394c38fe02cbb/lava_callback.json.gz 2026-04-03 02:45:55,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:45:55,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf29ac9e2394c38fe02cbb HTTP/1.1" 200 None 2026-04-03 02:45:55,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:45:55,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:45:55,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:45:55,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf29ac9e2394c38fe02cbb HTTP/1.1" 200 None 2026-04-03 02:45:55,908 - lava_callback - INFO - Completed processing callback for node 69cf29ac9e2394c38fe02cbb INFO: 172.21.0.1:59854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53510 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:46:12,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:46:12,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51248 - "GET /node/69c1dcb80efaf21d93103631%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51258 - "GET /node/6986ffe16e077e4126a7e133%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:58552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38064 - "GET /node/69935090523507f3806a69c8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:38074 - "GET /node/68826ba987dcced189497f38%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:38090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32794 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:48:00,386 - lava_callback - INFO - Received callback for node 69cc87578cd9cbcdc945b10e INFO: 172.21.0.1:47782 - "POST /node/69cc87578cd9cbcdc945b10e HTTP/1.0" 202 Accepted 2026-04-03 02:48:00,392 - lava_callback - INFO - Starting async job submit for node 69cc87578cd9cbcdc945b10e 2026-04-03 02:48:00,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:48:00,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc87578cd9cbcdc945b10e HTTP/1.1" 200 None 2026-04-03 02:48:00,410 - lava_callback - DEBUG - Processing node 69cc87578cd9cbcdc945b10e: baseline-arm-baylibre 2026-04-03 02:48:00,410 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc87578cd9cbcdc945b10e 2026-04-03 02:48:00,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:48:00,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc87578cd9cbcdc945b10e HTTP/1.1" 200 None 2026-04-03 02:48:00,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:48:00,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-03 02:48:00,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:48:00,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc87578cd9cbcdc945b10e HTTP/1.1" 200 None 2026-04-03 02:48:00,467 - lava_callback - ERROR - Error processing callback for node 69cc87578cd9cbcdc945b10e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34484 - "GET /node/69938cf3cebe99dd8e0720a8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:34488 - "GET /node/693afa5dfee8383e92b6d129%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:34500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45688 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:50:20,536 - lava_callback - INFO - Received callback for node 69cc87da8cd9cbcdc945b155 INFO: 172.21.0.1:48276 - "POST /node/69cc87da8cd9cbcdc945b155 HTTP/1.0" 202 Accepted 2026-04-03 02:50:20,542 - lava_callback - INFO - Starting async job submit for node 69cc87da8cd9cbcdc945b155 2026-04-03 02:50:20,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:50:20,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc87da8cd9cbcdc945b155 HTTP/1.1" 200 None 2026-04-03 02:50:20,559 - lava_callback - DEBUG - Processing node 69cc87da8cd9cbcdc945b155: baseline-arm-baylibre 2026-04-03 02:50:20,559 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc87da8cd9cbcdc945b155 2026-04-03 02:50:20,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:50:20,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc87da8cd9cbcdc945b155 HTTP/1.1" 200 None 2026-04-03 02:50:20,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:50:20,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-03 02:50:20,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:50:20,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc87da8cd9cbcdc945b155 HTTP/1.1" 200 None 2026-04-03 02:50:20,610 - lava_callback - ERROR - Error processing callback for node 69cc87da8cd9cbcdc945b155: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49420 - "GET /node/6939340d6e7b0b7f22e51bea%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-03 02:51:00,514 - lava_callback - INFO - Received callback for node 69cc88038cd9cbcdc945b162 INFO: 172.21.0.1:51302 - "POST /node/69cc88038cd9cbcdc945b162 HTTP/1.0" 202 Accepted 2026-04-03 02:51:00,520 - lava_callback - INFO - Starting async job submit for node 69cc88038cd9cbcdc945b162 2026-04-03 02:51:00,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:00,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc88038cd9cbcdc945b162 HTTP/1.1" 200 None 2026-04-03 02:51:00,539 - lava_callback - DEBUG - Processing node 69cc88038cd9cbcdc945b162: baseline-arm-baylibre 2026-04-03 02:51:00,539 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc88038cd9cbcdc945b162 2026-04-03 02:51:00,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:00,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc88038cd9cbcdc945b162 HTTP/1.1" 200 None 2026-04-03 02:51:00,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:00,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-03 02:51:00,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:00,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc88038cd9cbcdc945b162 HTTP/1.1" 200 None 2026-04-03 02:51:00,622 - lava_callback - ERROR - Error processing callback for node 69cc88038cd9cbcdc945b162: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51316 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:51:09,598 - lava_callback - INFO - Received callback for node 69cf26da9e2394c38fe02a1b INFO: 172.21.0.1:51326 - "POST /node/69cf26da9e2394c38fe02a1b HTTP/1.0" 202 Accepted 2026-04-03 02:51:09,608 - lava_callback - INFO - Starting async job submit for node 69cf26da9e2394c38fe02a1b 2026-04-03 02:51:09,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:09,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf26da9e2394c38fe02a1b HTTP/1.1" 200 None 2026-04-03 02:51:09,680 - lava_callback - DEBUG - Processing node 69cf26da9e2394c38fe02a1b: baseline-arm 2026-04-03 02:51:10,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:51:10,533 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:51:10,535 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf26da9e2394c38fe02a1b/log.txt.gz 2026-04-03 02:51:10,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:51:11,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:51:11,201 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf26da9e2394c38fe02a1b/lava-logs.yaml 2026-04-03 02:51:11,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:51:11,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:51:11,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf26da9e2394c38fe02a1b/lava_callback.json.gz DEBUG: Setup failed for 69cf26da9e2394c38fe02a1b. Transitting job node result: incomplete -> fail 2026-04-03 02:51:11,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:11,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf26da9e2394c38fe02a1b HTTP/1.1" 200 None 2026-04-03 02:51:11,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:11,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 02:51:11,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:11,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf26da9e2394c38fe02a1b HTTP/1.1" 200 None 2026-04-03 02:51:11,718 - lava_callback - INFO - Completed processing callback for node 69cf26da9e2394c38fe02a1b 2026-04-03 02:51:12,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:12,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41438 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:51:40,585 - lava_callback - INFO - Received callback for node 69cc882c8cd9cbcdc945b16d INFO: 172.21.0.1:34030 - "POST /node/69cc882c8cd9cbcdc945b16d HTTP/1.0" 202 Accepted 2026-04-03 02:51:40,591 - lava_callback - INFO - Starting async job submit for node 69cc882c8cd9cbcdc945b16d 2026-04-03 02:51:40,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:40,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc882c8cd9cbcdc945b16d HTTP/1.1" 200 None 2026-04-03 02:51:40,607 - lava_callback - DEBUG - Processing node 69cc882c8cd9cbcdc945b16d: baseline-arm-baylibre 2026-04-03 02:51:40,608 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc882c8cd9cbcdc945b16d 2026-04-03 02:51:40,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:40,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc882c8cd9cbcdc945b16d HTTP/1.1" 200 None 2026-04-03 02:51:40,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:40,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-03 02:51:40,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:51:40,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc882c8cd9cbcdc945b16d HTTP/1.1" 200 None 2026-04-03 02:51:40,673 - lava_callback - ERROR - Error processing callback for node 69cc882c8cd9cbcdc945b16d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33210 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:52:20,579 - lava_callback - INFO - Received callback for node 69cc88588cd9cbcdc945b17d INFO: 172.21.0.1:38444 - "POST /node/69cc88588cd9cbcdc945b17d HTTP/1.0" 202 Accepted 2026-04-03 02:52:20,584 - lava_callback - INFO - Starting async job submit for node 69cc88588cd9cbcdc945b17d 2026-04-03 02:52:20,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:52:20,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc88588cd9cbcdc945b17d HTTP/1.1" 200 None 2026-04-03 02:52:20,600 - lava_callback - DEBUG - Processing node 69cc88588cd9cbcdc945b17d: baseline-arm-baylibre 2026-04-03 02:52:20,601 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc88588cd9cbcdc945b17d 2026-04-03 02:52:20,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:52:20,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc88588cd9cbcdc945b17d HTTP/1.1" 200 None 2026-04-03 02:52:20,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:52:20,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-03 02:52:20,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:52:20,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc88588cd9cbcdc945b17d HTTP/1.1" 200 None 2026-04-03 02:52:20,668 - lava_callback - ERROR - Error processing callback for node 69cc88588cd9cbcdc945b17d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38448 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:52:29,446 - lava_callback - INFO - Received callback for node 69cf0fd49e2394c38fe0182a INFO: 172.21.0.1:38458 - "POST /node/69cf0fd49e2394c38fe0182a HTTP/1.0" 202 Accepted 2026-04-03 02:52:29,452 - lava_callback - INFO - Starting async job submit for node 69cf0fd49e2394c38fe0182a 2026-04-03 02:52:29,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:52:29,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fd49e2394c38fe0182a HTTP/1.1" 200 None 2026-04-03 02:52:29,517 - lava_callback - DEBUG - Processing node 69cf0fd49e2394c38fe0182a: baseline-arm64-broonie 2026-04-03 02:52:29,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:52:30,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:52:30,174 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd49e2394c38fe0182a/log.txt.gz 2026-04-03 02:52:30,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:52:30,643 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:52:30,646 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd49e2394c38fe0182a/lava-logs.yaml 2026-04-03 02:52:30,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:52:30,988 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:52:30,990 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fd49e2394c38fe0182a/lava_callback.json.gz 2026-04-03 02:52:31,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:52:31,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fd49e2394c38fe0182a HTTP/1.1" 200 None 2026-04-03 02:52:31,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:52:31,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:52:31,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:52:31,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fd49e2394c38fe0182a HTTP/1.1" 200 None 2026-04-03 02:52:31,171 - lava_callback - INFO - Completed processing callback for node 69cf0fd49e2394c38fe0182a INFO: 172.21.0.1:40152 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:52:42,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:52:42,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41502 - "GET /node/69294bd8a7fbd8958ccf1091%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-03 02:53:02,492 - lava_callback - INFO - Received callback for node 69cf2b069e2394c38fe02e20 INFO: 172.21.0.1:59496 - "POST /node/69cf2b069e2394c38fe02e20 HTTP/1.0" 202 Accepted 2026-04-03 02:53:02,499 - lava_callback - INFO - Starting async job submit for node 69cf2b069e2394c38fe02e20 2026-04-03 02:53:02,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:53:02,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2b069e2394c38fe02e20 HTTP/1.1" 200 None 2026-04-03 02:53:02,540 - lava_callback - DEBUG - Processing node 69cf2b069e2394c38fe02e20: baseline-arm64 2026-04-03 02:53:02,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:53:03,139 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:53:03,140 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf2b069e2394c38fe02e20/log.txt.gz 2026-04-03 02:53:03,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:53:03,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:53:03,649 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf2b069e2394c38fe02e20/lava-logs.yaml 2026-04-03 02:53:03,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 02:53:03,874 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 02:53:03,876 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf2b069e2394c38fe02e20/lava_callback.json.gz 2026-04-03 02:53:03,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:53:03,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2b069e2394c38fe02e20 HTTP/1.1" 200 None 2026-04-03 02:53:03,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:53:03,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 02:53:03,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:53:03,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf2b069e2394c38fe02e20 HTTP/1.1" 200 None 2026-04-03 02:53:04,046 - lava_callback - INFO - Completed processing callback for node 69cf2b069e2394c38fe02e20 INFO: 172.21.0.1:59510 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:53:12,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:53:12,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45720 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45722 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:54:17,139 - lava_callback - INFO - Received callback for node 69cc88d38cd9cbcdc945b1aa INFO: 172.21.0.1:45732 - "POST /node/69cc88d38cd9cbcdc945b1aa HTTP/1.0" 202 Accepted 2026-04-03 02:54:17,145 - lava_callback - INFO - Starting async job submit for node 69cc88d38cd9cbcdc945b1aa 2026-04-03 02:54:17,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:54:17,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc88d38cd9cbcdc945b1aa HTTP/1.1" 200 None 2026-04-03 02:54:17,166 - lava_callback - DEBUG - Processing node 69cc88d38cd9cbcdc945b1aa: baseline-arm-clabbe 2026-04-03 02:54:17,166 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc88d38cd9cbcdc945b1aa 2026-04-03 02:54:17,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:54:17,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc88d38cd9cbcdc945b1aa HTTP/1.1" 200 None 2026-04-03 02:54:17,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:54:17,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-03 02:54:17,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:54:17,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc88d38cd9cbcdc945b1aa HTTP/1.1" 200 None 2026-04-03 02:54:17,230 - lava_callback - ERROR - Error processing callback for node 69cc88d38cd9cbcdc945b1aa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48684 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:55:17,180 - lava_callback - INFO - Received callback for node 69cc88ff8cd9cbcdc945b1bc INFO: 172.21.0.1:48688 - "POST /node/69cc88ff8cd9cbcdc945b1bc HTTP/1.0" 202 Accepted 2026-04-03 02:55:17,186 - lava_callback - INFO - Starting async job submit for node 69cc88ff8cd9cbcdc945b1bc 2026-04-03 02:55:17,187 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:55:17,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc88ff8cd9cbcdc945b1bc HTTP/1.1" 200 None 2026-04-03 02:55:17,202 - lava_callback - DEBUG - Processing node 69cc88ff8cd9cbcdc945b1bc: baseline-arm-clabbe 2026-04-03 02:55:17,203 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc88ff8cd9cbcdc945b1bc 2026-04-03 02:55:17,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:55:17,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc88ff8cd9cbcdc945b1bc HTTP/1.1" 200 None 2026-04-03 02:55:17,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:55:17,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-03 02:55:17,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:55:17,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc88ff8cd9cbcdc945b1bc HTTP/1.1" 200 None 2026-04-03 02:55:17,260 - lava_callback - ERROR - Error processing callback for node 69cc88ff8cd9cbcdc945b1bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44810 - "GET /node/698e28e7952a14bf500e4028%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:44822 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:55:57,161 - lava_callback - INFO - Received callback for node 69cc892c8cd9cbcdc945b1c6 INFO: 172.21.0.1:44830 - "POST /node/69cc892c8cd9cbcdc945b1c6 HTTP/1.0" 202 Accepted 2026-04-03 02:55:57,167 - lava_callback - INFO - Starting async job submit for node 69cc892c8cd9cbcdc945b1c6 2026-04-03 02:55:57,167 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:55:57,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc892c8cd9cbcdc945b1c6 HTTP/1.1" 200 None 2026-04-03 02:55:57,182 - lava_callback - DEBUG - Processing node 69cc892c8cd9cbcdc945b1c6: baseline-arm-clabbe 2026-04-03 02:55:57,183 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc892c8cd9cbcdc945b1c6 2026-04-03 02:55:57,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:55:57,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc892c8cd9cbcdc945b1c6 HTTP/1.1" 200 None 2026-04-03 02:55:57,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:55:57,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-03 02:55:57,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:55:57,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc892c8cd9cbcdc945b1c6 HTTP/1.1" 200 None 2026-04-03 02:55:57,234 - lava_callback - ERROR - Error processing callback for node 69cc892c8cd9cbcdc945b1c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52516 - "GET /node/693b3ada4b47c3f87266d860%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45038 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:56:57,241 - lava_callback - INFO - Received callback for node 69cc89648cd9cbcdc945b1df INFO: 172.21.0.1:45042 - "POST /node/69cc89648cd9cbcdc945b1df HTTP/1.0" 202 Accepted 2026-04-03 02:56:57,246 - lava_callback - INFO - Starting async job submit for node 69cc89648cd9cbcdc945b1df 2026-04-03 02:56:57,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:56:57,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc89648cd9cbcdc945b1df HTTP/1.1" 200 None 2026-04-03 02:56:57,263 - lava_callback - DEBUG - Processing node 69cc89648cd9cbcdc945b1df: baseline-arm-clabbe 2026-04-03 02:56:57,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc89648cd9cbcdc945b1df 2026-04-03 02:56:57,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:56:57,277 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc89648cd9cbcdc945b1df HTTP/1.1" 200 None 2026-04-03 02:56:57,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:56:57,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-03 02:56:57,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:56:57,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc89648cd9cbcdc945b1df HTTP/1.1" 200 None 2026-04-03 02:56:57,314 - lava_callback - ERROR - Error processing callback for node 69cc89648cd9cbcdc945b1df: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32936 - "GET /node/693b0e36fee8383e92b7254d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:32952 - "GET /node/698ef6ddf0e3ffbba67d3258%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:40522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32974 - "GET /node/69a2aa5c8020dd22eabedf0d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:32976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46388 - "GET / HTTP/1.0" 200 OK 2026-04-03 02:58:57,280 - lava_callback - INFO - Received callback for node 69cc89e08cd9cbcdc945b219 INFO: 172.21.0.1:46396 - "POST /node/69cc89e08cd9cbcdc945b219 HTTP/1.0" 202 Accepted 2026-04-03 02:58:57,285 - lava_callback - INFO - Starting async job submit for node 69cc89e08cd9cbcdc945b219 2026-04-03 02:58:57,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:58:57,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc89e08cd9cbcdc945b219 HTTP/1.1" 200 None 2026-04-03 02:58:57,301 - lava_callback - DEBUG - Processing node 69cc89e08cd9cbcdc945b219: baseline-arm-clabbe 2026-04-03 02:58:57,301 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc89e08cd9cbcdc945b219 2026-04-03 02:58:57,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:58:57,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc89e08cd9cbcdc945b219 HTTP/1.1" 200 None 2026-04-03 02:58:57,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:58:57,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb83422a64fa9aff9368ff HTTP/1.1" 200 None 2026-04-03 02:58:57,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 02:58:57,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc89e08cd9cbcdc945b219 HTTP/1.1" 200 None 2026-04-03 02:58:57,356 - lava_callback - ERROR - Error processing callback for node 69cc89e08cd9cbcdc945b219: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53426 - "GET /node/6992691098a8bdb31c39230a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:53428 - "GET /node/69b7d2ce00b80c8ffdcb8a57%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:35662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58452 - "GET /node/693e7b8b3c8f1be6d3cf0b5f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39188 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:01:01,306 - lava_callback - INFO - Received callback for node 69cf2bb19e2394c38fe02ee1 INFO: 172.21.0.1:50404 - "POST /node/69cf2bb19e2394c38fe02ee1 HTTP/1.0" 202 Accepted 2026-04-03 03:01:01,314 - lava_callback - INFO - Starting async job submit for node 69cf2bb19e2394c38fe02ee1 2026-04-03 03:01:01,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:01:01,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2bb19e2394c38fe02ee1 HTTP/1.1" 200 None 2026-04-03 03:01:01,365 - lava_callback - DEBUG - Processing node 69cf2bb19e2394c38fe02ee1: baseline-arm64 2026-04-03 03:01:01,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:01:01,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:01:01,856 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf2bb19e2394c38fe02ee1/log.txt.gz 2026-04-03 03:01:02,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:01:02,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:01:02,330 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf2bb19e2394c38fe02ee1/lava-logs.yaml 2026-04-03 03:01:02,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:01:02,503 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:01:02,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf2bb19e2394c38fe02ee1/lava_callback.json.gz DEBUG: Setup failed for 69cf2bb19e2394c38fe02ee1. Transitting job node result: incomplete -> fail 2026-04-03 03:01:02,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:01:02,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2bb19e2394c38fe02ee1 HTTP/1.1" 200 None 2026-04-03 03:01:02,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:01:02,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 03:01:02,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:01:02,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf2bb19e2394c38fe02ee1 HTTP/1.1" 200 None 2026-04-03 03:01:02,741 - lava_callback - INFO - Completed processing callback for node 69cf2bb19e2394c38fe02ee1 INFO: 172.21.0.1:50420 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:01:12,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:01:12,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET /node/693b0e34fee8383e92b7253e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35506 - "GET /node/6988c5289fc35f969e1eea68%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:51530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42752 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:05:11,036 - lava_callback - INFO - Received callback for node 69cf2bf89e2394c38fe02f16 INFO: 172.21.0.1:57958 - "POST /node/69cf2bf89e2394c38fe02f16 HTTP/1.0" 202 Accepted 2026-04-03 03:05:11,042 - lava_callback - INFO - Starting async job submit for node 69cf2bf89e2394c38fe02f16 2026-04-03 03:05:11,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:05:11,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2bf89e2394c38fe02f16 HTTP/1.1" 200 None 2026-04-03 03:05:11,085 - lava_callback - DEBUG - Processing node 69cf2bf89e2394c38fe02f16: baseline-nfs-arm64 2026-04-03 03:05:11,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:05:11,891 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:05:11,893 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf2bf89e2394c38fe02f16/log.txt.gz 2026-04-03 03:05:12,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:05:12,411 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:05:12,413 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf2bf89e2394c38fe02f16/lava-logs.yaml 2026-04-03 03:05:12,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:05:12,590 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:05:12,592 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf2bf89e2394c38fe02f16/lava_callback.json.gz 2026-04-03 03:05:12,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:05:12,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2bf89e2394c38fe02f16 HTTP/1.1" 200 None 2026-04-03 03:05:12,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:05:12,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 03:05:12,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:05:12,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf2bf89e2394c38fe02f16 HTTP/1.1" 200 None 2026-04-03 03:05:12,746 - lava_callback - INFO - Completed processing callback for node 69cf2bf89e2394c38fe02f16 INFO: 172.21.0.1:57960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33590 - "GET /node/69116d54f15111fc8ce49d33%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:33592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50650 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:05:42,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:05:42,280 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36574 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:06:48,725 - lava_callback - INFO - Received callback for node 69cf2e6b9e2394c38fe031c6 INFO: 172.21.0.1:36576 - "POST /node/69cf2e6b9e2394c38fe031c6 HTTP/1.0" 202 Accepted 2026-04-03 03:06:48,732 - lava_callback - INFO - Starting async job submit for node 69cf2e6b9e2394c38fe031c6 2026-04-03 03:06:48,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:06:48,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2e6b9e2394c38fe031c6 HTTP/1.1" 200 None 2026-04-03 03:06:48,787 - lava_callback - DEBUG - Processing node 69cf2e6b9e2394c38fe031c6: baseline-arm64-broonie 2026-04-03 03:06:49,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:06:49,377 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:06:49,378 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf2e6b9e2394c38fe031c6/log.txt.gz 2026-04-03 03:06:49,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:06:49,800 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:06:49,802 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf2e6b9e2394c38fe031c6/lava-logs.yaml 2026-04-03 03:06:49,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:06:49,977 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:06:49,978 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf2e6b9e2394c38fe031c6/lava_callback.json.gz 2026-04-03 03:06:50,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:06:50,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2e6b9e2394c38fe031c6 HTTP/1.1" 200 None 2026-04-03 03:06:50,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:06:50,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 03:06:50,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:06:50,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf2e6b9e2394c38fe031c6 HTTP/1.1" 200 None 2026-04-03 03:06:50,059 - lava_callback - INFO - Completed processing callback for node 69cf2e6b9e2394c38fe031c6 INFO: 172.21.0.1:54446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58278 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:07:12,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:07:12,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46284 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49618 - "GET /node/698b6b750a16aea5a0dc7b9a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49622 - "GET /node/693af877fee8383e92b6c5d7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35024 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:08:29,913 - lava_callback - INFO - Received callback for node 69cf2cb69e2394c38fe02fd4 INFO: 172.21.0.1:46316 - "POST /node/69cf2cb69e2394c38fe02fd4 HTTP/1.0" 202 Accepted 2026-04-03 03:08:29,920 - lava_callback - INFO - Starting async job submit for node 69cf2cb69e2394c38fe02fd4 2026-04-03 03:08:29,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:08:29,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2cb69e2394c38fe02fd4 HTTP/1.1" 200 None 2026-04-03 03:08:29,966 - lava_callback - DEBUG - Processing node 69cf2cb69e2394c38fe02fd4: baseline-arm64 2026-04-03 03:08:30,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:08:30,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:08:30,789 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf2cb69e2394c38fe02fd4/log.txt.gz 2026-04-03 03:08:31,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:08:31,282 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:08:31,284 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf2cb69e2394c38fe02fd4/lava-logs.yaml 2026-04-03 03:08:31,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:08:31,469 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:08:31,471 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69cf2cb69e2394c38fe02fd4/lava_callback.json.gz 2026-04-03 03:08:31,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:08:31,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2cb69e2394c38fe02fd4 HTTP/1.1" 200 None 2026-04-03 03:08:31,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:08:31,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 03:08:31,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:08:31,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf2cb69e2394c38fe02fd4 HTTP/1.1" 200 None 2026-04-03 03:08:31,628 - lava_callback - INFO - Completed processing callback for node 69cf2cb69e2394c38fe02fd4 INFO: 172.21.0.1:46318 - "GET /node/69930b6531166481c8de2547%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:46334 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:08:42,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:08:42,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37028 - "GET /node/693b070efee8383e92b70a8c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:37042 - "GET /node/693771f80c55536e7dddab80%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:55340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35808 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:11:30,505 - lava_callback - INFO - Received callback for node 69cf2b529e2394c38fe02e79 INFO: 172.21.0.1:41050 - "POST /node/69cf2b529e2394c38fe02e79 HTTP/1.0" 202 Accepted 2026-04-03 03:11:30,556 - lava_callback - INFO - Starting async job submit for node 69cf2b529e2394c38fe02e79 2026-04-03 03:11:30,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:11:30,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2b529e2394c38fe02e79 HTTP/1.1" 200 None 2026-04-03 03:11:30,603 - lava_callback - DEBUG - Processing node 69cf2b529e2394c38fe02e79: baseline-arm-pengutronix 2026-04-03 03:11:30,715 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:11:30,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:11:30,858 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf2b529e2394c38fe02e79/log.txt.gz 2026-04-03 03:11:30,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:11:31,148 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:11:31,150 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf2b529e2394c38fe02e79/lava-logs.yaml 2026-04-03 03:11:31,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:11:31,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:11:31,288 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf2b529e2394c38fe02e79/lava_callback.json.gz DEBUG: Setup failed for 69cf2b529e2394c38fe02e79. Transitting job node result: incomplete -> fail 2026-04-03 03:11:31,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:11:31,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2b529e2394c38fe02e79 HTTP/1.1" 200 None 2026-04-03 03:11:31,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:11:31,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 03:11:31,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:11:31,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf2b529e2394c38fe02e79 HTTP/1.1" 200 None 2026-04-03 03:11:31,441 - lava_callback - INFO - Completed processing callback for node 69cf2b529e2394c38fe02e79 INFO: 172.21.0.1:41052 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:11:42,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:11:42,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59344 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:12:25,698 - lava_callback - INFO - Received callback for node 69cf2b6e9e2394c38fe02e7d INFO: 172.21.0.1:44152 - "POST /node/69cf2b6e9e2394c38fe02e7d HTTP/1.0" 202 Accepted 2026-04-03 03:12:25,706 - lava_callback - INFO - Starting async job submit for node 69cf2b6e9e2394c38fe02e7d 2026-04-03 03:12:25,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:12:25,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2b6e9e2394c38fe02e7d HTTP/1.1" 200 None 2026-04-03 03:12:25,765 - lava_callback - DEBUG - Processing node 69cf2b6e9e2394c38fe02e7d: baseline-nfs-arm64 INFO: 172.21.0.1:44166 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:12:27,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:12:27,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:12:27,253 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf2b6e9e2394c38fe02e7d/log.txt.gz 2026-04-03 03:12:27,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:12:28,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:12:28,129 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf2b6e9e2394c38fe02e7d/lava-logs.yaml 2026-04-03 03:12:28,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:12:28,345 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:12:28,347 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf2b6e9e2394c38fe02e7d/lava_callback.json.gz DEBUG: Setup failed for 69cf2b6e9e2394c38fe02e7d. Transitting job node result: incomplete -> fail 2026-04-03 03:12:28,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:12:28,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2b6e9e2394c38fe02e7d HTTP/1.1" 200 None 2026-04-03 03:12:28,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:12:28,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 03:12:28,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:12:28,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf2b6e9e2394c38fe02e7d HTTP/1.1" 200 None 2026-04-03 03:12:28,602 - lava_callback - INFO - Completed processing callback for node 69cf2b6e9e2394c38fe02e7d INFO: 172.21.0.1:41672 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:12:42,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:12:42,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43704 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:13:49,731 - lava_callback - INFO - Received callback for node 69cf0fcc9e2394c38fe0180e INFO: 172.21.0.1:56618 - "POST /node/69cf0fcc9e2394c38fe0180e HTTP/1.0" 202 Accepted 2026-04-03 03:13:49,738 - lava_callback - INFO - Starting async job submit for node 69cf0fcc9e2394c38fe0180e 2026-04-03 03:13:49,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:13:49,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fcc9e2394c38fe0180e HTTP/1.1" 200 None 2026-04-03 03:13:49,777 - lava_callback - DEBUG - Processing node 69cf0fcc9e2394c38fe0180e: baseline-arm64-broonie 2026-04-03 03:13:50,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:13:50,238 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:13:50,240 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fcc9e2394c38fe0180e/log.txt.gz 2026-04-03 03:13:50,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:13:50,613 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:13:50,615 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fcc9e2394c38fe0180e/lava-logs.yaml 2026-04-03 03:13:50,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:13:50,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:13:50,839 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf0fcc9e2394c38fe0180e/lava_callback.json.gz 2026-04-03 03:13:50,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:13:50,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf0fcc9e2394c38fe0180e HTTP/1.1" 200 None 2026-04-03 03:13:50,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:13:50,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 03:13:50,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:13:50,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf0fcc9e2394c38fe0180e HTTP/1.1" 200 None 2026-04-03 03:13:50,921 - lava_callback - INFO - Completed processing callback for node 69cf0fcc9e2394c38fe0180e INFO: 172.21.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43122 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:14:12,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:14:12,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42422 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43306 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:21:42,728 - lava_callback - INFO - Received callback for node 69cf2e169e2394c38fe0316f INFO: 172.21.0.1:45428 - "POST /node/69cf2e169e2394c38fe0316f HTTP/1.0" 202 Accepted 2026-04-03 03:21:42,735 - lava_callback - INFO - Starting async job submit for node 69cf2e169e2394c38fe0316f 2026-04-03 03:21:42,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:21:42,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2e169e2394c38fe0316f HTTP/1.1" 200 None 2026-04-03 03:21:42,811 - lava_callback - DEBUG - Processing node 69cf2e169e2394c38fe0316f: baseline-arm 2026-04-03 03:21:43,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:21:43,639 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:21:43,641 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69cf2e169e2394c38fe0316f/log.txt.gz 2026-04-03 03:21:44,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:21:44,261 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:21:44,263 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69cf2e169e2394c38fe0316f/lava-logs.yaml 2026-04-03 03:21:44,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:21:44,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:21:44,432 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69cf2e169e2394c38fe0316f/lava_callback.json.gz DEBUG: Setup failed for 69cf2e169e2394c38fe0316f. Transitting job node result: incomplete -> fail 2026-04-03 03:21:44,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:21:44,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2e169e2394c38fe0316f HTTP/1.1" 200 None 2026-04-03 03:21:44,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:21:44,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 03:21:44,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:21:44,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf2e169e2394c38fe0316f HTTP/1.1" 200 None 2026-04-03 03:21:44,705 - lava_callback - INFO - Completed processing callback for node 69cf2e169e2394c38fe0316f INFO: 172.21.0.1:45438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38464 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:22:12,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:22:12,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39286 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:28:22,596 - lava_callback - INFO - Received callback for node 69cc90c58cd9cbcdc945b465 INFO: 172.21.0.1:32830 - "POST /node/69cc90c58cd9cbcdc945b465 HTTP/1.0" 202 Accepted 2026-04-03 03:28:22,602 - lava_callback - INFO - Starting async job submit for node 69cc90c58cd9cbcdc945b465 2026-04-03 03:28:22,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:28:22,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc90c58cd9cbcdc945b465 HTTP/1.1" 200 None 2026-04-03 03:28:22,617 - lava_callback - DEBUG - Processing node 69cc90c58cd9cbcdc945b465: baseline-arm64-baylibre 2026-04-03 03:28:22,618 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc90c58cd9cbcdc945b465 2026-04-03 03:28:22,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:28:22,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc90c58cd9cbcdc945b465 HTTP/1.1" 200 None 2026-04-03 03:28:22,632 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:28:22,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:28:22,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:28:22,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc90c58cd9cbcdc945b465 HTTP/1.1" 200 None 2026-04-03 03:28:22,673 - lava_callback - ERROR - Error processing callback for node 69cc90c58cd9cbcdc945b465: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48716 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:31:10,975 - lava_callback - INFO - Received callback for node 69cf2ff09e2394c38fe03244 INFO: 172.21.0.1:58204 - "POST /node/69cf2ff09e2394c38fe03244 HTTP/1.0" 202 Accepted 2026-04-03 03:31:11,021 - lava_callback - INFO - Starting async job submit for node 69cf2ff09e2394c38fe03244 2026-04-03 03:31:11,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:31:11,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2ff09e2394c38fe03244 HTTP/1.1" 200 None 2026-04-03 03:31:11,066 - lava_callback - DEBUG - Processing node 69cf2ff09e2394c38fe03244: baseline-arm-pengutronix 2026-04-03 03:31:11,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:31:11,299 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:31:11,301 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf2ff09e2394c38fe03244/log.txt.gz 2026-04-03 03:31:11,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:31:11,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:31:11,530 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf2ff09e2394c38fe03244/lava-logs.yaml 2026-04-03 03:31:11,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:31:11,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:31:11,657 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69cf2ff09e2394c38fe03244/lava_callback.json.gz DEBUG: Setup failed for 69cf2ff09e2394c38fe03244. Transitting job node result: incomplete -> fail 2026-04-03 03:31:11,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:31:11,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf2ff09e2394c38fe03244 HTTP/1.1" 200 None 2026-04-03 03:31:11,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:31:11,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf06fe9e2394c38fe00f3f HTTP/1.1" 200 None 2026-04-03 03:31:11,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:31:11,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf2ff09e2394c38fe03244 HTTP/1.1" 200 None 2026-04-03 03:31:11,833 - lava_callback - INFO - Completed processing callback for node 69cf2ff09e2394c38fe03244 2026-04-03 03:31:12,427 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:31:12,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58226 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:31:22,763 - lava_callback - INFO - Received callback for node 69cc91778cd9cbcdc945b496 INFO: 172.21.0.1:53556 - "POST /node/69cc91778cd9cbcdc945b496 HTTP/1.0" 202 Accepted 2026-04-03 03:31:22,771 - lava_callback - INFO - Starting async job submit for node 69cc91778cd9cbcdc945b496 2026-04-03 03:31:22,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:31:22,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc91778cd9cbcdc945b496 HTTP/1.1" 200 None 2026-04-03 03:31:22,789 - lava_callback - DEBUG - Processing node 69cc91778cd9cbcdc945b496: baseline-arm64-baylibre 2026-04-03 03:31:22,789 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc91778cd9cbcdc945b496 2026-04-03 03:31:22,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:31:22,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc91778cd9cbcdc945b496 HTTP/1.1" 200 None 2026-04-03 03:31:22,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:31:22,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:31:22,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:31:22,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc91778cd9cbcdc945b496 HTTP/1.1" 200 None 2026-04-03 03:31:22,862 - lava_callback - ERROR - Error processing callback for node 69cc91778cd9cbcdc945b496: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46590 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:32:09,743 - lava_callback - INFO - Received callback for node 69cf14bd9e2394c38fe01c8a INFO: 172.21.0.1:44792 - "POST /node/69cf14bd9e2394c38fe01c8a HTTP/1.0" 202 Accepted 2026-04-03 03:32:09,749 - lava_callback - INFO - Starting async job submit for node 69cf14bd9e2394c38fe01c8a 2026-04-03 03:32:09,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:32:09,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf14bd9e2394c38fe01c8a HTTP/1.1" 200 None 2026-04-03 03:32:09,791 - lava_callback - DEBUG - Processing node 69cf14bd9e2394c38fe01c8a: baseline-arm64-broonie 2026-04-03 03:32:10,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:32:10,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:32:10,210 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf14bd9e2394c38fe01c8a/log.txt.gz 2026-04-03 03:32:10,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:32:10,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:32:10,769 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf14bd9e2394c38fe01c8a/lava-logs.yaml 2026-04-03 03:32:10,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:32:10,933 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:32:10,935 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69cf14bd9e2394c38fe01c8a/lava_callback.json.gz 2026-04-03 03:32:10,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:32:10,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf14bd9e2394c38fe01c8a HTTP/1.1" 200 None 2026-04-03 03:32:10,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:32:10,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 03:32:10,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:32:11,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf14bd9e2394c38fe01c8a HTTP/1.1" 200 None 2026-04-03 03:32:11,025 - lava_callback - INFO - Completed processing callback for node 69cf14bd9e2394c38fe01c8a 2026-04-03 03:32:12,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:32:12,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:44802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44806 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:32:26,223 - lava_callback - INFO - Received callback for node 69cf30249e2394c38fe0326b INFO: 172.21.0.1:57462 - "POST /node/69cf30249e2394c38fe0326b HTTP/1.0" 202 Accepted 2026-04-03 03:32:26,232 - lava_callback - INFO - Starting async job submit for node 69cf30249e2394c38fe0326b 2026-04-03 03:32:26,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:32:26,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf30249e2394c38fe0326b HTTP/1.1" 200 None 2026-04-03 03:32:26,304 - lava_callback - DEBUG - Processing node 69cf30249e2394c38fe0326b: baseline-nfs-arm64 INFO: 172.21.0.1:57468 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:32:27,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:32:27,719 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:32:27,721 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf30249e2394c38fe0326b/log.txt.gz 2026-04-03 03:32:28,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:32:28,695 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:32:28,698 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf30249e2394c38fe0326b/lava-logs.yaml 2026-04-03 03:32:28,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-03 03:32:28,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-03 03:32:28,900 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69cf30249e2394c38fe0326b/lava_callback.json.gz DEBUG: Setup failed for 69cf30249e2394c38fe0326b. Transitting job node result: incomplete -> fail 2026-04-03 03:32:29,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:32:29,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf30249e2394c38fe0326b HTTP/1.1" 200 None 2026-04-03 03:32:29,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:32:29,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cf070a9e2394c38fe00f4e HTTP/1.1" 200 None 2026-04-03 03:32:29,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:32:29,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cf30249e2394c38fe0326b HTTP/1.1" 200 None 2026-04-03 03:32:29,144 - lava_callback - INFO - Completed processing callback for node 69cf30249e2394c38fe0326b INFO: 172.21.0.1:51850 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:32:42,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:32:42,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50306 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:35:22,967 - lava_callback - INFO - Received callback for node 69cc92688cd9cbcdc945b500 INFO: 172.21.0.1:43574 - "POST /node/69cc92688cd9cbcdc945b500 HTTP/1.0" 202 Accepted 2026-04-03 03:35:22,975 - lava_callback - INFO - Starting async job submit for node 69cc92688cd9cbcdc945b500 2026-04-03 03:35:22,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:35:22,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc92688cd9cbcdc945b500 HTTP/1.1" 200 None 2026-04-03 03:35:22,992 - lava_callback - DEBUG - Processing node 69cc92688cd9cbcdc945b500: baseline-arm64-baylibre 2026-04-03 03:35:22,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc92688cd9cbcdc945b500 2026-04-03 03:35:22,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:35:23,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc92688cd9cbcdc945b500 HTTP/1.1" 200 None 2026-04-03 03:35:23,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:35:23,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:35:23,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:35:23,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc92688cd9cbcdc945b500 HTTP/1.1" 200 None 2026-04-03 03:35:23,075 - lava_callback - ERROR - Error processing callback for node 69cc92688cd9cbcdc945b500: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60208 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:38:23,105 - lava_callback - INFO - Received callback for node 69cc931d8cd9cbcdc945b535 INFO: 172.21.0.1:34960 - "POST /node/69cc931d8cd9cbcdc945b535 HTTP/1.0" 202 Accepted 2026-04-03 03:38:23,110 - lava_callback - INFO - Starting async job submit for node 69cc931d8cd9cbcdc945b535 2026-04-03 03:38:23,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:38:23,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc931d8cd9cbcdc945b535 HTTP/1.1" 200 None 2026-04-03 03:38:23,132 - lava_callback - DEBUG - Processing node 69cc931d8cd9cbcdc945b535: baseline-arm64-baylibre 2026-04-03 03:38:23,132 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc931d8cd9cbcdc945b535 2026-04-03 03:38:23,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:38:23,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc931d8cd9cbcdc945b535 HTTP/1.1" 200 None 2026-04-03 03:38:23,147 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:38:23,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:38:23,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:38:23,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc931d8cd9cbcdc945b535 HTTP/1.1" 200 None 2026-04-03 03:38:23,183 - lava_callback - ERROR - Error processing callback for node 69cc931d8cd9cbcdc945b535: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59986 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:39:43,164 - lava_callback - INFO - Received callback for node 69cc93758cd9cbcdc945b568 INFO: 172.21.0.1:38530 - "POST /node/69cc93758cd9cbcdc945b568 HTTP/1.0" 202 Accepted 2026-04-03 03:39:43,170 - lava_callback - INFO - Starting async job submit for node 69cc93758cd9cbcdc945b568 2026-04-03 03:39:43,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:39:43,185 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc93758cd9cbcdc945b568 HTTP/1.1" 200 None 2026-04-03 03:39:43,185 - lava_callback - DEBUG - Processing node 69cc93758cd9cbcdc945b568: baseline-arm64-baylibre 2026-04-03 03:39:43,186 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc93758cd9cbcdc945b568 2026-04-03 03:39:43,186 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:39:43,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc93758cd9cbcdc945b568 HTTP/1.1" 200 None 2026-04-03 03:39:43,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:39:43,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:39:43,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:39:43,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc93758cd9cbcdc945b568 HTTP/1.1" 200 None 2026-04-03 03:39:43,255 - lava_callback - ERROR - Error processing callback for node 69cc93758cd9cbcdc945b568: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51968 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:42:23,383 - lava_callback - INFO - Received callback for node 69cc940f8cd9cbcdc945b597 INFO: 172.21.0.1:33694 - "POST /node/69cc940f8cd9cbcdc945b597 HTTP/1.0" 202 Accepted 2026-04-03 03:42:23,388 - lava_callback - INFO - Starting async job submit for node 69cc940f8cd9cbcdc945b597 2026-04-03 03:42:23,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:42:23,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc940f8cd9cbcdc945b597 HTTP/1.1" 200 None 2026-04-03 03:42:23,404 - lava_callback - DEBUG - Processing node 69cc940f8cd9cbcdc945b597: baseline-arm64-baylibre 2026-04-03 03:42:23,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc940f8cd9cbcdc945b597 2026-04-03 03:42:23,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:42:23,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc940f8cd9cbcdc945b597 HTTP/1.1" 200 None 2026-04-03 03:42:23,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:42:23,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:42:23,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:42:23,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc940f8cd9cbcdc945b597 HTTP/1.1" 200 None 2026-04-03 03:42:23,483 - lava_callback - ERROR - Error processing callback for node 69cc940f8cd9cbcdc945b597: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57032 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:45:19,379 - lava_callback - INFO - Received callback for node 69cc94ca8cd9cbcdc945b5da INFO: 172.21.0.1:57044 - "POST /node/69cc94ca8cd9cbcdc945b5da HTTP/1.0" 202 Accepted 2026-04-03 03:45:19,385 - lava_callback - INFO - Starting async job submit for node 69cc94ca8cd9cbcdc945b5da 2026-04-03 03:45:19,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:45:19,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc94ca8cd9cbcdc945b5da HTTP/1.1" 200 None 2026-04-03 03:45:19,401 - lava_callback - DEBUG - Processing node 69cc94ca8cd9cbcdc945b5da: baseline-arm64-clabbe 2026-04-03 03:45:19,401 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc94ca8cd9cbcdc945b5da 2026-04-03 03:45:19,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:45:19,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc94ca8cd9cbcdc945b5da HTTP/1.1" 200 None 2026-04-03 03:45:19,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:45:19,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:45:19,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:45:19,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc94ca8cd9cbcdc945b5da HTTP/1.1" 200 None 2026-04-03 03:45:19,467 - lava_callback - ERROR - Error processing callback for node 69cc94ca8cd9cbcdc945b5da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55746 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:48:19,486 - lava_callback - INFO - Received callback for node 69cc957e8cd9cbcdc945b630 INFO: 172.21.0.1:55758 - "POST /node/69cc957e8cd9cbcdc945b630 HTTP/1.0" 202 Accepted 2026-04-03 03:48:19,493 - lava_callback - INFO - Starting async job submit for node 69cc957e8cd9cbcdc945b630 2026-04-03 03:48:19,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:48:19,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc957e8cd9cbcdc945b630 HTTP/1.1" 200 None 2026-04-03 03:48:19,510 - lava_callback - DEBUG - Processing node 69cc957e8cd9cbcdc945b630: baseline-arm64-clabbe 2026-04-03 03:48:19,511 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc957e8cd9cbcdc945b630 2026-04-03 03:48:19,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:48:19,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc957e8cd9cbcdc945b630 HTTP/1.1" 200 None 2026-04-03 03:48:19,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:48:19,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:48:19,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:48:19,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc957e8cd9cbcdc945b630 HTTP/1.1" 200 None 2026-04-03 03:48:19,576 - lava_callback - ERROR - Error processing callback for node 69cc957e8cd9cbcdc945b630: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39634 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:50:19,563 - lava_callback - INFO - Received callback for node 69cc95f78cd9cbcdc945b65a INFO: 172.21.0.1:39640 - "POST /node/69cc95f78cd9cbcdc945b65a HTTP/1.0" 202 Accepted 2026-04-03 03:50:19,568 - lava_callback - INFO - Starting async job submit for node 69cc95f78cd9cbcdc945b65a 2026-04-03 03:50:19,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:50:19,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc95f78cd9cbcdc945b65a HTTP/1.1" 200 None 2026-04-03 03:50:19,585 - lava_callback - DEBUG - Processing node 69cc95f78cd9cbcdc945b65a: baseline-arm64-clabbe 2026-04-03 03:50:19,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc95f78cd9cbcdc945b65a 2026-04-03 03:50:19,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:50:19,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc95f78cd9cbcdc945b65a HTTP/1.1" 200 None 2026-04-03 03:50:19,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:50:19,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:50:19,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:50:19,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc95f78cd9cbcdc945b65a HTTP/1.1" 200 None 2026-04-03 03:50:19,658 - lava_callback - ERROR - Error processing callback for node 69cc95f78cd9cbcdc945b65a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44290 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:51:59,594 - lava_callback - INFO - Received callback for node 69cc964d8cd9cbcdc945b69b INFO: 172.21.0.1:44294 - "POST /node/69cc964d8cd9cbcdc945b69b HTTP/1.0" 202 Accepted 2026-04-03 03:51:59,600 - lava_callback - INFO - Starting async job submit for node 69cc964d8cd9cbcdc945b69b 2026-04-03 03:51:59,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:51:59,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc964d8cd9cbcdc945b69b HTTP/1.1" 200 None 2026-04-03 03:51:59,616 - lava_callback - DEBUG - Processing node 69cc964d8cd9cbcdc945b69b: baseline-arm64-clabbe 2026-04-03 03:51:59,617 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc964d8cd9cbcdc945b69b 2026-04-03 03:51:59,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:51:59,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc964d8cd9cbcdc945b69b HTTP/1.1" 200 None 2026-04-03 03:51:59,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:51:59,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:51:59,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:51:59,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc964d8cd9cbcdc945b69b HTTP/1.1" 200 None 2026-04-03 03:51:59,666 - lava_callback - ERROR - Error processing callback for node 69cc964d8cd9cbcdc945b69b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41142 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:53:19,666 - lava_callback - INFO - Received callback for node 69cc96aa8cd9cbcdc945b6dc INFO: 172.21.0.1:41154 - "POST /node/69cc96aa8cd9cbcdc945b6dc HTTP/1.0" 202 Accepted 2026-04-03 03:53:19,671 - lava_callback - INFO - Starting async job submit for node 69cc96aa8cd9cbcdc945b6dc 2026-04-03 03:53:19,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:53:19,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc96aa8cd9cbcdc945b6dc HTTP/1.1" 200 None 2026-04-03 03:53:19,687 - lava_callback - DEBUG - Processing node 69cc96aa8cd9cbcdc945b6dc: baseline-arm64-clabbe 2026-04-03 03:53:19,687 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc96aa8cd9cbcdc945b6dc 2026-04-03 03:53:19,688 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:53:19,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc96aa8cd9cbcdc945b6dc HTTP/1.1" 200 None 2026-04-03 03:53:19,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:53:19,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:53:19,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:53:19,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc96aa8cd9cbcdc945b6dc HTTP/1.1" 200 None 2026-04-03 03:53:19,768 - lava_callback - ERROR - Error processing callback for node 69cc96aa8cd9cbcdc945b6dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49738 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41872 - "GET / HTTP/1.0" 200 OK 2026-04-03 03:54:59,740 - lava_callback - INFO - Received callback for node 69cc96fe8cd9cbcdc945b706 INFO: 172.21.0.1:46272 - "POST /node/69cc96fe8cd9cbcdc945b706 HTTP/1.0" 202 Accepted 2026-04-03 03:54:59,745 - lava_callback - INFO - Starting async job submit for node 69cc96fe8cd9cbcdc945b706 2026-04-03 03:54:59,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:54:59,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc96fe8cd9cbcdc945b706 HTTP/1.1" 200 None 2026-04-03 03:54:59,761 - lava_callback - DEBUG - Processing node 69cc96fe8cd9cbcdc945b706: baseline-arm64-clabbe 2026-04-03 03:54:59,762 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cc96fe8cd9cbcdc945b706 2026-04-03 03:54:59,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:54:59,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc96fe8cd9cbcdc945b706 HTTP/1.1" 200 None 2026-04-03 03:54:59,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:54:59,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb834d2a64fa9aff936913 HTTP/1.1" 200 None 2026-04-03 03:54:59,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 03:54:59,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cc96fe8cd9cbcdc945b706 HTTP/1.1" 200 None 2026-04-03 03:54:59,828 - lava_callback - ERROR - Error processing callback for node 69cc96fe8cd9cbcdc945b706: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45388 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50744 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39306 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56874 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48880 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49982 - "GET /node/698ff0ff76d4b6342e9db6ef%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49992 - "GET /node/698bd04b1e517752bab8ad96%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59026 - "GET /node/68fb3d215818c913a1e19a32%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:60370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43734 - "GET /node/6982adaa3507dcddca5909f4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:43748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32880 - "GET /node/697f204cad063048cae7c2f9%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:32896 - "GET /node/69937284cebe99dd8e069981%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58434 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60980 - "GET /node/693b43074b47c3f87266edf4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55408 - "GET /node/69936c59cebe99dd8e06786a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:38722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44278 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:44:06,818 - lava_callback - INFO - Received callback for node 69cca2908cd9cbcdc945b76a INFO: 172.21.0.1:44280 - "POST /node/69cca2908cd9cbcdc945b76a HTTP/1.0" 202 Accepted 2026-04-03 04:44:06,823 - lava_callback - INFO - Starting async job submit for node 69cca2908cd9cbcdc945b76a 2026-04-03 04:44:06,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:44:06,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca2908cd9cbcdc945b76a HTTP/1.1" 200 None 2026-04-03 04:44:06,840 - lava_callback - DEBUG - Processing node 69cca2908cd9cbcdc945b76a: baseline-arm-baylibre 2026-04-03 04:44:06,840 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca2908cd9cbcdc945b76a 2026-04-03 04:44:06,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:44:06,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca2908cd9cbcdc945b76a HTTP/1.1" 200 None 2026-04-03 04:44:06,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:44:06,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:44:06,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:44:06,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca2908cd9cbcdc945b76a HTTP/1.1" 200 None 2026-04-03 04:44:06,929 - lava_callback - ERROR - Error processing callback for node 69cca2908cd9cbcdc945b76a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54524 - "GET /node/6919c4eb40db568ee36c725c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:54536 - "GET /node/69a23a5990d011217c9f7f9b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:54544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54348 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:45:06,774 - lava_callback - INFO - Received callback for node 69cca2c78cd9cbcdc945b784 INFO: 172.21.0.1:54352 - "POST /node/69cca2c78cd9cbcdc945b784 HTTP/1.0" 202 Accepted 2026-04-03 04:45:06,780 - lava_callback - INFO - Starting async job submit for node 69cca2c78cd9cbcdc945b784 2026-04-03 04:45:06,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:45:06,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca2c78cd9cbcdc945b784 HTTP/1.1" 200 None 2026-04-03 04:45:06,797 - lava_callback - DEBUG - Processing node 69cca2c78cd9cbcdc945b784: baseline-arm-baylibre 2026-04-03 04:45:06,798 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca2c78cd9cbcdc945b784 2026-04-03 04:45:06,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:45:06,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca2c78cd9cbcdc945b784 HTTP/1.1" 200 None 2026-04-03 04:45:06,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:45:06,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:45:06,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:45:06,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca2c78cd9cbcdc945b784 HTTP/1.1" 200 None 2026-04-03 04:45:06,860 - lava_callback - ERROR - Error processing callback for node 69cca2c78cd9cbcdc945b784: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:45:46,825 - lava_callback - INFO - Received callback for node 69cca2f28cd9cbcdc945b79b INFO: 172.21.0.1:48304 - "POST /node/69cca2f28cd9cbcdc945b79b HTTP/1.0" 202 Accepted 2026-04-03 04:45:46,831 - lava_callback - INFO - Starting async job submit for node 69cca2f28cd9cbcdc945b79b 2026-04-03 04:45:46,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:45:46,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca2f28cd9cbcdc945b79b HTTP/1.1" 200 None 2026-04-03 04:45:46,848 - lava_callback - DEBUG - Processing node 69cca2f28cd9cbcdc945b79b: baseline-arm-baylibre 2026-04-03 04:45:46,848 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca2f28cd9cbcdc945b79b 2026-04-03 04:45:46,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:45:46,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca2f28cd9cbcdc945b79b HTTP/1.1" 200 None 2026-04-03 04:45:46,876 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:45:46,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:45:46,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:45:46,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca2f28cd9cbcdc945b79b HTTP/1.1" 200 None 2026-04-03 04:45:46,916 - lava_callback - ERROR - Error processing callback for node 69cca2f28cd9cbcdc945b79b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55906 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:46:26,849 - lava_callback - INFO - Received callback for node 69cca31c8cd9cbcdc945b7b1 INFO: 172.21.0.1:55908 - "POST /node/69cca31c8cd9cbcdc945b7b1 HTTP/1.0" 202 Accepted 2026-04-03 04:46:26,854 - lava_callback - INFO - Starting async job submit for node 69cca31c8cd9cbcdc945b7b1 2026-04-03 04:46:26,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:46:26,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca31c8cd9cbcdc945b7b1 HTTP/1.1" 200 None 2026-04-03 04:46:26,870 - lava_callback - DEBUG - Processing node 69cca31c8cd9cbcdc945b7b1: baseline-arm-baylibre 2026-04-03 04:46:26,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca31c8cd9cbcdc945b7b1 2026-04-03 04:46:26,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:46:26,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca31c8cd9cbcdc945b7b1 HTTP/1.1" 200 None 2026-04-03 04:46:26,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:46:26,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:46:26,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:46:26,927 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca31c8cd9cbcdc945b7b1 HTTP/1.1" 200 None 2026-04-03 04:46:26,938 - lava_callback - ERROR - Error processing callback for node 69cca31c8cd9cbcdc945b7b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60536 - "GET /node/693af763fee8383e92b6c03a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44206 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:47:06,924 - lava_callback - INFO - Received callback for node 69cca3458cd9cbcdc945b7c1 INFO: 172.21.0.1:44222 - "POST /node/69cca3458cd9cbcdc945b7c1 HTTP/1.0" 202 Accepted 2026-04-03 04:47:06,930 - lava_callback - INFO - Starting async job submit for node 69cca3458cd9cbcdc945b7c1 2026-04-03 04:47:06,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:47:06,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca3458cd9cbcdc945b7c1 HTTP/1.1" 200 None 2026-04-03 04:47:06,946 - lava_callback - DEBUG - Processing node 69cca3458cd9cbcdc945b7c1: baseline-arm-baylibre 2026-04-03 04:47:06,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca3458cd9cbcdc945b7c1 2026-04-03 04:47:06,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:47:06,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca3458cd9cbcdc945b7c1 HTTP/1.1" 200 None 2026-04-03 04:47:06,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:47:06,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:47:06,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:47:06,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca3458cd9cbcdc945b7c1 HTTP/1.1" 200 None 2026-04-03 04:47:07,003 - lava_callback - ERROR - Error processing callback for node 69cca3458cd9cbcdc945b7c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47768 - "GET /node/693b070cfee8383e92b70a72%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48762 - "GET /node/694afdf26c96ed51d3172c41%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60094 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45802 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:48:27,030 - lava_callback - INFO - Received callback for node 69cca3938cd9cbcdc945b7e3 INFO: 172.21.0.1:45810 - "POST /node/69cca3938cd9cbcdc945b7e3 HTTP/1.0" 202 Accepted 2026-04-03 04:48:27,036 - lava_callback - INFO - Starting async job submit for node 69cca3938cd9cbcdc945b7e3 2026-04-03 04:48:27,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:48:27,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca3938cd9cbcdc945b7e3 HTTP/1.1" 200 None 2026-04-03 04:48:27,063 - lava_callback - DEBUG - Processing node 69cca3938cd9cbcdc945b7e3: baseline-arm-baylibre 2026-04-03 04:48:27,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca3938cd9cbcdc945b7e3 2026-04-03 04:48:27,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:48:27,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca3938cd9cbcdc945b7e3 HTTP/1.1" 200 None 2026-04-03 04:48:27,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:48:27,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:48:27,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:48:27,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca3938cd9cbcdc945b7e3 HTTP/1.1" 200 None 2026-04-03 04:48:27,121 - lava_callback - ERROR - Error processing callback for node 69cca3938cd9cbcdc945b7e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52892 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:49:07,000 - lava_callback - INFO - Received callback for node 69cca3bd8cd9cbcdc945b7ff INFO: 172.21.0.1:52904 - "POST /node/69cca3bd8cd9cbcdc945b7ff HTTP/1.0" 202 Accepted 2026-04-03 04:49:07,006 - lava_callback - INFO - Starting async job submit for node 69cca3bd8cd9cbcdc945b7ff 2026-04-03 04:49:07,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:49:07,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca3bd8cd9cbcdc945b7ff HTTP/1.1" 200 None 2026-04-03 04:49:07,022 - lava_callback - DEBUG - Processing node 69cca3bd8cd9cbcdc945b7ff: baseline-arm-baylibre 2026-04-03 04:49:07,022 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca3bd8cd9cbcdc945b7ff 2026-04-03 04:49:07,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:49:07,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca3bd8cd9cbcdc945b7ff HTTP/1.1" 200 None 2026-04-03 04:49:07,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:49:07,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:49:07,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:49:07,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca3bd8cd9cbcdc945b7ff HTTP/1.1" 200 None 2026-04-03 04:49:07,075 - lava_callback - ERROR - Error processing callback for node 69cca3bd8cd9cbcdc945b7ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49572 - "GET /node/699381d9cebe99dd8e06e4d4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50386 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:49:47,054 - lava_callback - INFO - Received callback for node 69cca3e48cd9cbcdc945b812 INFO: 172.21.0.1:50400 - "POST /node/69cca3e48cd9cbcdc945b812 HTTP/1.0" 202 Accepted 2026-04-03 04:49:47,061 - lava_callback - INFO - Starting async job submit for node 69cca3e48cd9cbcdc945b812 2026-04-03 04:49:47,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:49:47,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca3e48cd9cbcdc945b812 HTTP/1.1" 200 None 2026-04-03 04:49:47,078 - lava_callback - DEBUG - Processing node 69cca3e48cd9cbcdc945b812: baseline-arm-baylibre 2026-04-03 04:49:47,078 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca3e48cd9cbcdc945b812 2026-04-03 04:49:47,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:49:47,094 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca3e48cd9cbcdc945b812 HTTP/1.1" 200 None 2026-04-03 04:49:47,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:49:47,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:49:47,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:49:47,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca3e48cd9cbcdc945b812 HTTP/1.1" 200 None 2026-04-03 04:49:47,149 - lava_callback - ERROR - Error processing callback for node 69cca3e48cd9cbcdc945b812: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47834 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:50:47,086 - lava_callback - INFO - Received callback for node 69cca4128cd9cbcdc945b827 INFO: 172.21.0.1:47836 - "POST /node/69cca4128cd9cbcdc945b827 HTTP/1.0" 202 Accepted 2026-04-03 04:50:47,092 - lava_callback - INFO - Starting async job submit for node 69cca4128cd9cbcdc945b827 2026-04-03 04:50:47,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:50:47,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca4128cd9cbcdc945b827 HTTP/1.1" 200 None 2026-04-03 04:50:47,108 - lava_callback - DEBUG - Processing node 69cca4128cd9cbcdc945b827: baseline-arm-baylibre 2026-04-03 04:50:47,108 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca4128cd9cbcdc945b827 2026-04-03 04:50:47,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:50:47,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca4128cd9cbcdc945b827 HTTP/1.1" 200 None 2026-04-03 04:50:47,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:50:47,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:50:47,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:50:47,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca4128cd9cbcdc945b827 HTTP/1.1" 200 None 2026-04-03 04:50:47,170 - lava_callback - ERROR - Error processing callback for node 69cca4128cd9cbcdc945b827: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39498 - "GET /node/69280d9cb1b5e06bf6c04d2d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38674 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:51:27,117 - lava_callback - INFO - Received callback for node 69cca43b8cd9cbcdc945b83c INFO: 172.21.0.1:38678 - "POST /node/69cca43b8cd9cbcdc945b83c HTTP/1.0" 202 Accepted 2026-04-03 04:51:27,123 - lava_callback - INFO - Starting async job submit for node 69cca43b8cd9cbcdc945b83c 2026-04-03 04:51:27,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:51:27,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca43b8cd9cbcdc945b83c HTTP/1.1" 200 None 2026-04-03 04:51:27,138 - lava_callback - DEBUG - Processing node 69cca43b8cd9cbcdc945b83c: baseline-arm-baylibre 2026-04-03 04:51:27,139 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca43b8cd9cbcdc945b83c 2026-04-03 04:51:27,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:51:27,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca43b8cd9cbcdc945b83c HTTP/1.1" 200 None 2026-04-03 04:51:27,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:51:27,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:51:27,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:51:27,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca43b8cd9cbcdc945b83c HTTP/1.1" 200 None 2026-04-03 04:51:27,190 - lava_callback - ERROR - Error processing callback for node 69cca43b8cd9cbcdc945b83c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34980 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:54:02,549 - lava_callback - INFO - Received callback for node 69cca4df8cd9cbcdc945b891 INFO: 172.21.0.1:35978 - "POST /node/69cca4df8cd9cbcdc945b891 HTTP/1.0" 202 Accepted 2026-04-03 04:54:02,554 - lava_callback - INFO - Starting async job submit for node 69cca4df8cd9cbcdc945b891 2026-04-03 04:54:02,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:54:02,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca4df8cd9cbcdc945b891 HTTP/1.1" 200 None 2026-04-03 04:54:02,574 - lava_callback - DEBUG - Processing node 69cca4df8cd9cbcdc945b891: baseline-arm-clabbe 2026-04-03 04:54:02,574 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca4df8cd9cbcdc945b891 2026-04-03 04:54:02,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:54:02,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca4df8cd9cbcdc945b891 HTTP/1.1" 200 None 2026-04-03 04:54:02,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:54:02,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:54:02,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:54:02,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca4df8cd9cbcdc945b891 HTTP/1.1" 200 None 2026-04-03 04:54:02,626 - lava_callback - ERROR - Error processing callback for node 69cca4df8cd9cbcdc945b891: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56870 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:54:42,300 - lava_callback - INFO - Received callback for node 69cca5088cd9cbcdc945b8a2 INFO: 172.21.0.1:49216 - "POST /node/69cca5088cd9cbcdc945b8a2 HTTP/1.0" 202 Accepted 2026-04-03 04:54:42,305 - lava_callback - INFO - Starting async job submit for node 69cca5088cd9cbcdc945b8a2 2026-04-03 04:54:42,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:54:42,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca5088cd9cbcdc945b8a2 HTTP/1.1" 200 None 2026-04-03 04:54:42,322 - lava_callback - DEBUG - Processing node 69cca5088cd9cbcdc945b8a2: baseline-arm-clabbe 2026-04-03 04:54:42,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca5088cd9cbcdc945b8a2 2026-04-03 04:54:42,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:54:42,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca5088cd9cbcdc945b8a2 HTTP/1.1" 200 None 2026-04-03 04:54:42,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:54:42,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:54:42,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:54:42,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca5088cd9cbcdc945b8a2 HTTP/1.1" 200 None 2026-04-03 04:54:42,372 - lava_callback - ERROR - Error processing callback for node 69cca5088cd9cbcdc945b8a2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34416 - "GET /node/693af70afee8383e92b6be7d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:34428 - "GET /node/69290321a7fbd8958ccf00bb%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:34444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:56:02,414 - lava_callback - INFO - Received callback for node 69cca5588cd9cbcdc945b8c8 INFO: 172.21.0.1:42628 - "POST /node/69cca5588cd9cbcdc945b8c8 HTTP/1.0" 202 Accepted 2026-04-03 04:56:02,420 - lava_callback - INFO - Starting async job submit for node 69cca5588cd9cbcdc945b8c8 2026-04-03 04:56:02,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:56:02,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca5588cd9cbcdc945b8c8 HTTP/1.1" 200 None 2026-04-03 04:56:02,435 - lava_callback - DEBUG - Processing node 69cca5588cd9cbcdc945b8c8: baseline-arm-clabbe 2026-04-03 04:56:02,436 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca5588cd9cbcdc945b8c8 2026-04-03 04:56:02,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:56:02,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca5588cd9cbcdc945b8c8 HTTP/1.1" 200 None 2026-04-03 04:56:02,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:56:02,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:56:02,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:56:02,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca5588cd9cbcdc945b8c8 HTTP/1.1" 200 None 2026-04-03 04:56:02,490 - lava_callback - ERROR - Error processing callback for node 69cca5588cd9cbcdc945b8c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41332 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:56:47,565 - lava_callback - INFO - Received callback for node 69cca57d8cd9cbcdc945b8e0 INFO: 172.21.0.1:41340 - "POST /node/69cca57d8cd9cbcdc945b8e0 HTTP/1.0" 202 Accepted 2026-04-03 04:56:47,571 - lava_callback - INFO - Starting async job submit for node 69cca57d8cd9cbcdc945b8e0 2026-04-03 04:56:47,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:56:47,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca57d8cd9cbcdc945b8e0 HTTP/1.1" 200 None 2026-04-03 04:56:47,604 - lava_callback - DEBUG - Processing node 69cca57d8cd9cbcdc945b8e0: baseline-arm-baylibre 2026-04-03 04:56:47,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca57d8cd9cbcdc945b8e0 2026-04-03 04:56:47,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:56:47,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca57d8cd9cbcdc945b8e0 HTTP/1.1" 200 None 2026-04-03 04:56:47,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:56:47,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:56:47,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:56:47,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca57d8cd9cbcdc945b8e0 HTTP/1.1" 200 None 2026-04-03 04:56:47,661 - lava_callback - ERROR - Error processing callback for node 69cca57d8cd9cbcdc945b8e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36408 - "GET /node/693af843fee8383e92b6c4d4%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-03 04:57:22,436 - lava_callback - INFO - Received callback for node 69cca5a78cd9cbcdc945b8f7 INFO: 172.21.0.1:36410 - "POST /node/69cca5a78cd9cbcdc945b8f7 HTTP/1.0" 202 Accepted 2026-04-03 04:57:22,442 - lava_callback - INFO - Starting async job submit for node 69cca5a78cd9cbcdc945b8f7 2026-04-03 04:57:22,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:57:22,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca5a78cd9cbcdc945b8f7 HTTP/1.1" 200 None 2026-04-03 04:57:22,463 - lava_callback - DEBUG - Processing node 69cca5a78cd9cbcdc945b8f7: baseline-arm-clabbe 2026-04-03 04:57:22,463 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca5a78cd9cbcdc945b8f7 2026-04-03 04:57:22,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:57:22,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca5a78cd9cbcdc945b8f7 HTTP/1.1" 200 None 2026-04-03 04:57:22,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:57:22,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:57:22,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:57:22,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca5a78cd9cbcdc945b8f7 HTTP/1.1" 200 None 2026-04-03 04:57:22,520 - lava_callback - ERROR - Error processing callback for node 69cca5a78cd9cbcdc945b8f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37206 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:58:02,411 - lava_callback - INFO - Received callback for node 69cca5d18cd9cbcdc945b909 INFO: 172.21.0.1:60432 - "POST /node/69cca5d18cd9cbcdc945b909 HTTP/1.0" 202 Accepted 2026-04-03 04:58:02,419 - lava_callback - INFO - Starting async job submit for node 69cca5d18cd9cbcdc945b909 2026-04-03 04:58:02,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:58:02,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca5d18cd9cbcdc945b909 HTTP/1.1" 200 None 2026-04-03 04:58:02,435 - lava_callback - DEBUG - Processing node 69cca5d18cd9cbcdc945b909: baseline-arm-clabbe 2026-04-03 04:58:02,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca5d18cd9cbcdc945b909 2026-04-03 04:58:02,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:58:02,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca5d18cd9cbcdc945b909 HTTP/1.1" 200 None 2026-04-03 04:58:02,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:58:02,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:58:02,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:58:02,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca5d18cd9cbcdc945b909 HTTP/1.1" 200 None 2026-04-03 04:58:02,496 - lava_callback - ERROR - Error processing callback for node 69cca5d18cd9cbcdc945b909: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48910 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:58:42,512 - lava_callback - INFO - Received callback for node 69cca5f98cd9cbcdc945b91a INFO: 172.21.0.1:40546 - "POST /node/69cca5f98cd9cbcdc945b91a HTTP/1.0" 202 Accepted 2026-04-03 04:58:42,518 - lava_callback - INFO - Starting async job submit for node 69cca5f98cd9cbcdc945b91a 2026-04-03 04:58:42,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:58:42,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca5f98cd9cbcdc945b91a HTTP/1.1" 200 None 2026-04-03 04:58:42,540 - lava_callback - DEBUG - Processing node 69cca5f98cd9cbcdc945b91a: baseline-arm-clabbe 2026-04-03 04:58:42,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca5f98cd9cbcdc945b91a 2026-04-03 04:58:42,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:58:42,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca5f98cd9cbcdc945b91a HTTP/1.1" 200 None 2026-04-03 04:58:42,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:58:42,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:58:42,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:58:42,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca5f98cd9cbcdc945b91a HTTP/1.1" 200 None 2026-04-03 04:58:42,622 - lava_callback - ERROR - Error processing callback for node 69cca5f98cd9cbcdc945b91a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58234 - "GET /node/698a8e2ed9a1cfa777a241e6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:58240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56486 - "GET / HTTP/1.0" 200 OK 2026-04-03 04:59:22,483 - lava_callback - INFO - Received callback for node 69cca6218cd9cbcdc945b92c INFO: 172.21.0.1:38358 - "POST /node/69cca6218cd9cbcdc945b92c HTTP/1.0" 202 Accepted 2026-04-03 04:59:22,488 - lava_callback - INFO - Starting async job submit for node 69cca6218cd9cbcdc945b92c 2026-04-03 04:59:22,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:59:22,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca6218cd9cbcdc945b92c HTTP/1.1" 200 None 2026-04-03 04:59:22,504 - lava_callback - DEBUG - Processing node 69cca6218cd9cbcdc945b92c: baseline-arm-clabbe 2026-04-03 04:59:22,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca6218cd9cbcdc945b92c 2026-04-03 04:59:22,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:59:22,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca6218cd9cbcdc945b92c HTTP/1.1" 200 None 2026-04-03 04:59:22,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:59:22,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 04:59:22,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 04:59:22,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca6218cd9cbcdc945b92c HTTP/1.1" 200 None 2026-04-03 04:59:22,567 - lava_callback - ERROR - Error processing callback for node 69cca6218cd9cbcdc945b92c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60452 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:00:02,668 - lava_callback - INFO - Received callback for node 69cca64b8cd9cbcdc945b93c INFO: 172.21.0.1:33814 - "POST /node/69cca64b8cd9cbcdc945b93c HTTP/1.0" 202 Accepted 2026-04-03 05:00:02,673 - lava_callback - INFO - Starting async job submit for node 69cca64b8cd9cbcdc945b93c 2026-04-03 05:00:02,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:00:02,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca64b8cd9cbcdc945b93c HTTP/1.1" 200 None 2026-04-03 05:00:02,690 - lava_callback - DEBUG - Processing node 69cca64b8cd9cbcdc945b93c: baseline-arm-clabbe 2026-04-03 05:00:02,691 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca64b8cd9cbcdc945b93c 2026-04-03 05:00:02,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:00:02,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca64b8cd9cbcdc945b93c HTTP/1.1" 200 None 2026-04-03 05:00:02,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:00:02,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 05:00:02,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:00:02,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca64b8cd9cbcdc945b93c HTTP/1.1" 200 None 2026-04-03 05:00:02,767 - lava_callback - ERROR - Error processing callback for node 69cca64b8cd9cbcdc945b93c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35602 - "GET /node/6983eea0969566dcb0d2a109%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:35618 - "GET /node/693b094bfee8383e92b7161a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:47432 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:00:42,525 - lava_callback - INFO - Received callback for node 69cca6748cd9cbcdc945b954 INFO: 172.21.0.1:48128 - "POST /node/69cca6748cd9cbcdc945b954 HTTP/1.0" 202 Accepted 2026-04-03 05:00:42,530 - lava_callback - INFO - Starting async job submit for node 69cca6748cd9cbcdc945b954 2026-04-03 05:00:42,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:00:42,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca6748cd9cbcdc945b954 HTTP/1.1" 200 None 2026-04-03 05:00:42,546 - lava_callback - DEBUG - Processing node 69cca6748cd9cbcdc945b954: baseline-arm-clabbe 2026-04-03 05:00:42,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca6748cd9cbcdc945b954 2026-04-03 05:00:42,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:00:42,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca6748cd9cbcdc945b954 HTTP/1.1" 200 None 2026-04-03 05:00:42,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:00:42,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 05:00:42,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:00:42,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca6748cd9cbcdc945b954 HTTP/1.1" 200 None 2026-04-03 05:00:42,614 - lava_callback - ERROR - Error processing callback for node 69cca6748cd9cbcdc945b954: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48738 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:01:22,619 - lava_callback - INFO - Received callback for node 69cca69d8cd9cbcdc945b95f INFO: 172.21.0.1:46248 - "POST /node/69cca69d8cd9cbcdc945b95f HTTP/1.0" 202 Accepted 2026-04-03 05:01:22,625 - lava_callback - INFO - Starting async job submit for node 69cca69d8cd9cbcdc945b95f 2026-04-03 05:01:22,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:01:22,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca69d8cd9cbcdc945b95f HTTP/1.1" 200 None 2026-04-03 05:01:22,641 - lava_callback - DEBUG - Processing node 69cca69d8cd9cbcdc945b95f: baseline-arm-clabbe 2026-04-03 05:01:22,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca69d8cd9cbcdc945b95f 2026-04-03 05:01:22,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:01:22,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca69d8cd9cbcdc945b95f HTTP/1.1" 200 None 2026-04-03 05:01:22,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:01:22,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 05:01:22,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:01:22,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca69d8cd9cbcdc945b95f HTTP/1.1" 200 None 2026-04-03 05:01:22,718 - lava_callback - ERROR - Error processing callback for node 69cca69d8cd9cbcdc945b95f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53390 - "GET /node/69487f099214b49387169829%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:52296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60654 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:02:02,603 - lava_callback - INFO - Received callback for node 69cca6c78cd9cbcdc945b971 INFO: 172.21.0.1:50548 - "POST /node/69cca6c78cd9cbcdc945b971 HTTP/1.0" 202 Accepted 2026-04-03 05:02:02,608 - lava_callback - INFO - Starting async job submit for node 69cca6c78cd9cbcdc945b971 2026-04-03 05:02:02,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:02:02,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca6c78cd9cbcdc945b971 HTTP/1.1" 200 None 2026-04-03 05:02:02,624 - lava_callback - DEBUG - Processing node 69cca6c78cd9cbcdc945b971: baseline-arm-clabbe 2026-04-03 05:02:02,624 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca6c78cd9cbcdc945b971 2026-04-03 05:02:02,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:02:02,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca6c78cd9cbcdc945b971 HTTP/1.1" 200 None 2026-04-03 05:02:02,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:02:02,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 05:02:02,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:02:02,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca6c78cd9cbcdc945b971 HTTP/1.1" 200 None 2026-04-03 05:02:02,689 - lava_callback - ERROR - Error processing callback for node 69cca6c78cd9cbcdc945b971: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48262 - "GET /node/6985b64971989fb1414968d5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:48264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42420 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:02:42,692 - lava_callback - INFO - Received callback for node 69cca6ef8cd9cbcdc945b983 INFO: 172.21.0.1:60162 - "POST /node/69cca6ef8cd9cbcdc945b983 HTTP/1.0" 202 Accepted 2026-04-03 05:02:42,697 - lava_callback - INFO - Starting async job submit for node 69cca6ef8cd9cbcdc945b983 2026-04-03 05:02:42,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:02:42,713 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca6ef8cd9cbcdc945b983 HTTP/1.1" 200 None 2026-04-03 05:02:42,714 - lava_callback - DEBUG - Processing node 69cca6ef8cd9cbcdc945b983: baseline-arm-clabbe 2026-04-03 05:02:42,714 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca6ef8cd9cbcdc945b983 2026-04-03 05:02:42,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:02:42,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca6ef8cd9cbcdc945b983 HTTP/1.1" 200 None 2026-04-03 05:02:42,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:02:42,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 05:02:42,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:02:42,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca6ef8cd9cbcdc945b983 HTTP/1.1" 200 None 2026-04-03 05:02:42,777 - lava_callback - ERROR - Error processing callback for node 69cca6ef8cd9cbcdc945b983: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60368 - "GET /node/698ef15ef0e3ffbba67d2b39%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:60370 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:03:22,652 - lava_callback - INFO - Received callback for node 69cca7178cd9cbcdc945b990 INFO: 172.21.0.1:59512 - "POST /node/69cca7178cd9cbcdc945b990 HTTP/1.0" 202 Accepted 2026-04-03 05:03:22,657 - lava_callback - INFO - Starting async job submit for node 69cca7178cd9cbcdc945b990 2026-04-03 05:03:22,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:03:22,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca7178cd9cbcdc945b990 HTTP/1.1" 200 None 2026-04-03 05:03:22,680 - lava_callback - DEBUG - Processing node 69cca7178cd9cbcdc945b990: baseline-arm-clabbe 2026-04-03 05:03:22,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca7178cd9cbcdc945b990 2026-04-03 05:03:22,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:03:22,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca7178cd9cbcdc945b990 HTTP/1.1" 200 None 2026-04-03 05:03:22,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:03:22,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 05:03:22,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:03:22,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca7178cd9cbcdc945b990 HTTP/1.1" 200 None 2026-04-03 05:03:22,751 - lava_callback - ERROR - Error processing callback for node 69cca7178cd9cbcdc945b990: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49846 - "GET /node/693fd3c4575bc060ce14c182%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49862 - "GET /node/69281221b1b5e06bf6c0512c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:57390 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:04:02,679 - lava_callback - INFO - Received callback for node 69cca73f8cd9cbcdc945b9a1 INFO: 172.21.0.1:40928 - "POST /node/69cca73f8cd9cbcdc945b9a1 HTTP/1.0" 202 Accepted 2026-04-03 05:04:02,685 - lava_callback - INFO - Starting async job submit for node 69cca73f8cd9cbcdc945b9a1 2026-04-03 05:04:02,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:04:02,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca73f8cd9cbcdc945b9a1 HTTP/1.1" 200 None 2026-04-03 05:04:02,700 - lava_callback - DEBUG - Processing node 69cca73f8cd9cbcdc945b9a1: baseline-arm-clabbe 2026-04-03 05:04:02,701 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca73f8cd9cbcdc945b9a1 2026-04-03 05:04:02,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:04:02,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca73f8cd9cbcdc945b9a1 HTTP/1.1" 200 None 2026-04-03 05:04:02,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:04:02,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 05:04:02,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:04:02,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca73f8cd9cbcdc945b9a1 HTTP/1.1" 200 None 2026-04-03 05:04:02,760 - lava_callback - ERROR - Error processing callback for node 69cca73f8cd9cbcdc945b9a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51614 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56796 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:05:02,724 - lava_callback - INFO - Received callback for node 69cca7698cd9cbcdc945b9b1 INFO: 172.21.0.1:60080 - "POST /node/69cca7698cd9cbcdc945b9b1 HTTP/1.0" 202 Accepted 2026-04-03 05:05:02,730 - lava_callback - INFO - Starting async job submit for node 69cca7698cd9cbcdc945b9b1 2026-04-03 05:05:02,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:05:02,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca7698cd9cbcdc945b9b1 HTTP/1.1" 200 None 2026-04-03 05:05:02,756 - lava_callback - DEBUG - Processing node 69cca7698cd9cbcdc945b9b1: baseline-arm-clabbe 2026-04-03 05:05:02,756 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca7698cd9cbcdc945b9b1 2026-04-03 05:05:02,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:05:02,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca7698cd9cbcdc945b9b1 HTTP/1.1" 200 None 2026-04-03 05:05:02,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:05:02,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950a9f2866272ea51b8c4 HTTP/1.1" 200 None 2026-04-03 05:05:02,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:05:02,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca7698cd9cbcdc945b9b1 HTTP/1.1" 200 None 2026-04-03 05:05:02,808 - lava_callback - ERROR - Error processing callback for node 69cca7698cd9cbcdc945b9b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49622 - "GET /node/693a92d8b6ebbedf18b2669e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34370 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55252 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:10:08,182 - lava_callback - INFO - Received callback for node 69cca8a58cd9cbcdc945ba27 INFO: 172.21.0.1:55260 - "POST /node/69cca8a58cd9cbcdc945ba27 HTTP/1.0" 202 Accepted 2026-04-03 05:10:08,187 - lava_callback - INFO - Starting async job submit for node 69cca8a58cd9cbcdc945ba27 2026-04-03 05:10:08,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:10:08,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca8a58cd9cbcdc945ba27 HTTP/1.1" 200 None 2026-04-03 05:10:08,203 - lava_callback - DEBUG - Processing node 69cca8a58cd9cbcdc945ba27: baseline-arm64-baylibre 2026-04-03 05:10:08,204 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca8a58cd9cbcdc945ba27 2026-04-03 05:10:08,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:10:08,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca8a58cd9cbcdc945ba27 HTTP/1.1" 200 None 2026-04-03 05:10:08,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:10:08,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:10:08,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:10:08,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca8a58cd9cbcdc945ba27 HTTP/1.1" 200 None 2026-04-03 05:10:08,260 - lava_callback - ERROR - Error processing callback for node 69cca8a58cd9cbcdc945ba27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37076 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:12:43,014 - lava_callback - INFO - Received callback for node 69cca9438cd9cbcdc945ba7d INFO: 172.21.0.1:40256 - "POST /node/69cca9438cd9cbcdc945ba7d HTTP/1.0" 202 Accepted 2026-04-03 05:12:43,019 - lava_callback - INFO - Starting async job submit for node 69cca9438cd9cbcdc945ba7d 2026-04-03 05:12:43,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:12:43,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca9438cd9cbcdc945ba7d HTTP/1.1" 200 None 2026-04-03 05:12:43,037 - lava_callback - DEBUG - Processing node 69cca9438cd9cbcdc945ba7d: baseline-arm-clabbe 2026-04-03 05:12:43,037 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca9438cd9cbcdc945ba7d 2026-04-03 05:12:43,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:12:43,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca9438cd9cbcdc945ba7d HTTP/1.1" 200 None 2026-04-03 05:12:43,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:12:43,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 05:12:43,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:12:43,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca9438cd9cbcdc945ba7d HTTP/1.1" 200 None 2026-04-03 05:12:43,090 - lava_callback - ERROR - Error processing callback for node 69cca9438cd9cbcdc945ba7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57518 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:13:28,282 - lava_callback - INFO - Received callback for node 69cca9718cd9cbcdc945ba88 INFO: 172.21.0.1:57524 - "POST /node/69cca9718cd9cbcdc945ba88 HTTP/1.0" 202 Accepted 2026-04-03 05:13:28,287 - lava_callback - INFO - Starting async job submit for node 69cca9718cd9cbcdc945ba88 2026-04-03 05:13:28,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:13:28,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca9718cd9cbcdc945ba88 HTTP/1.1" 200 None 2026-04-03 05:13:28,303 - lava_callback - DEBUG - Processing node 69cca9718cd9cbcdc945ba88: baseline-arm64-baylibre 2026-04-03 05:13:28,304 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca9718cd9cbcdc945ba88 2026-04-03 05:13:28,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:13:28,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca9718cd9cbcdc945ba88 HTTP/1.1" 200 None 2026-04-03 05:13:28,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:13:28,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:13:28,342 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:13:28,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca9718cd9cbcdc945ba88 HTTP/1.1" 200 None 2026-04-03 05:13:28,368 - lava_callback - ERROR - Error processing callback for node 69cca9718cd9cbcdc945ba88: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56696 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36650 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:14:43,087 - lava_callback - INFO - Received callback for node 69cca9bd8cd9cbcdc945baba INFO: 172.21.0.1:36738 - "POST /node/69cca9bd8cd9cbcdc945baba HTTP/1.0" 202 Accepted 2026-04-03 05:14:43,093 - lava_callback - INFO - Starting async job submit for node 69cca9bd8cd9cbcdc945baba 2026-04-03 05:14:43,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:14:43,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca9bd8cd9cbcdc945baba HTTP/1.1" 200 None 2026-04-03 05:14:43,110 - lava_callback - DEBUG - Processing node 69cca9bd8cd9cbcdc945baba: baseline-arm-clabbe 2026-04-03 05:14:43,110 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca9bd8cd9cbcdc945baba 2026-04-03 05:14:43,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:14:43,125 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca9bd8cd9cbcdc945baba HTTP/1.1" 200 None 2026-04-03 05:14:43,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:14:43,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 05:14:43,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:14:43,160 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca9bd8cd9cbcdc945baba HTTP/1.1" 200 None 2026-04-03 05:14:43,162 - lava_callback - ERROR - Error processing callback for node 69cca9bd8cd9cbcdc945baba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47534 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:15:23,093 - lava_callback - INFO - Received callback for node 69cca9e48cd9cbcdc945bac3 INFO: 172.21.0.1:53148 - "POST /node/69cca9e48cd9cbcdc945bac3 HTTP/1.0" 202 Accepted 2026-04-03 05:15:23,099 - lava_callback - INFO - Starting async job submit for node 69cca9e48cd9cbcdc945bac3 2026-04-03 05:15:23,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:15:23,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca9e48cd9cbcdc945bac3 HTTP/1.1" 200 None 2026-04-03 05:15:23,115 - lava_callback - DEBUG - Processing node 69cca9e48cd9cbcdc945bac3: baseline-arm-clabbe 2026-04-03 05:15:23,115 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cca9e48cd9cbcdc945bac3 2026-04-03 05:15:23,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:15:23,140 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cca9e48cd9cbcdc945bac3 HTTP/1.1" 200 None 2026-04-03 05:15:23,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:15:23,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 05:15:23,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:15:23,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cca9e48cd9cbcdc945bac3 HTTP/1.1" 200 None 2026-04-03 05:15:23,184 - lava_callback - ERROR - Error processing callback for node 69cca9e48cd9cbcdc945bac3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45554 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:16:08,384 - lava_callback - INFO - Received callback for node 69ccaa148cd9cbcdc945bad4 INFO: 172.21.0.1:45560 - "POST /node/69ccaa148cd9cbcdc945bad4 HTTP/1.0" 202 Accepted 2026-04-03 05:16:08,390 - lava_callback - INFO - Starting async job submit for node 69ccaa148cd9cbcdc945bad4 2026-04-03 05:16:08,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:16:08,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaa148cd9cbcdc945bad4 HTTP/1.1" 200 None 2026-04-03 05:16:08,407 - lava_callback - DEBUG - Processing node 69ccaa148cd9cbcdc945bad4: baseline-arm64-baylibre 2026-04-03 05:16:08,408 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccaa148cd9cbcdc945bad4 2026-04-03 05:16:08,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:16:08,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaa148cd9cbcdc945bad4 HTTP/1.1" 200 None 2026-04-03 05:16:08,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:16:08,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:16:08,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:16:08,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccaa148cd9cbcdc945bad4 HTTP/1.1" 200 None 2026-04-03 05:16:08,475 - lava_callback - ERROR - Error processing callback for node 69ccaa148cd9cbcdc945bad4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49606 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38878 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:17:23,182 - lava_callback - INFO - Received callback for node 69ccaa608cd9cbcdc945bafa INFO: 172.21.0.1:43408 - "POST /node/69ccaa608cd9cbcdc945bafa HTTP/1.0" 202 Accepted 2026-04-03 05:17:23,188 - lava_callback - INFO - Starting async job submit for node 69ccaa608cd9cbcdc945bafa 2026-04-03 05:17:23,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:17:23,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaa608cd9cbcdc945bafa HTTP/1.1" 200 None 2026-04-03 05:17:23,224 - lava_callback - DEBUG - Processing node 69ccaa608cd9cbcdc945bafa: baseline-arm-clabbe 2026-04-03 05:17:23,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccaa608cd9cbcdc945bafa 2026-04-03 05:17:23,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:17:23,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaa608cd9cbcdc945bafa HTTP/1.1" 200 None 2026-04-03 05:17:23,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:17:23,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 05:17:23,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:17:23,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccaa608cd9cbcdc945bafa HTTP/1.1" 200 None 2026-04-03 05:17:23,300 - lava_callback - ERROR - Error processing callback for node 69ccaa608cd9cbcdc945bafa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47646 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:18:23,213 - lava_callback - INFO - Received callback for node 69ccaa8a8cd9cbcdc945bb0e INFO: 172.21.0.1:34436 - "POST /node/69ccaa8a8cd9cbcdc945bb0e HTTP/1.0" 202 Accepted 2026-04-03 05:18:23,219 - lava_callback - INFO - Starting async job submit for node 69ccaa8a8cd9cbcdc945bb0e 2026-04-03 05:18:23,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:18:23,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaa8a8cd9cbcdc945bb0e HTTP/1.1" 200 None 2026-04-03 05:18:23,233 - lava_callback - DEBUG - Processing node 69ccaa8a8cd9cbcdc945bb0e: baseline-arm-clabbe 2026-04-03 05:18:23,234 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccaa8a8cd9cbcdc945bb0e 2026-04-03 05:18:23,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:18:23,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaa8a8cd9cbcdc945bb0e HTTP/1.1" 200 None 2026-04-03 05:18:23,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:18:23,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 05:18:23,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:18:23,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccaa8a8cd9cbcdc945bb0e HTTP/1.1" 200 None 2026-04-03 05:18:23,292 - lava_callback - ERROR - Error processing callback for node 69ccaa8a8cd9cbcdc945bb0e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32928 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:19:03,207 - lava_callback - INFO - Received callback for node 69ccaab18cd9cbcdc945bb1e INFO: 172.21.0.1:57922 - "POST /node/69ccaab18cd9cbcdc945bb1e HTTP/1.0" 202 Accepted 2026-04-03 05:19:03,213 - lava_callback - INFO - Starting async job submit for node 69ccaab18cd9cbcdc945bb1e 2026-04-03 05:19:03,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:19:03,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaab18cd9cbcdc945bb1e HTTP/1.1" 200 None 2026-04-03 05:19:03,249 - lava_callback - DEBUG - Processing node 69ccaab18cd9cbcdc945bb1e: baseline-arm-clabbe 2026-04-03 05:19:03,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccaab18cd9cbcdc945bb1e 2026-04-03 05:19:03,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:19:03,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaab18cd9cbcdc945bb1e HTTP/1.1" 200 None 2026-04-03 05:19:03,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:19:03,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4108df9359ff4897d25 HTTP/1.1" 200 None 2026-04-03 05:19:03,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:19:03,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccaab18cd9cbcdc945bb1e HTTP/1.1" 200 None 2026-04-03 05:19:03,308 - lava_callback - ERROR - Error processing callback for node 69ccaab18cd9cbcdc945bb1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57128 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49622 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:21:23,339 - lava_callback - INFO - Received callback for node 69ccab428cd9cbcdc945bb5c INFO: 172.21.0.1:58202 - "POST /node/69ccab428cd9cbcdc945bb5c HTTP/1.0" 202 Accepted 2026-04-03 05:21:23,345 - lava_callback - INFO - Starting async job submit for node 69ccab428cd9cbcdc945bb5c 2026-04-03 05:21:23,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:21:23,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccab428cd9cbcdc945bb5c HTTP/1.1" 200 None 2026-04-03 05:21:23,361 - lava_callback - DEBUG - Processing node 69ccab428cd9cbcdc945bb5c: baseline-arm64-clabbe 2026-04-03 05:21:23,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccab428cd9cbcdc945bb5c 2026-04-03 05:21:23,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:21:23,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccab428cd9cbcdc945bb5c HTTP/1.1" 200 None 2026-04-03 05:21:23,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:21:23,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:21:23,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:21:23,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccab428cd9cbcdc945bb5c HTTP/1.1" 200 None 2026-04-03 05:21:23,420 - lava_callback - ERROR - Error processing callback for node 69ccab428cd9cbcdc945bb5c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37060 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:26:03,596 - lava_callback - INFO - Received callback for node 69ccac578cd9cbcdc945bbe2 INFO: 172.21.0.1:51370 - "POST /node/69ccac578cd9cbcdc945bbe2 HTTP/1.0" 202 Accepted 2026-04-03 05:26:03,602 - lava_callback - INFO - Starting async job submit for node 69ccac578cd9cbcdc945bbe2 2026-04-03 05:26:03,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:26:03,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccac578cd9cbcdc945bbe2 HTTP/1.1" 200 None 2026-04-03 05:26:03,618 - lava_callback - DEBUG - Processing node 69ccac578cd9cbcdc945bbe2: baseline-arm64-clabbe 2026-04-03 05:26:03,618 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccac578cd9cbcdc945bbe2 2026-04-03 05:26:03,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:26:03,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccac578cd9cbcdc945bbe2 HTTP/1.1" 200 None 2026-04-03 05:26:03,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:26:03,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:26:03,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:26:03,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccac578cd9cbcdc945bbe2 HTTP/1.1" 200 None 2026-04-03 05:26:03,671 - lava_callback - ERROR - Error processing callback for node 69ccac578cd9cbcdc945bbe2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44196 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:28:43,728 - lava_callback - INFO - Received callback for node 69ccad068cd9cbcdc945bc4c INFO: 172.21.0.1:56310 - "POST /node/69ccad068cd9cbcdc945bc4c HTTP/1.0" 202 Accepted 2026-04-03 05:28:43,733 - lava_callback - INFO - Starting async job submit for node 69ccad068cd9cbcdc945bc4c 2026-04-03 05:28:43,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:28:43,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccad068cd9cbcdc945bc4c HTTP/1.1" 200 None 2026-04-03 05:28:43,749 - lava_callback - DEBUG - Processing node 69ccad068cd9cbcdc945bc4c: baseline-arm64-clabbe 2026-04-03 05:28:43,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccad068cd9cbcdc945bc4c 2026-04-03 05:28:43,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:28:43,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccad068cd9cbcdc945bc4c HTTP/1.1" 200 None 2026-04-03 05:28:43,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:28:43,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:28:43,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:28:43,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccad068cd9cbcdc945bc4c HTTP/1.1" 200 None 2026-04-03 05:28:43,801 - lava_callback - ERROR - Error processing callback for node 69ccad068cd9cbcdc945bc4c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54456 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:30:24,021 - lava_callback - INFO - Received callback for node 69ccad608cd9cbcdc945bc79 INFO: 172.21.0.1:57636 - "POST /node/69ccad608cd9cbcdc945bc79 HTTP/1.0" 202 Accepted 2026-04-03 05:30:24,027 - lava_callback - INFO - Starting async job submit for node 69ccad608cd9cbcdc945bc79 2026-04-03 05:30:24,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:30:24,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccad608cd9cbcdc945bc79 HTTP/1.1" 200 None 2026-04-03 05:30:24,043 - lava_callback - DEBUG - Processing node 69ccad608cd9cbcdc945bc79: baseline-arm64-clabbe 2026-04-03 05:30:24,043 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccad608cd9cbcdc945bc79 2026-04-03 05:30:24,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:30:24,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccad608cd9cbcdc945bc79 HTTP/1.1" 200 None 2026-04-03 05:30:24,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:30:24,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:30:24,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:30:24,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccad608cd9cbcdc945bc79 HTTP/1.1" 200 None 2026-04-03 05:30:24,104 - lava_callback - ERROR - Error processing callback for node 69ccad608cd9cbcdc945bc79: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58892 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:32:03,864 - lava_callback - INFO - Received callback for node 69ccadc08cd9cbcdc945bca3 INFO: 172.21.0.1:35818 - "POST /node/69ccadc08cd9cbcdc945bca3 HTTP/1.0" 202 Accepted 2026-04-03 05:32:03,870 - lava_callback - INFO - Starting async job submit for node 69ccadc08cd9cbcdc945bca3 2026-04-03 05:32:03,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:32:03,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccadc08cd9cbcdc945bca3 HTTP/1.1" 200 None 2026-04-03 05:32:03,890 - lava_callback - DEBUG - Processing node 69ccadc08cd9cbcdc945bca3: baseline-arm64-clabbe 2026-04-03 05:32:03,890 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccadc08cd9cbcdc945bca3 2026-04-03 05:32:03,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:32:03,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccadc08cd9cbcdc945bca3 HTTP/1.1" 200 None 2026-04-03 05:32:03,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:32:03,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:32:03,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:32:03,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccadc08cd9cbcdc945bca3 HTTP/1.1" 200 None 2026-04-03 05:32:03,947 - lava_callback - ERROR - Error processing callback for node 69ccadc08cd9cbcdc945bca3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44954 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:33:09,539 - lava_callback - INFO - Received callback for node 69ccae128cd9cbcdc945bccf INFO: 172.21.0.1:44960 - "POST /node/69ccae128cd9cbcdc945bccf HTTP/1.0" 202 Accepted 2026-04-03 05:33:09,544 - lava_callback - INFO - Starting async job submit for node 69ccae128cd9cbcdc945bccf 2026-04-03 05:33:09,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:33:09,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccae128cd9cbcdc945bccf HTTP/1.1" 200 None 2026-04-03 05:33:09,565 - lava_callback - DEBUG - Processing node 69ccae128cd9cbcdc945bccf: baseline-arm64-baylibre 2026-04-03 05:33:09,566 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccae128cd9cbcdc945bccf 2026-04-03 05:33:09,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:33:09,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccae128cd9cbcdc945bccf HTTP/1.1" 200 None 2026-04-03 05:33:09,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:33:09,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:33:09,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:33:09,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccae128cd9cbcdc945bccf HTTP/1.1" 200 None 2026-04-03 05:33:09,617 - lava_callback - ERROR - Error processing callback for node 69ccae128cd9cbcdc945bccf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34358 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:35:03,922 - lava_callback - INFO - Received callback for node 69ccae728cd9cbcdc945bcf1 INFO: 172.21.0.1:51928 - "POST /node/69ccae728cd9cbcdc945bcf1 HTTP/1.0" 202 Accepted 2026-04-03 05:35:03,928 - lava_callback - INFO - Starting async job submit for node 69ccae728cd9cbcdc945bcf1 2026-04-03 05:35:03,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:35:03,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccae728cd9cbcdc945bcf1 HTTP/1.1" 200 None 2026-04-03 05:35:03,944 - lava_callback - DEBUG - Processing node 69ccae728cd9cbcdc945bcf1: baseline-arm64-clabbe 2026-04-03 05:35:03,945 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccae728cd9cbcdc945bcf1 2026-04-03 05:35:03,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:35:03,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccae728cd9cbcdc945bcf1 HTTP/1.1" 200 None 2026-04-03 05:35:03,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:35:03,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:35:03,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:35:04,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccae728cd9cbcdc945bcf1 HTTP/1.1" 200 None 2026-04-03 05:35:04,004 - lava_callback - ERROR - Error processing callback for node 69ccae728cd9cbcdc945bcf1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59936 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:36:09,552 - lava_callback - INFO - Received callback for node 69ccaec18cd9cbcdc945bd0c INFO: 172.21.0.1:59948 - "POST /node/69ccaec18cd9cbcdc945bd0c HTTP/1.0" 202 Accepted 2026-04-03 05:36:09,557 - lava_callback - INFO - Starting async job submit for node 69ccaec18cd9cbcdc945bd0c 2026-04-03 05:36:09,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:36:09,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaec18cd9cbcdc945bd0c HTTP/1.1" 200 None 2026-04-03 05:36:09,573 - lava_callback - DEBUG - Processing node 69ccaec18cd9cbcdc945bd0c: baseline-arm64-baylibre 2026-04-03 05:36:09,574 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccaec18cd9cbcdc945bd0c 2026-04-03 05:36:09,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:36:09,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaec18cd9cbcdc945bd0c HTTP/1.1" 200 None 2026-04-03 05:36:09,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:36:09,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:36:09,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:36:09,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccaec18cd9cbcdc945bd0c HTTP/1.1" 200 None 2026-04-03 05:36:09,630 - lava_callback - ERROR - Error processing callback for node 69ccaec18cd9cbcdc945bd0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43042 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41460 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:37:29,584 - lava_callback - INFO - Received callback for node 69ccaf188cd9cbcdc945bd31 INFO: 172.21.0.1:41468 - "POST /node/69ccaf188cd9cbcdc945bd31 HTTP/1.0" 202 Accepted 2026-04-03 05:37:29,590 - lava_callback - INFO - Starting async job submit for node 69ccaf188cd9cbcdc945bd31 2026-04-03 05:37:29,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:37:29,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaf188cd9cbcdc945bd31 HTTP/1.1" 200 None 2026-04-03 05:37:29,605 - lava_callback - DEBUG - Processing node 69ccaf188cd9cbcdc945bd31: baseline-arm64-baylibre 2026-04-03 05:37:29,605 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccaf188cd9cbcdc945bd31 2026-04-03 05:37:29,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:37:29,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaf188cd9cbcdc945bd31 HTTP/1.1" 200 None 2026-04-03 05:37:29,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:37:29,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:37:29,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:37:29,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccaf188cd9cbcdc945bd31 HTTP/1.1" 200 None 2026-04-03 05:37:29,671 - lava_callback - ERROR - Error processing callback for node 69ccaf188cd9cbcdc945bd31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36480 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40562 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:39:09,758 - lava_callback - INFO - Received callback for node 69ccaf6e8cd9cbcdc945bd59 INFO: 172.21.0.1:33244 - "POST /node/69ccaf6e8cd9cbcdc945bd59 HTTP/1.0" 202 Accepted 2026-04-03 05:39:09,763 - lava_callback - INFO - Starting async job submit for node 69ccaf6e8cd9cbcdc945bd59 2026-04-03 05:39:09,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:39:09,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaf6e8cd9cbcdc945bd59 HTTP/1.1" 200 None 2026-04-03 05:39:09,781 - lava_callback - DEBUG - Processing node 69ccaf6e8cd9cbcdc945bd59: baseline-arm64-baylibre 2026-04-03 05:39:09,782 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccaf6e8cd9cbcdc945bd59 2026-04-03 05:39:09,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:39:09,795 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccaf6e8cd9cbcdc945bd59 HTTP/1.1" 200 None 2026-04-03 05:39:09,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:39:09,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:39:09,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:39:09,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccaf6e8cd9cbcdc945bd59 HTTP/1.1" 200 None 2026-04-03 05:39:09,863 - lava_callback - ERROR - Error processing callback for node 69ccaf6e8cd9cbcdc945bd59: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33250 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58334 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:40:29,794 - lava_callback - INFO - Received callback for node 69ccafc68cd9cbcdc945bd7d INFO: 172.21.0.1:50988 - "POST /node/69ccafc68cd9cbcdc945bd7d HTTP/1.0" 202 Accepted 2026-04-03 05:40:29,799 - lava_callback - INFO - Starting async job submit for node 69ccafc68cd9cbcdc945bd7d 2026-04-03 05:40:29,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:40:29,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccafc68cd9cbcdc945bd7d HTTP/1.1" 200 None 2026-04-03 05:40:29,817 - lava_callback - DEBUG - Processing node 69ccafc68cd9cbcdc945bd7d: baseline-arm64-baylibre 2026-04-03 05:40:29,817 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccafc68cd9cbcdc945bd7d 2026-04-03 05:40:29,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:40:29,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccafc68cd9cbcdc945bd7d HTTP/1.1" 200 None 2026-04-03 05:40:29,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:40:29,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:40:29,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:40:29,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccafc68cd9cbcdc945bd7d HTTP/1.1" 200 None 2026-04-03 05:40:29,898 - lava_callback - ERROR - Error processing callback for node 69ccafc68cd9cbcdc945bd7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48760 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:46:44,481 - lava_callback - INFO - Received callback for node 69ccb1308cd9cbcdc945be21 INFO: 172.21.0.1:60588 - "POST /node/69ccb1308cd9cbcdc945be21 HTTP/1.0" 202 Accepted 2026-04-03 05:46:44,486 - lava_callback - INFO - Starting async job submit for node 69ccb1308cd9cbcdc945be21 2026-04-03 05:46:44,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:46:44,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb1308cd9cbcdc945be21 HTTP/1.1" 200 None 2026-04-03 05:46:44,505 - lava_callback - DEBUG - Processing node 69ccb1308cd9cbcdc945be21: baseline-arm64-clabbe 2026-04-03 05:46:44,506 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb1308cd9cbcdc945be21 2026-04-03 05:46:44,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:46:44,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb1308cd9cbcdc945be21 HTTP/1.1" 200 None 2026-04-03 05:46:44,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:46:44,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:46:44,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:46:44,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb1308cd9cbcdc945be21 HTTP/1.1" 200 None 2026-04-03 05:46:44,557 - lava_callback - ERROR - Error processing callback for node 69ccb1308cd9cbcdc945be21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38830 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:48:04,513 - lava_callback - INFO - Received callback for node 69ccb18d8cd9cbcdc945be44 INFO: 172.21.0.1:39876 - "POST /node/69ccb18d8cd9cbcdc945be44 HTTP/1.0" 202 Accepted 2026-04-03 05:48:04,518 - lava_callback - INFO - Starting async job submit for node 69ccb18d8cd9cbcdc945be44 2026-04-03 05:48:04,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:48:04,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb18d8cd9cbcdc945be44 HTTP/1.1" 200 None 2026-04-03 05:48:04,534 - lava_callback - DEBUG - Processing node 69ccb18d8cd9cbcdc945be44: baseline-arm64-clabbe 2026-04-03 05:48:04,534 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb18d8cd9cbcdc945be44 2026-04-03 05:48:04,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:48:04,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb18d8cd9cbcdc945be44 HTTP/1.1" 200 None 2026-04-03 05:48:04,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:48:04,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:48:04,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:48:04,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb18d8cd9cbcdc945be44 HTTP/1.1" 200 None 2026-04-03 05:48:04,602 - lava_callback - ERROR - Error processing callback for node 69ccb18d8cd9cbcdc945be44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34100 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:49:44,564 - lava_callback - INFO - Received callback for node 69ccb1e38cd9cbcdc945be68 INFO: 172.21.0.1:41050 - "POST /node/69ccb1e38cd9cbcdc945be68 HTTP/1.0" 202 Accepted 2026-04-03 05:49:44,570 - lava_callback - INFO - Starting async job submit for node 69ccb1e38cd9cbcdc945be68 2026-04-03 05:49:44,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:49:44,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb1e38cd9cbcdc945be68 HTTP/1.1" 200 None 2026-04-03 05:49:44,586 - lava_callback - DEBUG - Processing node 69ccb1e38cd9cbcdc945be68: baseline-arm64-clabbe 2026-04-03 05:49:44,586 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb1e38cd9cbcdc945be68 2026-04-03 05:49:44,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:49:44,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb1e38cd9cbcdc945be68 HTTP/1.1" 200 None 2026-04-03 05:49:44,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:49:44,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:49:44,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:49:44,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb1e38cd9cbcdc945be68 HTTP/1.1" 200 None 2026-04-03 05:49:44,651 - lava_callback - ERROR - Error processing callback for node 69ccb1e38cd9cbcdc945be68: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42258 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:51:04,667 - lava_callback - INFO - Received callback for node 69ccb23a8cd9cbcdc945be96 INFO: 172.21.0.1:41812 - "POST /node/69ccb23a8cd9cbcdc945be96 HTTP/1.0" 202 Accepted 2026-04-03 05:51:04,673 - lava_callback - INFO - Starting async job submit for node 69ccb23a8cd9cbcdc945be96 2026-04-03 05:51:04,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:51:04,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb23a8cd9cbcdc945be96 HTTP/1.1" 200 None 2026-04-03 05:51:04,689 - lava_callback - DEBUG - Processing node 69ccb23a8cd9cbcdc945be96: baseline-arm64-clabbe 2026-04-03 05:51:04,689 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb23a8cd9cbcdc945be96 2026-04-03 05:51:04,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:51:04,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb23a8cd9cbcdc945be96 HTTP/1.1" 200 None 2026-04-03 05:51:04,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:51:04,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:51:04,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:51:04,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb23a8cd9cbcdc945be96 HTTP/1.1" 200 None 2026-04-03 05:51:04,759 - lava_callback - ERROR - Error processing callback for node 69ccb23a8cd9cbcdc945be96: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48334 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:52:24,665 - lava_callback - INFO - Received callback for node 69ccb2928cd9cbcdc945bec3 INFO: 172.21.0.1:56178 - "POST /node/69ccb2928cd9cbcdc945bec3 HTTP/1.0" 202 Accepted 2026-04-03 05:52:24,671 - lava_callback - INFO - Starting async job submit for node 69ccb2928cd9cbcdc945bec3 2026-04-03 05:52:24,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:52:24,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb2928cd9cbcdc945bec3 HTTP/1.1" 200 None 2026-04-03 05:52:24,693 - lava_callback - DEBUG - Processing node 69ccb2928cd9cbcdc945bec3: baseline-arm64-clabbe 2026-04-03 05:52:24,693 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb2928cd9cbcdc945bec3 2026-04-03 05:52:24,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:52:24,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb2928cd9cbcdc945bec3 HTTP/1.1" 200 None 2026-04-03 05:52:24,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:52:24,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:52:24,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:52:24,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb2928cd9cbcdc945bec3 HTTP/1.1" 200 None 2026-04-03 05:52:24,750 - lava_callback - ERROR - Error processing callback for node 69ccb2928cd9cbcdc945bec3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41188 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:54:04,742 - lava_callback - INFO - Received callback for node 69ccb2f08cd9cbcdc945bee6 INFO: 172.21.0.1:54832 - "POST /node/69ccb2f08cd9cbcdc945bee6 HTTP/1.0" 202 Accepted 2026-04-03 05:54:04,747 - lava_callback - INFO - Starting async job submit for node 69ccb2f08cd9cbcdc945bee6 2026-04-03 05:54:04,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:54:04,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb2f08cd9cbcdc945bee6 HTTP/1.1" 200 None 2026-04-03 05:54:04,763 - lava_callback - DEBUG - Processing node 69ccb2f08cd9cbcdc945bee6: baseline-arm64-clabbe 2026-04-03 05:54:04,764 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb2f08cd9cbcdc945bee6 2026-04-03 05:54:04,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:54:04,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb2f08cd9cbcdc945bee6 HTTP/1.1" 200 None 2026-04-03 05:54:04,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:54:04,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cbf4278df9359ff4897d37 HTTP/1.1" 200 None 2026-04-03 05:54:04,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:54:04,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb2f08cd9cbcdc945bee6 HTTP/1.1" 200 None 2026-04-03 05:54:04,830 - lava_callback - ERROR - Error processing callback for node 69ccb2f08cd9cbcdc945bee6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38006 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:55:30,941 - lava_callback - INFO - Received callback for node 69ccb33e8cd9cbcdc945bf0c INFO: 172.21.0.1:35190 - "POST /node/69ccb33e8cd9cbcdc945bf0c HTTP/1.0" 202 Accepted 2026-04-03 05:55:30,946 - lava_callback - INFO - Starting async job submit for node 69ccb33e8cd9cbcdc945bf0c 2026-04-03 05:55:30,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:55:30,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb33e8cd9cbcdc945bf0c HTTP/1.1" 200 None 2026-04-03 05:55:30,963 - lava_callback - DEBUG - Processing node 69ccb33e8cd9cbcdc945bf0c: baseline-arm64-baylibre 2026-04-03 05:55:30,963 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb33e8cd9cbcdc945bf0c 2026-04-03 05:55:30,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:55:30,978 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb33e8cd9cbcdc945bf0c HTTP/1.1" 200 None 2026-04-03 05:55:30,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:55:30,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:55:30,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:55:31,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb33e8cd9cbcdc945bf0c HTTP/1.1" 200 None 2026-04-03 05:55:31,034 - lava_callback - ERROR - Error processing callback for node 69ccb33e8cd9cbcdc945bf0c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53016 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:56:50,648 - lava_callback - INFO - Received callback for node 69ccb3948cd9cbcdc945bf31 INFO: 172.21.0.1:39890 - "POST /node/69ccb3948cd9cbcdc945bf31 HTTP/1.0" 202 Accepted 2026-04-03 05:56:50,653 - lava_callback - INFO - Starting async job submit for node 69ccb3948cd9cbcdc945bf31 2026-04-03 05:56:50,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:56:50,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb3948cd9cbcdc945bf31 HTTP/1.1" 200 None 2026-04-03 05:56:50,692 - lava_callback - DEBUG - Processing node 69ccb3948cd9cbcdc945bf31: baseline-arm64-baylibre 2026-04-03 05:56:50,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb3948cd9cbcdc945bf31 2026-04-03 05:56:50,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:56:50,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb3948cd9cbcdc945bf31 HTTP/1.1" 200 None 2026-04-03 05:56:50,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:56:50,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:56:50,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:56:50,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb3948cd9cbcdc945bf31 HTTP/1.1" 200 None 2026-04-03 05:56:50,752 - lava_callback - ERROR - Error processing callback for node 69ccb3948cd9cbcdc945bf31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57938 - "GET / HTTP/1.0" 200 OK 2026-04-03 05:59:10,769 - lava_callback - INFO - Received callback for node 69ccb4298cd9cbcdc945bf5a INFO: 172.21.0.1:54230 - "POST /node/69ccb4298cd9cbcdc945bf5a HTTP/1.0" 202 Accepted 2026-04-03 05:59:10,774 - lava_callback - INFO - Starting async job submit for node 69ccb4298cd9cbcdc945bf5a 2026-04-03 05:59:10,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:59:10,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb4298cd9cbcdc945bf5a HTTP/1.1" 200 None 2026-04-03 05:59:10,790 - lava_callback - DEBUG - Processing node 69ccb4298cd9cbcdc945bf5a: baseline-arm64-baylibre 2026-04-03 05:59:10,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb4298cd9cbcdc945bf5a 2026-04-03 05:59:10,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:59:10,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb4298cd9cbcdc945bf5a HTTP/1.1" 200 None 2026-04-03 05:59:10,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:59:10,819 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 05:59:10,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 05:59:10,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb4298cd9cbcdc945bf5a HTTP/1.1" 200 None 2026-04-03 05:59:10,842 - lava_callback - ERROR - Error processing callback for node 69ccb4298cd9cbcdc945bf5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46622 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:02:10,947 - lava_callback - INFO - Received callback for node 69ccb4da8cd9cbcdc945bfa9 INFO: 172.21.0.1:47484 - "POST /node/69ccb4da8cd9cbcdc945bfa9 HTTP/1.0" 202 Accepted 2026-04-03 06:02:10,953 - lava_callback - INFO - Starting async job submit for node 69ccb4da8cd9cbcdc945bfa9 2026-04-03 06:02:10,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:02:10,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb4da8cd9cbcdc945bfa9 HTTP/1.1" 200 None 2026-04-03 06:02:10,989 - lava_callback - DEBUG - Processing node 69ccb4da8cd9cbcdc945bfa9: baseline-arm64-baylibre 2026-04-03 06:02:10,992 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb4da8cd9cbcdc945bfa9 2026-04-03 06:02:10,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:02:11,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb4da8cd9cbcdc945bfa9 HTTP/1.1" 200 None 2026-04-03 06:02:11,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:02:11,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:02:11,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:02:11,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb4da8cd9cbcdc945bfa9 HTTP/1.1" 200 None 2026-04-03 06:02:11,119 - lava_callback - ERROR - Error processing callback for node 69ccb4da8cd9cbcdc945bfa9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60352 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:03:45,251 - lava_callback - INFO - Received callback for node 69ccb5388cd9cbcdc945bfd4 INFO: 172.21.0.1:60636 - "POST /node/69ccb5388cd9cbcdc945bfd4 HTTP/1.0" 202 Accepted 2026-04-03 06:03:45,257 - lava_callback - INFO - Starting async job submit for node 69ccb5388cd9cbcdc945bfd4 2026-04-03 06:03:45,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:03:45,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb5388cd9cbcdc945bfd4 HTTP/1.1" 200 None 2026-04-03 06:03:45,272 - lava_callback - DEBUG - Processing node 69ccb5388cd9cbcdc945bfd4: baseline-arm64-clabbe 2026-04-03 06:03:45,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb5388cd9cbcdc945bfd4 2026-04-03 06:03:45,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:03:45,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb5388cd9cbcdc945bfd4 HTTP/1.1" 200 None 2026-04-03 06:03:45,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:03:45,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:03:45,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:03:45,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb5388cd9cbcdc945bfd4 HTTP/1.1" 200 None 2026-04-03 06:03:45,327 - lava_callback - ERROR - Error processing callback for node 69ccb5388cd9cbcdc945bfd4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51890 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:05:05,154 - lava_callback - INFO - Received callback for node 69ccb58d8cd9cbcdc945c001 INFO: 172.21.0.1:59532 - "POST /node/69ccb58d8cd9cbcdc945c001 HTTP/1.0" 202 Accepted 2026-04-03 06:05:05,159 - lava_callback - INFO - Starting async job submit for node 69ccb58d8cd9cbcdc945c001 2026-04-03 06:05:05,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:05:05,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb58d8cd9cbcdc945c001 HTTP/1.1" 200 None 2026-04-03 06:05:05,176 - lava_callback - DEBUG - Processing node 69ccb58d8cd9cbcdc945c001: baseline-arm64-clabbe 2026-04-03 06:05:05,176 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb58d8cd9cbcdc945c001 2026-04-03 06:05:05,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:05:05,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb58d8cd9cbcdc945c001 HTTP/1.1" 200 None 2026-04-03 06:05:05,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:05:05,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:05:05,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:05:05,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb58d8cd9cbcdc945c001 HTTP/1.1" 200 None 2026-04-03 06:05:05,233 - lava_callback - ERROR - Error processing callback for node 69ccb58d8cd9cbcdc945c001: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44454 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:06:31,202 - lava_callback - INFO - Received callback for node 69ccb5de8cd9cbcdc945c02c INFO: 172.21.0.1:36388 - "POST /node/69ccb5de8cd9cbcdc945c02c HTTP/1.0" 202 Accepted 2026-04-03 06:06:31,207 - lava_callback - INFO - Starting async job submit for node 69ccb5de8cd9cbcdc945c02c 2026-04-03 06:06:31,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:06:31,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb5de8cd9cbcdc945c02c HTTP/1.1" 200 None 2026-04-03 06:06:31,223 - lava_callback - DEBUG - Processing node 69ccb5de8cd9cbcdc945c02c: baseline-arm64-baylibre 2026-04-03 06:06:31,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb5de8cd9cbcdc945c02c 2026-04-03 06:06:31,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:06:31,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb5de8cd9cbcdc945c02c HTTP/1.1" 200 None 2026-04-03 06:06:31,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:06:31,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:06:31,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:06:31,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb5de8cd9cbcdc945c02c HTTP/1.1" 200 None 2026-04-03 06:06:31,274 - lava_callback - ERROR - Error processing callback for node 69ccb5de8cd9cbcdc945c02c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39440 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:08:11,238 - lava_callback - INFO - Received callback for node 69ccb6368cd9cbcdc945c058 INFO: 172.21.0.1:35464 - "POST /node/69ccb6368cd9cbcdc945c058 HTTP/1.0" 202 Accepted 2026-04-03 06:08:11,244 - lava_callback - INFO - Starting async job submit for node 69ccb6368cd9cbcdc945c058 2026-04-03 06:08:11,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:08:11,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb6368cd9cbcdc945c058 HTTP/1.1" 200 None 2026-04-03 06:08:11,262 - lava_callback - DEBUG - Processing node 69ccb6368cd9cbcdc945c058: baseline-arm64-baylibre 2026-04-03 06:08:11,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb6368cd9cbcdc945c058 2026-04-03 06:08:11,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:08:11,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb6368cd9cbcdc945c058 HTTP/1.1" 200 None 2026-04-03 06:08:11,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:08:11,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:08:11,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:08:11,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb6368cd9cbcdc945c058 HTTP/1.1" 200 None 2026-04-03 06:08:11,347 - lava_callback - ERROR - Error processing callback for node 69ccb6368cd9cbcdc945c058: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58180 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:09:45,458 - lava_callback - INFO - Received callback for node 69ccb6938cd9cbcdc945c082 INFO: 172.21.0.1:50808 - "POST /node/69ccb6938cd9cbcdc945c082 HTTP/1.0" 202 Accepted 2026-04-03 06:09:45,464 - lava_callback - INFO - Starting async job submit for node 69ccb6938cd9cbcdc945c082 2026-04-03 06:09:45,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:09:45,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb6938cd9cbcdc945c082 HTTP/1.1" 200 None 2026-04-03 06:09:45,480 - lava_callback - DEBUG - Processing node 69ccb6938cd9cbcdc945c082: baseline-arm64-clabbe 2026-04-03 06:09:45,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb6938cd9cbcdc945c082 2026-04-03 06:09:45,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:09:45,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb6938cd9cbcdc945c082 HTTP/1.1" 200 None 2026-04-03 06:09:45,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:09:45,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:09:45,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:09:45,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb6938cd9cbcdc945c082 HTTP/1.1" 200 None 2026-04-03 06:09:45,557 - lava_callback - ERROR - Error processing callback for node 69ccb6938cd9cbcdc945c082: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58604 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:11:05,501 - lava_callback - INFO - Received callback for node 69ccb6ea8cd9cbcdc945c0b0 INFO: 172.21.0.1:43864 - "POST /node/69ccb6ea8cd9cbcdc945c0b0 HTTP/1.0" 202 Accepted 2026-04-03 06:11:05,506 - lava_callback - INFO - Starting async job submit for node 69ccb6ea8cd9cbcdc945c0b0 2026-04-03 06:11:05,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:11:05,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb6ea8cd9cbcdc945c0b0 HTTP/1.1" 200 None 2026-04-03 06:11:05,522 - lava_callback - DEBUG - Processing node 69ccb6ea8cd9cbcdc945c0b0: baseline-arm64-clabbe 2026-04-03 06:11:05,523 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb6ea8cd9cbcdc945c0b0 2026-04-03 06:11:05,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:11:05,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb6ea8cd9cbcdc945c0b0 HTTP/1.1" 200 None 2026-04-03 06:11:05,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:11:05,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:11:05,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:11:05,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb6ea8cd9cbcdc945c0b0 HTTP/1.1" 200 None 2026-04-03 06:11:05,574 - lava_callback - ERROR - Error processing callback for node 69ccb6ea8cd9cbcdc945c0b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53594 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:12:25,538 - lava_callback - INFO - Received callback for node 69ccb7418cd9cbcdc945c0d4 INFO: 172.21.0.1:39572 - "POST /node/69ccb7418cd9cbcdc945c0d4 HTTP/1.0" 202 Accepted 2026-04-03 06:12:25,544 - lava_callback - INFO - Starting async job submit for node 69ccb7418cd9cbcdc945c0d4 2026-04-03 06:12:25,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:12:25,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb7418cd9cbcdc945c0d4 HTTP/1.1" 200 None 2026-04-03 06:12:25,560 - lava_callback - DEBUG - Processing node 69ccb7418cd9cbcdc945c0d4: baseline-arm64-clabbe 2026-04-03 06:12:25,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb7418cd9cbcdc945c0d4 2026-04-03 06:12:25,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:12:25,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb7418cd9cbcdc945c0d4 HTTP/1.1" 200 None 2026-04-03 06:12:25,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:12:25,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:12:25,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:12:25,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb7418cd9cbcdc945c0d4 HTTP/1.1" 200 None 2026-04-03 06:12:25,618 - lava_callback - ERROR - Error processing callback for node 69ccb7418cd9cbcdc945c0d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54054 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:14:05,582 - lava_callback - INFO - Received callback for node 69ccb7978cd9cbcdc945c0fa INFO: 172.21.0.1:56610 - "POST /node/69ccb7978cd9cbcdc945c0fa HTTP/1.0" 202 Accepted 2026-04-03 06:14:05,587 - lava_callback - INFO - Starting async job submit for node 69ccb7978cd9cbcdc945c0fa 2026-04-03 06:14:05,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:14:05,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb7978cd9cbcdc945c0fa HTTP/1.1" 200 None 2026-04-03 06:14:05,604 - lava_callback - DEBUG - Processing node 69ccb7978cd9cbcdc945c0fa: baseline-arm64-clabbe 2026-04-03 06:14:05,604 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb7978cd9cbcdc945c0fa 2026-04-03 06:14:05,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:14:05,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb7978cd9cbcdc945c0fa HTTP/1.1" 200 None 2026-04-03 06:14:05,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:14:05,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:14:05,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:14:05,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb7978cd9cbcdc945c0fa HTTP/1.1" 200 None 2026-04-03 06:14:05,670 - lava_callback - ERROR - Error processing callback for node 69ccb7978cd9cbcdc945c0fa: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:15:25,684 - lava_callback - INFO - Received callback for node 69ccb7ef8cd9cbcdc945c11f INFO: 172.21.0.1:50198 - "POST /node/69ccb7ef8cd9cbcdc945c11f HTTP/1.0" 202 Accepted 2026-04-03 06:15:25,690 - lava_callback - INFO - Starting async job submit for node 69ccb7ef8cd9cbcdc945c11f 2026-04-03 06:15:25,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:15:25,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb7ef8cd9cbcdc945c11f HTTP/1.1" 200 None 2026-04-03 06:15:25,706 - lava_callback - DEBUG - Processing node 69ccb7ef8cd9cbcdc945c11f: baseline-arm64-clabbe 2026-04-03 06:15:25,706 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb7ef8cd9cbcdc945c11f 2026-04-03 06:15:25,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:15:25,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb7ef8cd9cbcdc945c11f HTTP/1.1" 200 None 2026-04-03 06:15:25,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:15:25,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:15:25,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:15:25,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb7ef8cd9cbcdc945c11f HTTP/1.1" 200 None 2026-04-03 06:15:25,763 - lava_callback - ERROR - Error processing callback for node 69ccb7ef8cd9cbcdc945c11f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49590 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:16:45,763 - lava_callback - INFO - Received callback for node 69ccb8468cd9cbcdc945c141 INFO: 172.21.0.1:56348 - "POST /node/69ccb8468cd9cbcdc945c141 HTTP/1.0" 202 Accepted 2026-04-03 06:16:45,768 - lava_callback - INFO - Starting async job submit for node 69ccb8468cd9cbcdc945c141 2026-04-03 06:16:45,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:16:45,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb8468cd9cbcdc945c141 HTTP/1.1" 200 None 2026-04-03 06:16:45,793 - lava_callback - DEBUG - Processing node 69ccb8468cd9cbcdc945c141: baseline-arm64-clabbe 2026-04-03 06:16:45,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb8468cd9cbcdc945c141 2026-04-03 06:16:45,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:16:45,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb8468cd9cbcdc945c141 HTTP/1.1" 200 None 2026-04-03 06:16:45,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:16:45,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:16:45,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:16:45,868 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb8468cd9cbcdc945c141 HTTP/1.1" 200 None 2026-04-03 06:16:45,869 - lava_callback - ERROR - Error processing callback for node 69ccb8468cd9cbcdc945c141: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46418 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:18:25,809 - lava_callback - INFO - Received callback for node 69ccb89c8cd9cbcdc945c15c INFO: 172.21.0.1:35888 - "POST /node/69ccb89c8cd9cbcdc945c15c HTTP/1.0" 202 Accepted 2026-04-03 06:18:25,815 - lava_callback - INFO - Starting async job submit for node 69ccb89c8cd9cbcdc945c15c 2026-04-03 06:18:25,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:18:25,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb89c8cd9cbcdc945c15c HTTP/1.1" 200 None 2026-04-03 06:18:25,834 - lava_callback - DEBUG - Processing node 69ccb89c8cd9cbcdc945c15c: baseline-arm64-clabbe 2026-04-03 06:18:25,835 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb89c8cd9cbcdc945c15c 2026-04-03 06:18:25,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:18:25,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb89c8cd9cbcdc945c15c HTTP/1.1" 200 None 2026-04-03 06:18:25,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:18:25,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:18:25,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:18:25,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb89c8cd9cbcdc945c15c HTTP/1.1" 200 None 2026-04-03 06:18:25,893 - lava_callback - ERROR - Error processing callback for node 69ccb89c8cd9cbcdc945c15c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43430 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42970 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:19:45,883 - lava_callback - INFO - Received callback for node 69ccb8f48cd9cbcdc945c189 INFO: 172.21.0.1:56490 - "POST /node/69ccb8f48cd9cbcdc945c189 HTTP/1.0" 202 Accepted 2026-04-03 06:19:45,888 - lava_callback - INFO - Starting async job submit for node 69ccb8f48cd9cbcdc945c189 2026-04-03 06:19:45,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:19:45,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb8f48cd9cbcdc945c189 HTTP/1.1" 200 None 2026-04-03 06:19:45,906 - lava_callback - DEBUG - Processing node 69ccb8f48cd9cbcdc945c189: baseline-arm64-clabbe 2026-04-03 06:19:45,906 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb8f48cd9cbcdc945c189 2026-04-03 06:19:45,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:19:45,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb8f48cd9cbcdc945c189 HTTP/1.1" 200 None 2026-04-03 06:19:45,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:19:45,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:19:45,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:19:45,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb8f48cd9cbcdc945c189 HTTP/1.1" 200 None 2026-04-03 06:19:45,962 - lava_callback - ERROR - Error processing callback for node 69ccb8f48cd9cbcdc945c189: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34498 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53486 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:21:05,920 - lava_callback - INFO - Received callback for node 69ccb94a8cd9cbcdc945c1bb INFO: 172.21.0.1:36292 - "POST /node/69ccb94a8cd9cbcdc945c1bb HTTP/1.0" 202 Accepted 2026-04-03 06:21:05,926 - lava_callback - INFO - Starting async job submit for node 69ccb94a8cd9cbcdc945c1bb 2026-04-03 06:21:05,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:21:05,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb94a8cd9cbcdc945c1bb HTTP/1.1" 200 None 2026-04-03 06:21:05,942 - lava_callback - DEBUG - Processing node 69ccb94a8cd9cbcdc945c1bb: baseline-arm64-clabbe 2026-04-03 06:21:05,943 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb94a8cd9cbcdc945c1bb 2026-04-03 06:21:05,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:21:05,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb94a8cd9cbcdc945c1bb HTTP/1.1" 200 None 2026-04-03 06:21:05,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:21:05,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:21:05,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:21:06,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb94a8cd9cbcdc945c1bb HTTP/1.1" 200 None 2026-04-03 06:21:06,005 - lava_callback - ERROR - Error processing callback for node 69ccb94a8cd9cbcdc945c1bb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60402 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:22:46,052 - lava_callback - INFO - Received callback for node 69ccb9a38cd9cbcdc945c1e0 INFO: 172.21.0.1:52078 - "POST /node/69ccb9a38cd9cbcdc945c1e0 HTTP/1.0" 202 Accepted 2026-04-03 06:22:46,057 - lava_callback - INFO - Starting async job submit for node 69ccb9a38cd9cbcdc945c1e0 2026-04-03 06:22:46,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:22:46,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb9a38cd9cbcdc945c1e0 HTTP/1.1" 200 None 2026-04-03 06:22:46,073 - lava_callback - DEBUG - Processing node 69ccb9a38cd9cbcdc945c1e0: baseline-arm64-clabbe 2026-04-03 06:22:46,073 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccb9a38cd9cbcdc945c1e0 2026-04-03 06:22:46,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:22:46,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccb9a38cd9cbcdc945c1e0 HTTP/1.1" 200 None 2026-04-03 06:22:46,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:22:46,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c950b6f2866272ea51b8d9 HTTP/1.1" 200 None 2026-04-03 06:22:46,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:22:46,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccb9a38cd9cbcdc945c1e0 HTTP/1.1" 200 None 2026-04-03 06:22:46,146 - lava_callback - ERROR - Error processing callback for node 69ccb9a38cd9cbcdc945c1e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39162 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37188 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38014 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:37:32,982 - lava_callback - INFO - Received callback for node 69ccbd268cd9cbcdc945c242 INFO: 172.21.0.1:38790 - "POST /node/69ccbd268cd9cbcdc945c242 HTTP/1.0" 202 Accepted 2026-04-03 06:37:32,991 - lava_callback - INFO - Starting async job submit for node 69ccbd268cd9cbcdc945c242 2026-04-03 06:37:32,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:37:33,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbd268cd9cbcdc945c242 HTTP/1.1" 200 None 2026-04-03 06:37:33,008 - lava_callback - DEBUG - Processing node 69ccbd268cd9cbcdc945c242: baseline-arm-baylibre 2026-04-03 06:37:33,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbd268cd9cbcdc945c242 2026-04-03 06:37:33,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:37:33,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbd268cd9cbcdc945c242 HTTP/1.1" 200 None 2026-04-03 06:37:33,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:37:33,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 06:37:33,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:37:33,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbd268cd9cbcdc945c242 HTTP/1.1" 200 None 2026-04-03 06:37:33,079 - lava_callback - ERROR - Error processing callback for node 69ccbd268cd9cbcdc945c242: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39414 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:38:12,939 - lava_callback - INFO - Received callback for node 69ccbd4b8cd9cbcdc945c259 INFO: 172.21.0.1:34664 - "POST /node/69ccbd4b8cd9cbcdc945c259 HTTP/1.0" 202 Accepted 2026-04-03 06:38:12,944 - lava_callback - INFO - Starting async job submit for node 69ccbd4b8cd9cbcdc945c259 2026-04-03 06:38:12,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:38:12,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbd4b8cd9cbcdc945c259 HTTP/1.1" 200 None 2026-04-03 06:38:12,961 - lava_callback - DEBUG - Processing node 69ccbd4b8cd9cbcdc945c259: baseline-arm-baylibre 2026-04-03 06:38:12,961 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbd4b8cd9cbcdc945c259 2026-04-03 06:38:12,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:38:12,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbd4b8cd9cbcdc945c259 HTTP/1.1" 200 None 2026-04-03 06:38:12,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:38:13,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 06:38:13,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:38:13,043 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbd4b8cd9cbcdc945c259 HTTP/1.1" 200 None 2026-04-03 06:38:13,046 - lava_callback - ERROR - Error processing callback for node 69ccbd4b8cd9cbcdc945c259: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48708 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:40:46,886 - lava_callback - INFO - Received callback for node 69ccbddd8cd9cbcdc945c289 INFO: 172.21.0.1:48720 - "POST /node/69ccbddd8cd9cbcdc945c289 HTTP/1.0" 202 Accepted 2026-04-03 06:40:46,892 - lava_callback - INFO - Starting async job submit for node 69ccbddd8cd9cbcdc945c289 2026-04-03 06:40:46,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:40:46,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbddd8cd9cbcdc945c289 HTTP/1.1" 200 None 2026-04-03 06:40:46,908 - lava_callback - DEBUG - Processing node 69ccbddd8cd9cbcdc945c289: baseline-arm-clabbe 2026-04-03 06:40:46,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbddd8cd9cbcdc945c289 2026-04-03 06:40:46,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:40:46,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbddd8cd9cbcdc945c289 HTTP/1.1" 200 None 2026-04-03 06:40:46,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:40:46,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 06:40:46,954 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:40:46,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbddd8cd9cbcdc945c289 HTTP/1.1" 200 None 2026-04-03 06:40:46,977 - lava_callback - ERROR - Error processing callback for node 69ccbddd8cd9cbcdc945c289: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40398 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34688 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:41:26,805 - lava_callback - INFO - Received callback for node 69ccbe028cd9cbcdc945c296 INFO: 172.21.0.1:34694 - "POST /node/69ccbe028cd9cbcdc945c296 HTTP/1.0" 202 Accepted 2026-04-03 06:41:26,814 - lava_callback - INFO - Starting async job submit for node 69ccbe028cd9cbcdc945c296 2026-04-03 06:41:26,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:41:26,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbe028cd9cbcdc945c296 HTTP/1.1" 200 None 2026-04-03 06:41:26,837 - lava_callback - DEBUG - Processing node 69ccbe028cd9cbcdc945c296: baseline-arm-clabbe 2026-04-03 06:41:26,838 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbe028cd9cbcdc945c296 2026-04-03 06:41:26,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:41:26,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbe028cd9cbcdc945c296 HTTP/1.1" 200 None 2026-04-03 06:41:26,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:41:26,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 06:41:26,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:41:26,889 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbe028cd9cbcdc945c296 HTTP/1.1" 200 None 2026-04-03 06:41:26,899 - lava_callback - ERROR - Error processing callback for node 69ccbe028cd9cbcdc945c296: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45224 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:41:46,883 - lava_callback - INFO - Received callback for node 69ccbe278cd9cbcdc945c2a5 INFO: 172.21.0.1:45238 - "POST /node/69ccbe278cd9cbcdc945c2a5 HTTP/1.0" 202 Accepted 2026-04-03 06:41:46,889 - lava_callback - INFO - Starting async job submit for node 69ccbe278cd9cbcdc945c2a5 2026-04-03 06:41:46,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:41:46,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbe278cd9cbcdc945c2a5 HTTP/1.1" 200 None 2026-04-03 06:41:46,906 - lava_callback - DEBUG - Processing node 69ccbe278cd9cbcdc945c2a5: baseline-arm-clabbe 2026-04-03 06:41:46,906 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbe278cd9cbcdc945c2a5 2026-04-03 06:41:46,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:41:46,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbe278cd9cbcdc945c2a5 HTTP/1.1" 200 None 2026-04-03 06:41:46,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:41:46,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 06:41:46,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:41:46,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbe278cd9cbcdc945c2a5 HTTP/1.1" 200 None 2026-04-03 06:41:46,961 - lava_callback - ERROR - Error processing callback for node 69ccbe278cd9cbcdc945c2a5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39500 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:42:26,852 - lava_callback - INFO - Received callback for node 69ccbe4c8cd9cbcdc945c2b0 INFO: 172.21.0.1:39504 - "POST /node/69ccbe4c8cd9cbcdc945c2b0 HTTP/1.0" 202 Accepted 2026-04-03 06:42:26,857 - lava_callback - INFO - Starting async job submit for node 69ccbe4c8cd9cbcdc945c2b0 2026-04-03 06:42:26,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:42:26,875 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbe4c8cd9cbcdc945c2b0 HTTP/1.1" 200 None 2026-04-03 06:42:26,876 - lava_callback - DEBUG - Processing node 69ccbe4c8cd9cbcdc945c2b0: baseline-arm-clabbe 2026-04-03 06:42:26,876 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbe4c8cd9cbcdc945c2b0 2026-04-03 06:42:26,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:42:26,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbe4c8cd9cbcdc945c2b0 HTTP/1.1" 200 None 2026-04-03 06:42:26,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:42:26,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 06:42:26,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:42:26,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbe4c8cd9cbcdc945c2b0 HTTP/1.1" 200 None 2026-04-03 06:42:26,933 - lava_callback - ERROR - Error processing callback for node 69ccbe4c8cd9cbcdc945c2b0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45606 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:43:06,961 - lava_callback - INFO - Received callback for node 69ccbe718cd9cbcdc945c2bd INFO: 172.21.0.1:45608 - "POST /node/69ccbe718cd9cbcdc945c2bd HTTP/1.0" 202 Accepted 2026-04-03 06:43:06,967 - lava_callback - INFO - Starting async job submit for node 69ccbe718cd9cbcdc945c2bd 2026-04-03 06:43:06,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:43:06,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbe718cd9cbcdc945c2bd HTTP/1.1" 200 None 2026-04-03 06:43:06,982 - lava_callback - DEBUG - Processing node 69ccbe718cd9cbcdc945c2bd: baseline-arm-clabbe 2026-04-03 06:43:06,982 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbe718cd9cbcdc945c2bd 2026-04-03 06:43:06,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:43:06,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbe718cd9cbcdc945c2bd HTTP/1.1" 200 None 2026-04-03 06:43:06,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:43:07,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 06:43:07,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:43:07,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbe718cd9cbcdc945c2bd HTTP/1.1" 200 None 2026-04-03 06:43:07,045 - lava_callback - ERROR - Error processing callback for node 69ccbe718cd9cbcdc945c2bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60962 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:43:46,914 - lava_callback - INFO - Received callback for node 69ccbe968cd9cbcdc945c2cc INFO: 172.21.0.1:60966 - "POST /node/69ccbe968cd9cbcdc945c2cc HTTP/1.0" 202 Accepted 2026-04-03 06:43:46,920 - lava_callback - INFO - Starting async job submit for node 69ccbe968cd9cbcdc945c2cc 2026-04-03 06:43:46,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:43:46,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbe968cd9cbcdc945c2cc HTTP/1.1" 200 None 2026-04-03 06:43:46,936 - lava_callback - DEBUG - Processing node 69ccbe968cd9cbcdc945c2cc: baseline-arm-clabbe 2026-04-03 06:43:46,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbe968cd9cbcdc945c2cc 2026-04-03 06:43:46,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:43:46,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbe968cd9cbcdc945c2cc HTTP/1.1" 200 None 2026-04-03 06:43:46,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:43:46,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc647b8cd9cbcdc9459647 HTTP/1.1" 200 None 2026-04-03 06:43:46,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:43:47,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbe968cd9cbcdc945c2cc HTTP/1.1" 200 None 2026-04-03 06:43:47,027 - lava_callback - ERROR - Error processing callback for node 69ccbe968cd9cbcdc945c2cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37176 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:44:13,356 - lava_callback - INFO - Received callback for node 69ccbeb78cd9cbcdc945c2da INFO: 172.21.0.1:33692 - "POST /node/69ccbeb78cd9cbcdc945c2da HTTP/1.0" 202 Accepted 2026-04-03 06:44:13,362 - lava_callback - INFO - Starting async job submit for node 69ccbeb78cd9cbcdc945c2da 2026-04-03 06:44:13,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:44:13,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbeb78cd9cbcdc945c2da HTTP/1.1" 200 None 2026-04-03 06:44:13,378 - lava_callback - DEBUG - Processing node 69ccbeb78cd9cbcdc945c2da: baseline-arm-baylibre 2026-04-03 06:44:13,378 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbeb78cd9cbcdc945c2da 2026-04-03 06:44:13,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:44:13,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbeb78cd9cbcdc945c2da HTTP/1.1" 200 None 2026-04-03 06:44:13,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:44:13,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:44:13,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:44:13,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbeb78cd9cbcdc945c2da HTTP/1.1" 200 None 2026-04-03 06:44:13,444 - lava_callback - ERROR - Error processing callback for node 69ccbeb78cd9cbcdc945c2da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33700 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55770 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:44:53,292 - lava_callback - INFO - Received callback for node 69ccbedb8cd9cbcdc945c2e9 INFO: 172.21.0.1:55776 - "POST /node/69ccbedb8cd9cbcdc945c2e9 HTTP/1.0" 202 Accepted 2026-04-03 06:44:53,297 - lava_callback - INFO - Starting async job submit for node 69ccbedb8cd9cbcdc945c2e9 2026-04-03 06:44:53,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:44:53,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbedb8cd9cbcdc945c2e9 HTTP/1.1" 200 None 2026-04-03 06:44:53,317 - lava_callback - DEBUG - Processing node 69ccbedb8cd9cbcdc945c2e9: baseline-arm-baylibre 2026-04-03 06:44:53,317 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbedb8cd9cbcdc945c2e9 2026-04-03 06:44:53,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:44:53,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbedb8cd9cbcdc945c2e9 HTTP/1.1" 200 None 2026-04-03 06:44:53,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:44:53,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:44:53,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:44:53,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbedb8cd9cbcdc945c2e9 HTTP/1.1" 200 None 2026-04-03 06:44:53,380 - lava_callback - ERROR - Error processing callback for node 69ccbedb8cd9cbcdc945c2e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53672 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:45:33,325 - lava_callback - INFO - Received callback for node 69ccbf008cd9cbcdc945c2f6 INFO: 172.21.0.1:55270 - "POST /node/69ccbf008cd9cbcdc945c2f6 HTTP/1.0" 202 Accepted 2026-04-03 06:45:33,330 - lava_callback - INFO - Starting async job submit for node 69ccbf008cd9cbcdc945c2f6 2026-04-03 06:45:33,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:45:33,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbf008cd9cbcdc945c2f6 HTTP/1.1" 200 None 2026-04-03 06:45:33,348 - lava_callback - DEBUG - Processing node 69ccbf008cd9cbcdc945c2f6: baseline-arm-baylibre 2026-04-03 06:45:33,348 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbf008cd9cbcdc945c2f6 2026-04-03 06:45:33,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:45:33,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbf008cd9cbcdc945c2f6 HTTP/1.1" 200 None 2026-04-03 06:45:33,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:45:33,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:45:33,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:45:33,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbf008cd9cbcdc945c2f6 HTTP/1.1" 200 None 2026-04-03 06:45:33,446 - lava_callback - ERROR - Error processing callback for node 69ccbf008cd9cbcdc945c2f6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57874 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:46:13,341 - lava_callback - INFO - Received callback for node 69ccbf268cd9cbcdc945c303 INFO: 172.21.0.1:41290 - "POST /node/69ccbf268cd9cbcdc945c303 HTTP/1.0" 202 Accepted 2026-04-03 06:46:13,346 - lava_callback - INFO - Starting async job submit for node 69ccbf268cd9cbcdc945c303 2026-04-03 06:46:13,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:46:13,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbf268cd9cbcdc945c303 HTTP/1.1" 200 None 2026-04-03 06:46:13,387 - lava_callback - DEBUG - Processing node 69ccbf268cd9cbcdc945c303: baseline-arm-baylibre 2026-04-03 06:46:13,387 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbf268cd9cbcdc945c303 2026-04-03 06:46:13,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:46:13,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbf268cd9cbcdc945c303 HTTP/1.1" 200 None 2026-04-03 06:46:13,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:46:13,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:46:13,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:46:13,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbf268cd9cbcdc945c303 HTTP/1.1" 200 None 2026-04-03 06:46:13,442 - lava_callback - ERROR - Error processing callback for node 69ccbf268cd9cbcdc945c303: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33544 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:46:53,453 - lava_callback - INFO - Received callback for node 69ccbf4c8cd9cbcdc945c30f INFO: 172.21.0.1:33546 - "POST /node/69ccbf4c8cd9cbcdc945c30f HTTP/1.0" 202 Accepted 2026-04-03 06:46:53,459 - lava_callback - INFO - Starting async job submit for node 69ccbf4c8cd9cbcdc945c30f 2026-04-03 06:46:53,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:46:53,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbf4c8cd9cbcdc945c30f HTTP/1.1" 200 None 2026-04-03 06:46:53,475 - lava_callback - DEBUG - Processing node 69ccbf4c8cd9cbcdc945c30f: baseline-arm-baylibre 2026-04-03 06:46:53,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbf4c8cd9cbcdc945c30f 2026-04-03 06:46:53,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:46:53,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbf4c8cd9cbcdc945c30f HTTP/1.1" 200 None 2026-04-03 06:46:53,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:46:53,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:46:53,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:46:53,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbf4c8cd9cbcdc945c30f HTTP/1.1" 200 None 2026-04-03 06:46:53,534 - lava_callback - ERROR - Error processing callback for node 69ccbf4c8cd9cbcdc945c30f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35286 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:48:33,504 - lava_callback - INFO - Received callback for node 69ccbfb38cd9cbcdc945c32c INFO: 172.21.0.1:46800 - "POST /node/69ccbfb38cd9cbcdc945c32c HTTP/1.0" 202 Accepted 2026-04-03 06:48:33,510 - lava_callback - INFO - Starting async job submit for node 69ccbfb38cd9cbcdc945c32c 2026-04-03 06:48:33,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:48:33,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbfb38cd9cbcdc945c32c HTTP/1.1" 200 None 2026-04-03 06:48:33,526 - lava_callback - DEBUG - Processing node 69ccbfb38cd9cbcdc945c32c: baseline-arm-baylibre 2026-04-03 06:48:33,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbfb38cd9cbcdc945c32c 2026-04-03 06:48:33,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:48:33,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbfb38cd9cbcdc945c32c HTTP/1.1" 200 None 2026-04-03 06:48:33,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:48:33,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:48:33,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:48:33,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbfb38cd9cbcdc945c32c HTTP/1.1" 200 None 2026-04-03 06:48:33,581 - lava_callback - ERROR - Error processing callback for node 69ccbfb38cd9cbcdc945c32c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48246 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:49:07,263 - lava_callback - INFO - Received callback for node 69ccbfde8cd9cbcdc945c340 INFO: 172.21.0.1:48258 - "POST /node/69ccbfde8cd9cbcdc945c340 HTTP/1.0" 202 Accepted 2026-04-03 06:49:07,269 - lava_callback - INFO - Starting async job submit for node 69ccbfde8cd9cbcdc945c340 2026-04-03 06:49:07,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:49:07,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbfde8cd9cbcdc945c340 HTTP/1.1" 200 None 2026-04-03 06:49:07,284 - lava_callback - DEBUG - Processing node 69ccbfde8cd9cbcdc945c340: baseline-arm-clabbe 2026-04-03 06:49:07,284 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccbfde8cd9cbcdc945c340 2026-04-03 06:49:07,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:49:07,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccbfde8cd9cbcdc945c340 HTTP/1.1" 200 None 2026-04-03 06:49:07,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:49:07,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:49:07,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:49:07,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccbfde8cd9cbcdc945c340 HTTP/1.1" 200 None 2026-04-03 06:49:07,336 - lava_callback - ERROR - Error processing callback for node 69ccbfde8cd9cbcdc945c340: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36882 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:49:47,269 - lava_callback - INFO - Received callback for node 69ccc0038cd9cbcdc945c35d INFO: 172.21.0.1:36898 - "POST /node/69ccc0038cd9cbcdc945c35d HTTP/1.0" 202 Accepted 2026-04-03 06:49:47,274 - lava_callback - INFO - Starting async job submit for node 69ccc0038cd9cbcdc945c35d 2026-04-03 06:49:47,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:49:47,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc0038cd9cbcdc945c35d HTTP/1.1" 200 None 2026-04-03 06:49:47,290 - lava_callback - DEBUG - Processing node 69ccc0038cd9cbcdc945c35d: baseline-arm-clabbe 2026-04-03 06:49:47,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc0038cd9cbcdc945c35d 2026-04-03 06:49:47,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:49:47,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc0038cd9cbcdc945c35d HTTP/1.1" 200 None 2026-04-03 06:49:47,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:49:47,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:49:47,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:49:47,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc0038cd9cbcdc945c35d HTTP/1.1" 200 None 2026-04-03 06:49:47,347 - lava_callback - ERROR - Error processing callback for node 69ccc0038cd9cbcdc945c35d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56232 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:51:07,344 - lava_callback - INFO - Received callback for node 69ccc04b8cd9cbcdc945c374 INFO: 172.21.0.1:56242 - "POST /node/69ccc04b8cd9cbcdc945c374 HTTP/1.0" 202 Accepted 2026-04-03 06:51:07,350 - lava_callback - INFO - Starting async job submit for node 69ccc04b8cd9cbcdc945c374 2026-04-03 06:51:07,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:51:07,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc04b8cd9cbcdc945c374 HTTP/1.1" 200 None 2026-04-03 06:51:07,367 - lava_callback - DEBUG - Processing node 69ccc04b8cd9cbcdc945c374: baseline-arm-clabbe 2026-04-03 06:51:07,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc04b8cd9cbcdc945c374 2026-04-03 06:51:07,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:51:07,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc04b8cd9cbcdc945c374 HTTP/1.1" 200 None 2026-04-03 06:51:07,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:51:07,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:51:07,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:51:07,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc04b8cd9cbcdc945c374 HTTP/1.1" 200 None 2026-04-03 06:51:07,418 - lava_callback - ERROR - Error processing callback for node 69ccc04b8cd9cbcdc945c374: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38282 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:52:47,619 - lava_callback - INFO - Received callback for node 69ccc0bc8cd9cbcdc945c3a8 INFO: 172.21.0.1:38292 - "POST /node/69ccc0bc8cd9cbcdc945c3a8 HTTP/1.0" 202 Accepted 2026-04-03 06:52:47,624 - lava_callback - INFO - Starting async job submit for node 69ccc0bc8cd9cbcdc945c3a8 2026-04-03 06:52:47,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:52:47,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc0bc8cd9cbcdc945c3a8 HTTP/1.1" 200 None 2026-04-03 06:52:47,641 - lava_callback - DEBUG - Processing node 69ccc0bc8cd9cbcdc945c3a8: baseline-arm-clabbe 2026-04-03 06:52:47,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc0bc8cd9cbcdc945c3a8 2026-04-03 06:52:47,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:52:47,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc0bc8cd9cbcdc945c3a8 HTTP/1.1" 200 None 2026-04-03 06:52:47,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:52:47,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:52:47,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:52:47,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc0bc8cd9cbcdc945c3a8 HTTP/1.1" 200 None 2026-04-03 06:52:47,725 - lava_callback - ERROR - Error processing callback for node 69ccc0bc8cd9cbcdc945c3a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54112 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47014 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:53:27,422 - lava_callback - INFO - Received callback for node 69ccc0e18cd9cbcdc945c3ba INFO: 172.21.0.1:47020 - "POST /node/69ccc0e18cd9cbcdc945c3ba HTTP/1.0" 202 Accepted 2026-04-03 06:53:27,428 - lava_callback - INFO - Starting async job submit for node 69ccc0e18cd9cbcdc945c3ba 2026-04-03 06:53:27,429 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:53:27,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc0e18cd9cbcdc945c3ba HTTP/1.1" 200 None 2026-04-03 06:53:27,444 - lava_callback - DEBUG - Processing node 69ccc0e18cd9cbcdc945c3ba: baseline-arm-clabbe 2026-04-03 06:53:27,444 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc0e18cd9cbcdc945c3ba 2026-04-03 06:53:27,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:53:27,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc0e18cd9cbcdc945c3ba HTTP/1.1" 200 None 2026-04-03 06:53:27,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:53:27,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:53:27,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:53:27,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc0e18cd9cbcdc945c3ba HTTP/1.1" 200 None 2026-04-03 06:53:27,500 - lava_callback - ERROR - Error processing callback for node 69ccc0e18cd9cbcdc945c3ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45592 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:54:13,901 - lava_callback - INFO - Received callback for node 69ccc1038cd9cbcdc945c3c6 INFO: 172.21.0.1:52600 - "POST /node/69ccc1038cd9cbcdc945c3c6 HTTP/1.0" 202 Accepted 2026-04-03 06:54:13,906 - lava_callback - INFO - Starting async job submit for node 69ccc1038cd9cbcdc945c3c6 2026-04-03 06:54:13,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:54:13,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc1038cd9cbcdc945c3c6 HTTP/1.1" 200 None 2026-04-03 06:54:13,922 - lava_callback - DEBUG - Processing node 69ccc1038cd9cbcdc945c3c6: baseline-arm-baylibre 2026-04-03 06:54:13,922 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc1038cd9cbcdc945c3c6 2026-04-03 06:54:13,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:54:13,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc1038cd9cbcdc945c3c6 HTTP/1.1" 200 None 2026-04-03 06:54:13,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:54:13,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:54:13,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:54:13,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc1038cd9cbcdc945c3c6 HTTP/1.1" 200 None 2026-04-03 06:54:13,985 - lava_callback - ERROR - Error processing callback for node 69ccc1038cd9cbcdc945c3c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43210 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:54:47,515 - lava_callback - INFO - Received callback for node 69ccc12b8cd9cbcdc945c3d8 INFO: 172.21.0.1:43216 - "POST /node/69ccc12b8cd9cbcdc945c3d8 HTTP/1.0" 202 Accepted 2026-04-03 06:54:47,520 - lava_callback - INFO - Starting async job submit for node 69ccc12b8cd9cbcdc945c3d8 2026-04-03 06:54:47,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:54:47,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc12b8cd9cbcdc945c3d8 HTTP/1.1" 200 None 2026-04-03 06:54:47,543 - lava_callback - DEBUG - Processing node 69ccc12b8cd9cbcdc945c3d8: baseline-arm-clabbe 2026-04-03 06:54:47,543 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc12b8cd9cbcdc945c3d8 2026-04-03 06:54:47,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:54:47,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc12b8cd9cbcdc945c3d8 HTTP/1.1" 200 None 2026-04-03 06:54:47,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:54:47,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:54:47,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:54:47,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc12b8cd9cbcdc945c3d8 HTTP/1.1" 200 None 2026-04-03 06:54:47,598 - lava_callback - ERROR - Error processing callback for node 69ccc12b8cd9cbcdc945c3d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40986 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44522 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:55:27,484 - lava_callback - INFO - Received callback for node 69ccc1508cd9cbcdc945c3e9 INFO: 172.21.0.1:44536 - "POST /node/69ccc1508cd9cbcdc945c3e9 HTTP/1.0" 202 Accepted 2026-04-03 06:55:27,490 - lava_callback - INFO - Starting async job submit for node 69ccc1508cd9cbcdc945c3e9 2026-04-03 06:55:27,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:55:27,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc1508cd9cbcdc945c3e9 HTTP/1.1" 200 None 2026-04-03 06:55:27,506 - lava_callback - DEBUG - Processing node 69ccc1508cd9cbcdc945c3e9: baseline-arm-clabbe 2026-04-03 06:55:27,506 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc1508cd9cbcdc945c3e9 2026-04-03 06:55:27,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:55:27,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc1508cd9cbcdc945c3e9 HTTP/1.1" 200 None 2026-04-03 06:55:27,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:55:27,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:55:27,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:55:27,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc1508cd9cbcdc945c3e9 HTTP/1.1" 200 None 2026-04-03 06:55:27,563 - lava_callback - ERROR - Error processing callback for node 69ccc1508cd9cbcdc945c3e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45562 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:55:54,083 - lava_callback - INFO - Received callback for node 69ccc1708cd9cbcdc945c3f3 INFO: 172.21.0.1:45574 - "POST /node/69ccc1708cd9cbcdc945c3f3 HTTP/1.0" 202 Accepted 2026-04-03 06:55:54,088 - lava_callback - INFO - Starting async job submit for node 69ccc1708cd9cbcdc945c3f3 2026-04-03 06:55:54,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:55:54,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc1708cd9cbcdc945c3f3 HTTP/1.1" 200 None 2026-04-03 06:55:54,104 - lava_callback - DEBUG - Processing node 69ccc1708cd9cbcdc945c3f3: baseline-arm-baylibre 2026-04-03 06:55:54,104 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc1708cd9cbcdc945c3f3 2026-04-03 06:55:54,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:55:54,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc1708cd9cbcdc945c3f3 HTTP/1.1" 200 None 2026-04-03 06:55:54,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:55:54,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 06:55:54,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:55:54,154 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc1708cd9cbcdc945c3f3 HTTP/1.1" 200 None 2026-04-03 06:55:54,156 - lava_callback - ERROR - Error processing callback for node 69ccc1708cd9cbcdc945c3f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54976 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:56:47,569 - lava_callback - INFO - Received callback for node 69ccc19b8cd9cbcdc945c402 INFO: 172.21.0.1:54990 - "POST /node/69ccc19b8cd9cbcdc945c402 HTTP/1.0" 202 Accepted 2026-04-03 06:56:47,575 - lava_callback - INFO - Starting async job submit for node 69ccc19b8cd9cbcdc945c402 2026-04-03 06:56:47,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:56:47,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc19b8cd9cbcdc945c402 HTTP/1.1" 200 None 2026-04-03 06:56:47,590 - lava_callback - DEBUG - Processing node 69ccc19b8cd9cbcdc945c402: baseline-arm-clabbe 2026-04-03 06:56:47,591 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc19b8cd9cbcdc945c402 2026-04-03 06:56:47,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:56:47,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc19b8cd9cbcdc945c402 HTTP/1.1" 200 None 2026-04-03 06:56:47,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:56:47,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:56:47,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:56:47,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc19b8cd9cbcdc945c402 HTTP/1.1" 200 None 2026-04-03 06:56:47,646 - lava_callback - ERROR - Error processing callback for node 69ccc19b8cd9cbcdc945c402: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35150 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:57:14,274 - lava_callback - INFO - Received callback for node 69ccc1bb8cd9cbcdc945c40f INFO: 172.21.0.1:36230 - "POST /node/69ccc1bb8cd9cbcdc945c40f HTTP/1.0" 202 Accepted 2026-04-03 06:57:14,279 - lava_callback - INFO - Starting async job submit for node 69ccc1bb8cd9cbcdc945c40f 2026-04-03 06:57:14,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:57:14,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc1bb8cd9cbcdc945c40f HTTP/1.1" 200 None 2026-04-03 06:57:14,295 - lava_callback - DEBUG - Processing node 69ccc1bb8cd9cbcdc945c40f: baseline-arm-baylibre 2026-04-03 06:57:14,295 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc1bb8cd9cbcdc945c40f 2026-04-03 06:57:14,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:57:14,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc1bb8cd9cbcdc945c40f HTTP/1.1" 200 None 2026-04-03 06:57:14,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:57:14,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 06:57:14,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:57:14,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc1bb8cd9cbcdc945c40f HTTP/1.1" 200 None 2026-04-03 06:57:14,352 - lava_callback - ERROR - Error processing callback for node 69ccc1bb8cd9cbcdc945c40f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53362 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:57:47,611 - lava_callback - INFO - Received callback for node 69ccc1e38cd9cbcdc945c425 INFO: 172.21.0.1:53366 - "POST /node/69ccc1e38cd9cbcdc945c425 HTTP/1.0" 202 Accepted 2026-04-03 06:57:47,616 - lava_callback - INFO - Starting async job submit for node 69ccc1e38cd9cbcdc945c425 2026-04-03 06:57:47,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:57:47,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc1e38cd9cbcdc945c425 HTTP/1.1" 200 None 2026-04-03 06:57:47,635 - lava_callback - DEBUG - Processing node 69ccc1e38cd9cbcdc945c425: baseline-arm-clabbe 2026-04-03 06:57:47,636 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc1e38cd9cbcdc945c425 2026-04-03 06:57:47,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:57:47,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc1e38cd9cbcdc945c425 HTTP/1.1" 200 None 2026-04-03 06:57:47,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:57:47,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:57:47,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:57:47,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc1e38cd9cbcdc945c425 HTTP/1.1" 200 None 2026-04-03 06:57:47,700 - lava_callback - ERROR - Error processing callback for node 69ccc1e38cd9cbcdc945c425: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36952 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:58:27,621 - lava_callback - INFO - Received callback for node 69ccc2088cd9cbcdc945c42d INFO: 172.21.0.1:36956 - "POST /node/69ccc2088cd9cbcdc945c42d HTTP/1.0" 202 Accepted 2026-04-03 06:58:27,627 - lava_callback - INFO - Starting async job submit for node 69ccc2088cd9cbcdc945c42d 2026-04-03 06:58:27,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:58:27,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc2088cd9cbcdc945c42d HTTP/1.1" 200 None 2026-04-03 06:58:27,642 - lava_callback - DEBUG - Processing node 69ccc2088cd9cbcdc945c42d: baseline-arm-clabbe 2026-04-03 06:58:27,643 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc2088cd9cbcdc945c42d 2026-04-03 06:58:27,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:58:27,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc2088cd9cbcdc945c42d HTTP/1.1" 200 None 2026-04-03 06:58:27,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:58:27,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c11b3c3aafdab4be7746 HTTP/1.1" 200 None 2026-04-03 06:58:27,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:58:27,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc2088cd9cbcdc945c42d HTTP/1.1" 200 None 2026-04-03 06:58:27,695 - lava_callback - ERROR - Error processing callback for node 69ccc2088cd9cbcdc945c42d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52412 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:59:07,690 - lava_callback - INFO - Received callback for node 69ccc22e8cd9cbcdc945c43d INFO: 172.21.0.1:52416 - "POST /node/69ccc22e8cd9cbcdc945c43d HTTP/1.0" 202 Accepted 2026-04-03 06:59:07,695 - lava_callback - INFO - Starting async job submit for node 69ccc22e8cd9cbcdc945c43d 2026-04-03 06:59:07,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:59:07,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc22e8cd9cbcdc945c43d HTTP/1.1" 200 None 2026-04-03 06:59:07,711 - lava_callback - DEBUG - Processing node 69ccc22e8cd9cbcdc945c43d: baseline-arm-clabbe 2026-04-03 06:59:07,712 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc22e8cd9cbcdc945c43d 2026-04-03 06:59:07,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:59:07,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc22e8cd9cbcdc945c43d HTTP/1.1" 200 None 2026-04-03 06:59:07,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:59:07,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 06:59:07,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:59:07,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc22e8cd9cbcdc945c43d HTTP/1.1" 200 None 2026-04-03 06:59:07,763 - lava_callback - ERROR - Error processing callback for node 69ccc22e8cd9cbcdc945c43d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57074 - "GET / HTTP/1.0" 200 OK 2026-04-03 06:59:47,692 - lava_callback - INFO - Received callback for node 69ccc2538cd9cbcdc945c44d INFO: 172.21.0.1:57084 - "POST /node/69ccc2538cd9cbcdc945c44d HTTP/1.0" 202 Accepted 2026-04-03 06:59:47,698 - lava_callback - INFO - Starting async job submit for node 69ccc2538cd9cbcdc945c44d 2026-04-03 06:59:47,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:59:47,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc2538cd9cbcdc945c44d HTTP/1.1" 200 None 2026-04-03 06:59:47,720 - lava_callback - DEBUG - Processing node 69ccc2538cd9cbcdc945c44d: baseline-arm-clabbe 2026-04-03 06:59:47,720 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc2538cd9cbcdc945c44d 2026-04-03 06:59:47,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:59:47,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc2538cd9cbcdc945c44d HTTP/1.1" 200 None 2026-04-03 06:59:47,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:59:47,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 06:59:47,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 06:59:47,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc2538cd9cbcdc945c44d HTTP/1.1" 200 None 2026-04-03 06:59:47,802 - lava_callback - ERROR - Error processing callback for node 69ccc2538cd9cbcdc945c44d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:00:27,705 - lava_callback - INFO - Received callback for node 69ccc2798cd9cbcdc945c464 INFO: 172.21.0.1:48186 - "POST /node/69ccc2798cd9cbcdc945c464 HTTP/1.0" 202 Accepted 2026-04-03 07:00:27,710 - lava_callback - INFO - Starting async job submit for node 69ccc2798cd9cbcdc945c464 2026-04-03 07:00:27,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:00:27,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc2798cd9cbcdc945c464 HTTP/1.1" 200 None 2026-04-03 07:00:27,726 - lava_callback - DEBUG - Processing node 69ccc2798cd9cbcdc945c464: baseline-arm-clabbe 2026-04-03 07:00:27,726 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc2798cd9cbcdc945c464 2026-04-03 07:00:27,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:00:27,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc2798cd9cbcdc945c464 HTTP/1.1" 200 None 2026-04-03 07:00:27,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:00:27,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:00:27,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:00:27,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc2798cd9cbcdc945c464 HTTP/1.1" 200 None 2026-04-03 07:00:27,789 - lava_callback - ERROR - Error processing callback for node 69ccc2798cd9cbcdc945c464: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44682 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:01:07,717 - lava_callback - INFO - Received callback for node 69ccc29f8cd9cbcdc945c46d INFO: 172.21.0.1:44698 - "POST /node/69ccc29f8cd9cbcdc945c46d HTTP/1.0" 202 Accepted 2026-04-03 07:01:07,722 - lava_callback - INFO - Starting async job submit for node 69ccc29f8cd9cbcdc945c46d 2026-04-03 07:01:07,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:01:07,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc29f8cd9cbcdc945c46d HTTP/1.1" 200 None 2026-04-03 07:01:07,739 - lava_callback - DEBUG - Processing node 69ccc29f8cd9cbcdc945c46d: baseline-arm-clabbe 2026-04-03 07:01:07,739 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc29f8cd9cbcdc945c46d 2026-04-03 07:01:07,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:01:07,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc29f8cd9cbcdc945c46d HTTP/1.1" 200 None 2026-04-03 07:01:07,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:01:07,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:01:07,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:01:07,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc29f8cd9cbcdc945c46d HTTP/1.1" 200 None 2026-04-03 07:01:07,811 - lava_callback - ERROR - Error processing callback for node 69ccc29f8cd9cbcdc945c46d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50186 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:01:34,246 - lava_callback - INFO - Received callback for node 69ccc2be8cd9cbcdc945c47c INFO: 172.21.0.1:48792 - "POST /node/69ccc2be8cd9cbcdc945c47c HTTP/1.0" 202 Accepted 2026-04-03 07:01:34,254 - lava_callback - INFO - Starting async job submit for node 69ccc2be8cd9cbcdc945c47c 2026-04-03 07:01:34,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:01:34,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc2be8cd9cbcdc945c47c HTTP/1.1" 200 None 2026-04-03 07:01:34,274 - lava_callback - DEBUG - Processing node 69ccc2be8cd9cbcdc945c47c: baseline-arm-baylibre 2026-04-03 07:01:34,274 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc2be8cd9cbcdc945c47c 2026-04-03 07:01:34,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:01:34,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc2be8cd9cbcdc945c47c HTTP/1.1" 200 None 2026-04-03 07:01:34,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:01:34,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:01:34,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:01:34,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc2be8cd9cbcdc945c47c HTTP/1.1" 200 None 2026-04-03 07:01:34,390 - lava_callback - ERROR - Error processing callback for node 69ccc2be8cd9cbcdc945c47c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38552 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:02:14,195 - lava_callback - INFO - Received callback for node 69ccc2e38cd9cbcdc945c48a INFO: 172.21.0.1:32770 - "POST /node/69ccc2e38cd9cbcdc945c48a HTTP/1.0" 202 Accepted 2026-04-03 07:02:14,200 - lava_callback - INFO - Starting async job submit for node 69ccc2e38cd9cbcdc945c48a 2026-04-03 07:02:14,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:02:14,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc2e38cd9cbcdc945c48a HTTP/1.1" 200 None 2026-04-03 07:02:14,216 - lava_callback - DEBUG - Processing node 69ccc2e38cd9cbcdc945c48a: baseline-arm-baylibre 2026-04-03 07:02:14,216 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc2e38cd9cbcdc945c48a 2026-04-03 07:02:14,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:02:14,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc2e38cd9cbcdc945c48a HTTP/1.1" 200 None 2026-04-03 07:02:14,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:02:14,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:02:14,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:02:14,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc2e38cd9cbcdc945c48a HTTP/1.1" 200 None 2026-04-03 07:02:14,273 - lava_callback - ERROR - Error processing callback for node 69ccc2e38cd9cbcdc945c48a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49088 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:02:47,826 - lava_callback - INFO - Received callback for node 69ccc30c8cd9cbcdc945c49e INFO: 172.21.0.1:49094 - "POST /node/69ccc30c8cd9cbcdc945c49e HTTP/1.0" 202 Accepted 2026-04-03 07:02:47,832 - lava_callback - INFO - Starting async job submit for node 69ccc30c8cd9cbcdc945c49e 2026-04-03 07:02:47,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:02:47,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc30c8cd9cbcdc945c49e HTTP/1.1" 200 None 2026-04-03 07:02:47,847 - lava_callback - DEBUG - Processing node 69ccc30c8cd9cbcdc945c49e: baseline-arm-clabbe 2026-04-03 07:02:47,847 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc30c8cd9cbcdc945c49e 2026-04-03 07:02:47,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:02:47,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc30c8cd9cbcdc945c49e HTTP/1.1" 200 None 2026-04-03 07:02:47,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:02:47,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:02:47,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:02:47,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc30c8cd9cbcdc945c49e HTTP/1.1" 200 None 2026-04-03 07:02:47,914 - lava_callback - ERROR - Error processing callback for node 69ccc30c8cd9cbcdc945c49e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55290 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:03:14,319 - lava_callback - INFO - Received callback for node 69ccc32d8cd9cbcdc945c4b1 INFO: 172.21.0.1:40582 - "POST /node/69ccc32d8cd9cbcdc945c4b1 HTTP/1.0" 202 Accepted 2026-04-03 07:03:14,325 - lava_callback - INFO - Starting async job submit for node 69ccc32d8cd9cbcdc945c4b1 2026-04-03 07:03:14,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:03:14,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc32d8cd9cbcdc945c4b1 HTTP/1.1" 200 None 2026-04-03 07:03:14,345 - lava_callback - DEBUG - Processing node 69ccc32d8cd9cbcdc945c4b1: baseline-arm-baylibre 2026-04-03 07:03:14,346 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc32d8cd9cbcdc945c4b1 2026-04-03 07:03:14,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:03:14,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc32d8cd9cbcdc945c4b1 HTTP/1.1" 200 None 2026-04-03 07:03:14,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:03:14,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:03:14,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:03:14,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc32d8cd9cbcdc945c4b1 HTTP/1.1" 200 None 2026-04-03 07:03:14,401 - lava_callback - ERROR - Error processing callback for node 69ccc32d8cd9cbcdc945c4b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41380 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:03:54,332 - lava_callback - INFO - Received callback for node 69ccc3528cd9cbcdc945c4be INFO: 172.21.0.1:41382 - "POST /node/69ccc3528cd9cbcdc945c4be HTTP/1.0" 202 Accepted 2026-04-03 07:03:54,337 - lava_callback - INFO - Starting async job submit for node 69ccc3528cd9cbcdc945c4be 2026-04-03 07:03:54,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:03:54,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc3528cd9cbcdc945c4be HTTP/1.1" 200 None 2026-04-03 07:03:54,354 - lava_callback - DEBUG - Processing node 69ccc3528cd9cbcdc945c4be: baseline-arm-baylibre 2026-04-03 07:03:54,355 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc3528cd9cbcdc945c4be 2026-04-03 07:03:54,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:03:54,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc3528cd9cbcdc945c4be HTTP/1.1" 200 None 2026-04-03 07:03:54,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:03:54,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:03:54,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:03:54,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc3528cd9cbcdc945c4be HTTP/1.1" 200 None 2026-04-03 07:03:54,424 - lava_callback - ERROR - Error processing callback for node 69ccc3528cd9cbcdc945c4be: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39146 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:04:34,365 - lava_callback - INFO - Received callback for node 69ccc3768cd9cbcdc945c4cd INFO: 172.21.0.1:34002 - "POST /node/69ccc3768cd9cbcdc945c4cd HTTP/1.0" 202 Accepted 2026-04-03 07:04:34,371 - lava_callback - INFO - Starting async job submit for node 69ccc3768cd9cbcdc945c4cd 2026-04-03 07:04:34,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:04:34,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc3768cd9cbcdc945c4cd HTTP/1.1" 200 None 2026-04-03 07:04:34,394 - lava_callback - DEBUG - Processing node 69ccc3768cd9cbcdc945c4cd: baseline-arm-baylibre 2026-04-03 07:04:34,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc3768cd9cbcdc945c4cd 2026-04-03 07:04:34,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:04:34,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc3768cd9cbcdc945c4cd HTTP/1.1" 200 None 2026-04-03 07:04:34,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:04:34,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:04:34,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:04:34,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc3768cd9cbcdc945c4cd HTTP/1.1" 200 None 2026-04-03 07:04:34,456 - lava_callback - ERROR - Error processing callback for node 69ccc3768cd9cbcdc945c4cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51180 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:05:47,927 - lava_callback - INFO - Received callback for node 69ccc3c28cd9cbcdc945c4ed INFO: 172.21.0.1:51196 - "POST /node/69ccc3c28cd9cbcdc945c4ed HTTP/1.0" 202 Accepted 2026-04-03 07:05:47,932 - lava_callback - INFO - Starting async job submit for node 69ccc3c28cd9cbcdc945c4ed 2026-04-03 07:05:47,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:05:47,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc3c28cd9cbcdc945c4ed HTTP/1.1" 200 None 2026-04-03 07:05:47,948 - lava_callback - DEBUG - Processing node 69ccc3c28cd9cbcdc945c4ed: baseline-arm-clabbe 2026-04-03 07:05:47,948 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc3c28cd9cbcdc945c4ed 2026-04-03 07:05:47,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:05:47,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc3c28cd9cbcdc945c4ed HTTP/1.1" 200 None 2026-04-03 07:05:47,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:05:47,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:05:47,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:05:47,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc3c28cd9cbcdc945c4ed HTTP/1.1" 200 None 2026-04-03 07:05:48,008 - lava_callback - ERROR - Error processing callback for node 69ccc3c28cd9cbcdc945c4ed: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51362 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:06:54,552 - lava_callback - INFO - Received callback for node 69ccc4098cd9cbcdc945c508 INFO: 172.21.0.1:51376 - "POST /node/69ccc4098cd9cbcdc945c508 HTTP/1.0" 202 Accepted 2026-04-03 07:06:54,557 - lava_callback - INFO - Starting async job submit for node 69ccc4098cd9cbcdc945c508 2026-04-03 07:06:54,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:06:54,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc4098cd9cbcdc945c508 HTTP/1.1" 200 None 2026-04-03 07:06:54,573 - lava_callback - DEBUG - Processing node 69ccc4098cd9cbcdc945c508: baseline-arm-baylibre 2026-04-03 07:06:54,573 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc4098cd9cbcdc945c508 2026-04-03 07:06:54,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:06:54,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc4098cd9cbcdc945c508 HTTP/1.1" 200 None 2026-04-03 07:06:54,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:06:54,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:06:54,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:06:54,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc4098cd9cbcdc945c508 HTTP/1.1" 200 None 2026-04-03 07:06:54,632 - lava_callback - ERROR - Error processing callback for node 69ccc4098cd9cbcdc945c508: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50396 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:08:14,588 - lava_callback - INFO - Received callback for node 69ccc4528cd9cbcdc945c525 INFO: 172.21.0.1:56466 - "POST /node/69ccc4528cd9cbcdc945c525 HTTP/1.0" 202 Accepted 2026-04-03 07:08:14,595 - lava_callback - INFO - Starting async job submit for node 69ccc4528cd9cbcdc945c525 2026-04-03 07:08:14,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:08:14,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc4528cd9cbcdc945c525 HTTP/1.1" 200 None 2026-04-03 07:08:14,616 - lava_callback - DEBUG - Processing node 69ccc4528cd9cbcdc945c525: baseline-arm-baylibre 2026-04-03 07:08:14,617 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc4528cd9cbcdc945c525 2026-04-03 07:08:14,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:08:14,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc4528cd9cbcdc945c525 HTTP/1.1" 200 None 2026-04-03 07:08:14,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:08:14,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:08:14,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:08:14,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc4528cd9cbcdc945c525 HTTP/1.1" 200 None 2026-04-03 07:08:14,677 - lava_callback - ERROR - Error processing callback for node 69ccc4528cd9cbcdc945c525: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51500 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:08:48,033 - lava_callback - INFO - Received callback for node 69ccc4788cd9cbcdc945c54a INFO: 172.21.0.1:51512 - "POST /node/69ccc4788cd9cbcdc945c54a HTTP/1.0" 202 Accepted 2026-04-03 07:08:48,039 - lava_callback - INFO - Starting async job submit for node 69ccc4788cd9cbcdc945c54a 2026-04-03 07:08:48,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:08:48,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc4788cd9cbcdc945c54a HTTP/1.1" 200 None 2026-04-03 07:08:48,063 - lava_callback - DEBUG - Processing node 69ccc4788cd9cbcdc945c54a: baseline-arm-clabbe 2026-04-03 07:08:48,063 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc4788cd9cbcdc945c54a 2026-04-03 07:08:48,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:08:48,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc4788cd9cbcdc945c54a HTTP/1.1" 200 None 2026-04-03 07:08:48,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:08:48,096 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:08:48,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:08:48,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc4788cd9cbcdc945c54a HTTP/1.1" 200 None 2026-04-03 07:08:48,123 - lava_callback - ERROR - Error processing callback for node 69ccc4788cd9cbcdc945c54a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55022 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49756 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:10:08,128 - lava_callback - INFO - Received callback for node 69ccc4c08cd9cbcdc945c581 INFO: 172.21.0.1:49770 - "POST /node/69ccc4c08cd9cbcdc945c581 HTTP/1.0" 202 Accepted 2026-04-03 07:10:08,133 - lava_callback - INFO - Starting async job submit for node 69ccc4c08cd9cbcdc945c581 2026-04-03 07:10:08,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:10:08,149 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc4c08cd9cbcdc945c581 HTTP/1.1" 200 None 2026-04-03 07:10:08,149 - lava_callback - DEBUG - Processing node 69ccc4c08cd9cbcdc945c581: baseline-arm-clabbe 2026-04-03 07:10:08,150 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc4c08cd9cbcdc945c581 2026-04-03 07:10:08,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:10:08,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc4c08cd9cbcdc945c581 HTTP/1.1" 200 None 2026-04-03 07:10:08,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:10:08,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:10:08,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:10:08,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc4c08cd9cbcdc945c581 HTTP/1.1" 200 None 2026-04-03 07:10:08,204 - lava_callback - ERROR - Error processing callback for node 69ccc4c08cd9cbcdc945c581: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52086 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46202 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:17:08,368 - lava_callback - INFO - Received callback for node 69ccc6618cd9cbcdc945c640 INFO: 172.21.0.1:46214 - "POST /node/69ccc6618cd9cbcdc945c640 HTTP/1.0" 202 Accepted 2026-04-03 07:17:08,373 - lava_callback - INFO - Starting async job submit for node 69ccc6618cd9cbcdc945c640 2026-04-03 07:17:08,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:17:08,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc6618cd9cbcdc945c640 HTTP/1.1" 200 None 2026-04-03 07:17:08,389 - lava_callback - DEBUG - Processing node 69ccc6618cd9cbcdc945c640: baseline-arm-clabbe 2026-04-03 07:17:08,389 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc6618cd9cbcdc945c640 2026-04-03 07:17:08,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:17:08,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc6618cd9cbcdc945c640 HTTP/1.1" 200 None 2026-04-03 07:17:08,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:17:08,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:17:08,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:17:08,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc6618cd9cbcdc945c640 HTTP/1.1" 200 None 2026-04-03 07:17:08,446 - lava_callback - ERROR - Error processing callback for node 69ccc6618cd9cbcdc945c640: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42678 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:17:35,165 - lava_callback - INFO - Received callback for node 69ccc68c8cd9cbcdc945c64e INFO: 172.21.0.1:34216 - "POST /node/69ccc68c8cd9cbcdc945c64e HTTP/1.0" 202 Accepted 2026-04-03 07:17:35,174 - lava_callback - INFO - Starting async job submit for node 69ccc68c8cd9cbcdc945c64e 2026-04-03 07:17:35,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:17:35,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc68c8cd9cbcdc945c64e HTTP/1.1" 200 None 2026-04-03 07:17:35,194 - lava_callback - DEBUG - Processing node 69ccc68c8cd9cbcdc945c64e: baseline-arm64-baylibre 2026-04-03 07:17:35,195 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc68c8cd9cbcdc945c64e 2026-04-03 07:17:35,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:17:35,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc68c8cd9cbcdc945c64e HTTP/1.1" 200 None 2026-04-03 07:17:35,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:17:35,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:17:35,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:17:35,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc68c8cd9cbcdc945c64e HTTP/1.1" 200 None 2026-04-03 07:17:35,277 - lava_callback - ERROR - Error processing callback for node 69ccc68c8cd9cbcdc945c64e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51254 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 07:19:15,165 - lava_callback - INFO - Received callback for node 69ccc6e78cd9cbcdc945c66b INFO: 172.21.0.1:51264 - "POST /node/69ccc6e78cd9cbcdc945c66b HTTP/1.0" 202 Accepted 2026-04-03 07:19:15,170 - lava_callback - INFO - Starting async job submit for node 69ccc6e78cd9cbcdc945c66b 2026-04-03 07:19:15,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:19:15,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc6e78cd9cbcdc945c66b HTTP/1.1" 200 None 2026-04-03 07:19:15,187 - lava_callback - DEBUG - Processing node 69ccc6e78cd9cbcdc945c66b: baseline-arm64-baylibre 2026-04-03 07:19:15,187 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc6e78cd9cbcdc945c66b 2026-04-03 07:19:15,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:19:15,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc6e78cd9cbcdc945c66b HTTP/1.1" 200 None 2026-04-03 07:19:15,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:19:15,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:19:15,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:19:15,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc6e78cd9cbcdc945c66b HTTP/1.1" 200 None 2026-04-03 07:19:15,244 - lava_callback - ERROR - Error processing callback for node 69ccc6e78cd9cbcdc945c66b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36526 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:20:55,268 - lava_callback - INFO - Received callback for node 69ccc7458cd9cbcdc945c695 INFO: 172.21.0.1:36540 - "POST /node/69ccc7458cd9cbcdc945c695 HTTP/1.0" 202 Accepted 2026-04-03 07:20:55,274 - lava_callback - INFO - Starting async job submit for node 69ccc7458cd9cbcdc945c695 2026-04-03 07:20:55,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:20:55,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc7458cd9cbcdc945c695 HTTP/1.1" 200 None 2026-04-03 07:20:55,291 - lava_callback - DEBUG - Processing node 69ccc7458cd9cbcdc945c695: baseline-arm64-baylibre 2026-04-03 07:20:55,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc7458cd9cbcdc945c695 2026-04-03 07:20:55,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:20:55,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc7458cd9cbcdc945c695 HTTP/1.1" 200 None 2026-04-03 07:20:55,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:20:55,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:20:55,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:20:55,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc7458cd9cbcdc945c695 HTTP/1.1" 200 None 2026-04-03 07:20:55,386 - lava_callback - ERROR - Error processing callback for node 69ccc7458cd9cbcdc945c695: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35420 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 07:22:15,322 - lava_callback - INFO - Received callback for node 69ccc79b8cd9cbcdc945c6d6 INFO: 172.21.0.1:35426 - "POST /node/69ccc79b8cd9cbcdc945c6d6 HTTP/1.0" 202 Accepted 2026-04-03 07:22:15,328 - lava_callback - INFO - Starting async job submit for node 69ccc79b8cd9cbcdc945c6d6 2026-04-03 07:22:15,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:22:15,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc79b8cd9cbcdc945c6d6 HTTP/1.1" 200 None 2026-04-03 07:22:15,346 - lava_callback - DEBUG - Processing node 69ccc79b8cd9cbcdc945c6d6: baseline-arm64-baylibre 2026-04-03 07:22:15,347 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc79b8cd9cbcdc945c6d6 2026-04-03 07:22:15,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:22:15,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc79b8cd9cbcdc945c6d6 HTTP/1.1" 200 None 2026-04-03 07:22:15,367 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:22:15,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:22:15,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:22:15,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc79b8cd9cbcdc945c6d6 HTTP/1.1" 200 None 2026-04-03 07:22:15,403 - lava_callback - ERROR - Error processing callback for node 69ccc79b8cd9cbcdc945c6d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33850 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:23:35,366 - lava_callback - INFO - Received callback for node 69ccc7f18cd9cbcdc945c705 INFO: 172.21.0.1:56154 - "POST /node/69ccc7f18cd9cbcdc945c705 HTTP/1.0" 202 Accepted 2026-04-03 07:23:35,372 - lava_callback - INFO - Starting async job submit for node 69ccc7f18cd9cbcdc945c705 2026-04-03 07:23:35,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:23:35,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc7f18cd9cbcdc945c705 HTTP/1.1" 200 None 2026-04-03 07:23:35,390 - lava_callback - DEBUG - Processing node 69ccc7f18cd9cbcdc945c705: baseline-arm64-baylibre 2026-04-03 07:23:35,390 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc7f18cd9cbcdc945c705 2026-04-03 07:23:35,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:23:35,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc7f18cd9cbcdc945c705 HTTP/1.1" 200 None 2026-04-03 07:23:35,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:23:35,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:23:35,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:23:35,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc7f18cd9cbcdc945c705 HTTP/1.1" 200 None 2026-04-03 07:23:35,452 - lava_callback - ERROR - Error processing callback for node 69ccc7f18cd9cbcdc945c705: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43086 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:25:08,724 - lava_callback - INFO - Received callback for node 69ccc84e8cd9cbcdc945c729 INFO: 172.21.0.1:43096 - "POST /node/69ccc84e8cd9cbcdc945c729 HTTP/1.0" 202 Accepted 2026-04-03 07:25:08,730 - lava_callback - INFO - Starting async job submit for node 69ccc84e8cd9cbcdc945c729 2026-04-03 07:25:08,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:25:08,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc84e8cd9cbcdc945c729 HTTP/1.1" 200 None 2026-04-03 07:25:08,757 - lava_callback - DEBUG - Processing node 69ccc84e8cd9cbcdc945c729: baseline-arm64-clabbe 2026-04-03 07:25:08,757 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc84e8cd9cbcdc945c729 2026-04-03 07:25:08,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:25:08,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc84e8cd9cbcdc945c729 HTTP/1.1" 200 None 2026-04-03 07:25:08,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:25:08,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:25:08,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:25:08,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc84e8cd9cbcdc945c729 HTTP/1.1" 200 None 2026-04-03 07:25:08,814 - lava_callback - ERROR - Error processing callback for node 69ccc84e8cd9cbcdc945c729: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37974 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52366 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:26:28,707 - lava_callback - INFO - Received callback for node 69ccc8978cd9cbcdc945c751 INFO: 172.21.0.1:52368 - "POST /node/69ccc8978cd9cbcdc945c751 HTTP/1.0" 202 Accepted 2026-04-03 07:26:28,712 - lava_callback - INFO - Starting async job submit for node 69ccc8978cd9cbcdc945c751 2026-04-03 07:26:28,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:26:28,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc8978cd9cbcdc945c751 HTTP/1.1" 200 None 2026-04-03 07:26:28,728 - lava_callback - DEBUG - Processing node 69ccc8978cd9cbcdc945c751: baseline-arm-clabbe 2026-04-03 07:26:28,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc8978cd9cbcdc945c751 2026-04-03 07:26:28,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:26:28,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc8978cd9cbcdc945c751 HTTP/1.1" 200 None 2026-04-03 07:26:28,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:26:28,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:26:28,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:26:28,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc8978cd9cbcdc945c751 HTTP/1.1" 200 None 2026-04-03 07:26:28,829 - lava_callback - ERROR - Error processing callback for node 69ccc8978cd9cbcdc945c751: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54070 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:27:28,721 - lava_callback - INFO - Received callback for node 69ccc8cc8cd9cbcdc945c75c INFO: 172.21.0.1:54072 - "POST /node/69ccc8cc8cd9cbcdc945c75c HTTP/1.0" 202 Accepted 2026-04-03 07:27:28,727 - lava_callback - INFO - Starting async job submit for node 69ccc8cc8cd9cbcdc945c75c 2026-04-03 07:27:28,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:27:28,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc8cc8cd9cbcdc945c75c HTTP/1.1" 200 None 2026-04-03 07:27:28,744 - lava_callback - DEBUG - Processing node 69ccc8cc8cd9cbcdc945c75c: baseline-arm64-clabbe 2026-04-03 07:27:28,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc8cc8cd9cbcdc945c75c 2026-04-03 07:27:28,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:27:28,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc8cc8cd9cbcdc945c75c HTTP/1.1" 200 None 2026-04-03 07:27:28,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:27:28,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:27:28,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:27:28,812 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc8cc8cd9cbcdc945c75c HTTP/1.1" 200 None 2026-04-03 07:27:28,819 - lava_callback - ERROR - Error processing callback for node 69ccc8cc8cd9cbcdc945c75c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51470 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:28:48,821 - lava_callback - INFO - Received callback for node 69ccc9248cd9cbcdc945c77e INFO: 172.21.0.1:51482 - "POST /node/69ccc9248cd9cbcdc945c77e HTTP/1.0" 202 Accepted 2026-04-03 07:28:48,826 - lava_callback - INFO - Starting async job submit for node 69ccc9248cd9cbcdc945c77e 2026-04-03 07:28:48,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:28:48,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc9248cd9cbcdc945c77e HTTP/1.1" 200 None 2026-04-03 07:28:48,843 - lava_callback - DEBUG - Processing node 69ccc9248cd9cbcdc945c77e: baseline-arm64-clabbe 2026-04-03 07:28:48,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc9248cd9cbcdc945c77e 2026-04-03 07:28:48,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:28:48,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc9248cd9cbcdc945c77e HTTP/1.1" 200 None 2026-04-03 07:28:48,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:28:48,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:28:48,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:28:48,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc9248cd9cbcdc945c77e HTTP/1.1" 200 None 2026-04-03 07:28:48,899 - lava_callback - ERROR - Error processing callback for node 69ccc9248cd9cbcdc945c77e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54736 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:30:28,922 - lava_callback - INFO - Received callback for node 69ccc97e8cd9cbcdc945c7a3 INFO: 172.21.0.1:54738 - "POST /node/69ccc97e8cd9cbcdc945c7a3 HTTP/1.0" 202 Accepted 2026-04-03 07:30:28,931 - lava_callback - INFO - Starting async job submit for node 69ccc97e8cd9cbcdc945c7a3 2026-04-03 07:30:28,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:30:28,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc97e8cd9cbcdc945c7a3 HTTP/1.1" 200 None 2026-04-03 07:30:28,947 - lava_callback - DEBUG - Processing node 69ccc97e8cd9cbcdc945c7a3: baseline-arm64-clabbe 2026-04-03 07:30:28,947 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc97e8cd9cbcdc945c7a3 2026-04-03 07:30:28,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:30:28,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc97e8cd9cbcdc945c7a3 HTTP/1.1" 200 None 2026-04-03 07:30:28,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:30:28,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:30:28,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:30:29,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc97e8cd9cbcdc945c7a3 HTTP/1.1" 200 None 2026-04-03 07:30:29,022 - lava_callback - ERROR - Error processing callback for node 69ccc97e8cd9cbcdc945c7a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58910 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:31:48,961 - lava_callback - INFO - Received callback for node 69ccc9d58cd9cbcdc945c7c9 INFO: 172.21.0.1:58916 - "POST /node/69ccc9d58cd9cbcdc945c7c9 HTTP/1.0" 202 Accepted 2026-04-03 07:31:48,966 - lava_callback - INFO - Starting async job submit for node 69ccc9d58cd9cbcdc945c7c9 2026-04-03 07:31:48,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:31:48,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc9d58cd9cbcdc945c7c9 HTTP/1.1" 200 None 2026-04-03 07:31:48,982 - lava_callback - DEBUG - Processing node 69ccc9d58cd9cbcdc945c7c9: baseline-arm64-clabbe 2026-04-03 07:31:48,982 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccc9d58cd9cbcdc945c7c9 2026-04-03 07:31:48,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:31:48,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccc9d58cd9cbcdc945c7c9 HTTP/1.1" 200 None 2026-04-03 07:31:48,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:31:49,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:31:49,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:31:49,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccc9d58cd9cbcdc945c7c9 HTTP/1.1" 200 None 2026-04-03 07:31:49,034 - lava_callback - ERROR - Error processing callback for node 69ccc9d58cd9cbcdc945c7c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38954 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:33:09,058 - lava_callback - INFO - Received callback for node 69ccca2f8cd9cbcdc945c7ea INFO: 172.21.0.1:38970 - "POST /node/69ccca2f8cd9cbcdc945c7ea HTTP/1.0" 202 Accepted 2026-04-03 07:33:09,064 - lava_callback - INFO - Starting async job submit for node 69ccca2f8cd9cbcdc945c7ea 2026-04-03 07:33:09,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:33:09,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccca2f8cd9cbcdc945c7ea HTTP/1.1" 200 None 2026-04-03 07:33:09,099 - lava_callback - DEBUG - Processing node 69ccca2f8cd9cbcdc945c7ea: baseline-arm64-clabbe 2026-04-03 07:33:09,100 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccca2f8cd9cbcdc945c7ea 2026-04-03 07:33:09,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:33:09,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccca2f8cd9cbcdc945c7ea HTTP/1.1" 200 None 2026-04-03 07:33:09,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:33:09,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cc648c8cd9cbcdc9459656 HTTP/1.1" 200 None 2026-04-03 07:33:09,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:33:09,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccca2f8cd9cbcdc945c7ea HTTP/1.1" 200 None 2026-04-03 07:33:09,162 - lava_callback - ERROR - Error processing callback for node 69ccca2f8cd9cbcdc945c7ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39526 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:34:29,044 - lava_callback - INFO - Received callback for node 69ccca798cd9cbcdc945c812 INFO: 172.21.0.1:39530 - "POST /node/69ccca798cd9cbcdc945c812 HTTP/1.0" 202 Accepted 2026-04-03 07:34:29,051 - lava_callback - INFO - Starting async job submit for node 69ccca798cd9cbcdc945c812 2026-04-03 07:34:29,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:34:29,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccca798cd9cbcdc945c812 HTTP/1.1" 200 None 2026-04-03 07:34:29,075 - lava_callback - DEBUG - Processing node 69ccca798cd9cbcdc945c812: baseline-arm-clabbe 2026-04-03 07:34:29,075 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccca798cd9cbcdc945c812 2026-04-03 07:34:29,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:34:29,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccca798cd9cbcdc945c812 HTTP/1.1" 200 None 2026-04-03 07:34:29,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:34:29,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:34:29,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:34:29,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccca798cd9cbcdc945c812 HTTP/1.1" 200 None 2026-04-03 07:34:29,128 - lava_callback - ERROR - Error processing callback for node 69ccca798cd9cbcdc945c812: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58468 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:35:09,143 - lava_callback - INFO - Received callback for node 69ccca9d8cd9cbcdc945c81e INFO: 172.21.0.1:58476 - "POST /node/69ccca9d8cd9cbcdc945c81e HTTP/1.0" 202 Accepted 2026-04-03 07:35:09,149 - lava_callback - INFO - Starting async job submit for node 69ccca9d8cd9cbcdc945c81e 2026-04-03 07:35:09,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:35:09,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccca9d8cd9cbcdc945c81e HTTP/1.1" 200 None 2026-04-03 07:35:09,165 - lava_callback - DEBUG - Processing node 69ccca9d8cd9cbcdc945c81e: baseline-arm-clabbe 2026-04-03 07:35:09,165 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccca9d8cd9cbcdc945c81e 2026-04-03 07:35:09,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:35:09,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccca9d8cd9cbcdc945c81e HTTP/1.1" 200 None 2026-04-03 07:35:09,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:35:09,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:35:09,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:35:09,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccca9d8cd9cbcdc945c81e HTTP/1.1" 200 None 2026-04-03 07:35:09,218 - lava_callback - ERROR - Error processing callback for node 69ccca9d8cd9cbcdc945c81e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48818 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:52946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52956 - "GET /wiki HTTP/1.0" 404 Not Found 2026-04-03 07:35:49,111 - lava_callback - INFO - Received callback for node 69cccac28cd9cbcdc945c82c INFO: 172.21.0.1:52968 - "POST /node/69cccac28cd9cbcdc945c82c HTTP/1.0" 202 Accepted 2026-04-03 07:35:49,117 - lava_callback - INFO - Starting async job submit for node 69cccac28cd9cbcdc945c82c 2026-04-03 07:35:49,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:35:49,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccac28cd9cbcdc945c82c HTTP/1.1" 200 None 2026-04-03 07:35:49,133 - lava_callback - DEBUG - Processing node 69cccac28cd9cbcdc945c82c: baseline-arm-clabbe 2026-04-03 07:35:49,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccac28cd9cbcdc945c82c 2026-04-03 07:35:49,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:35:49,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccac28cd9cbcdc945c82c HTTP/1.1" 200 None 2026-04-03 07:35:49,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:35:49,162 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:35:49,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:35:49,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccac28cd9cbcdc945c82c HTTP/1.1" 200 None 2026-04-03 07:35:49,246 - lava_callback - ERROR - Error processing callback for node 69cccac28cd9cbcdc945c82c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59282 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:36:29,130 - lava_callback - INFO - Received callback for node 69cccae98cd9cbcdc945c838 INFO: 172.21.0.1:59288 - "POST /node/69cccae98cd9cbcdc945c838 HTTP/1.0" 202 Accepted 2026-04-03 07:36:29,135 - lava_callback - INFO - Starting async job submit for node 69cccae98cd9cbcdc945c838 2026-04-03 07:36:29,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:36:29,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccae98cd9cbcdc945c838 HTTP/1.1" 200 None 2026-04-03 07:36:29,150 - lava_callback - DEBUG - Processing node 69cccae98cd9cbcdc945c838: baseline-arm-clabbe 2026-04-03 07:36:29,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccae98cd9cbcdc945c838 2026-04-03 07:36:29,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:36:29,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccae98cd9cbcdc945c838 HTTP/1.1" 200 None 2026-04-03 07:36:29,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:36:29,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:36:29,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:36:29,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccae98cd9cbcdc945c838 HTTP/1.1" 200 None 2026-04-03 07:36:29,211 - lava_callback - ERROR - Error processing callback for node 69cccae98cd9cbcdc945c838: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:32950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58732 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:36:49,166 - lava_callback - INFO - Received callback for node 69cccb0d8cd9cbcdc945c84c INFO: 172.21.0.1:58748 - "POST /node/69cccb0d8cd9cbcdc945c84c HTTP/1.0" 202 Accepted 2026-04-03 07:36:49,171 - lava_callback - INFO - Starting async job submit for node 69cccb0d8cd9cbcdc945c84c 2026-04-03 07:36:49,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:36:49,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccb0d8cd9cbcdc945c84c HTTP/1.1" 200 None 2026-04-03 07:36:49,188 - lava_callback - DEBUG - Processing node 69cccb0d8cd9cbcdc945c84c: baseline-arm-clabbe 2026-04-03 07:36:49,189 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccb0d8cd9cbcdc945c84c 2026-04-03 07:36:49,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:36:49,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccb0d8cd9cbcdc945c84c HTTP/1.1" 200 None 2026-04-03 07:36:49,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:36:49,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-04-03 07:36:49,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:36:49,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccb0d8cd9cbcdc945c84c HTTP/1.1" 200 None 2026-04-03 07:36:49,241 - lava_callback - ERROR - Error processing callback for node 69cccb0d8cd9cbcdc945c84c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54286 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:37:36,201 - lava_callback - INFO - Received callback for node 69cccb398cd9cbcdc945c85a INFO: 172.21.0.1:51038 - "POST /node/69cccb398cd9cbcdc945c85a HTTP/1.0" 202 Accepted 2026-04-03 07:37:36,206 - lava_callback - INFO - Starting async job submit for node 69cccb398cd9cbcdc945c85a 2026-04-03 07:37:36,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:37:36,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccb398cd9cbcdc945c85a HTTP/1.1" 200 None 2026-04-03 07:37:36,223 - lava_callback - DEBUG - Processing node 69cccb398cd9cbcdc945c85a: baseline-arm64-baylibre 2026-04-03 07:37:36,223 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccb398cd9cbcdc945c85a 2026-04-03 07:37:36,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:37:36,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccb398cd9cbcdc945c85a HTTP/1.1" 200 None 2026-04-03 07:37:36,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:37:36,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:37:36,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:37:36,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccb398cd9cbcdc945c85a HTTP/1.1" 200 None 2026-04-03 07:37:36,284 - lava_callback - ERROR - Error processing callback for node 69cccb398cd9cbcdc945c85a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37242 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-03 07:39:16,187 - lava_callback - INFO - Received callback for node 69cccb8f8cd9cbcdc945c87c INFO: 172.21.0.1:37258 - "POST /node/69cccb8f8cd9cbcdc945c87c HTTP/1.0" 202 Accepted 2026-04-03 07:39:16,193 - lava_callback - INFO - Starting async job submit for node 69cccb8f8cd9cbcdc945c87c 2026-04-03 07:39:16,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:39:16,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccb8f8cd9cbcdc945c87c HTTP/1.1" 200 None 2026-04-03 07:39:16,209 - lava_callback - DEBUG - Processing node 69cccb8f8cd9cbcdc945c87c: baseline-arm64-baylibre 2026-04-03 07:39:16,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccb8f8cd9cbcdc945c87c 2026-04-03 07:39:16,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:39:16,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccb8f8cd9cbcdc945c87c HTTP/1.1" 200 None 2026-04-03 07:39:16,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:39:16,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:39:16,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:39:16,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccb8f8cd9cbcdc945c87c HTTP/1.1" 200 None 2026-04-03 07:39:16,259 - lava_callback - ERROR - Error processing callback for node 69cccb8f8cd9cbcdc945c87c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48282 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:40:36,285 - lava_callback - INFO - Received callback for node 69cccbe88cd9cbcdc945c8a7 INFO: 172.21.0.1:33032 - "POST /node/69cccbe88cd9cbcdc945c8a7 HTTP/1.0" 202 Accepted 2026-04-03 07:40:36,290 - lava_callback - INFO - Starting async job submit for node 69cccbe88cd9cbcdc945c8a7 2026-04-03 07:40:36,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:40:36,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccbe88cd9cbcdc945c8a7 HTTP/1.1" 200 None 2026-04-03 07:40:36,306 - lava_callback - DEBUG - Processing node 69cccbe88cd9cbcdc945c8a7: baseline-arm64-baylibre 2026-04-03 07:40:36,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccbe88cd9cbcdc945c8a7 2026-04-03 07:40:36,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:40:36,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccbe88cd9cbcdc945c8a7 HTTP/1.1" 200 None 2026-04-03 07:40:36,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:40:36,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:40:36,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:40:36,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccbe88cd9cbcdc945c8a7 HTTP/1.1" 200 None 2026-04-03 07:40:36,361 - lava_callback - ERROR - Error processing callback for node 69cccbe88cd9cbcdc945c8a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53506 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:41:56,344 - lava_callback - INFO - Received callback for node 69cccc408cd9cbcdc945c8c8 INFO: 172.21.0.1:53514 - "POST /node/69cccc408cd9cbcdc945c8c8 HTTP/1.0" 202 Accepted 2026-04-03 07:41:56,350 - lava_callback - INFO - Starting async job submit for node 69cccc408cd9cbcdc945c8c8 2026-04-03 07:41:56,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:41:56,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccc408cd9cbcdc945c8c8 HTTP/1.1" 200 None 2026-04-03 07:41:56,372 - lava_callback - DEBUG - Processing node 69cccc408cd9cbcdc945c8c8: baseline-arm64-baylibre 2026-04-03 07:41:56,373 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccc408cd9cbcdc945c8c8 2026-04-03 07:41:56,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:41:56,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccc408cd9cbcdc945c8c8 HTTP/1.1" 200 None 2026-04-03 07:41:56,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:41:56,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:41:56,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:41:56,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccc408cd9cbcdc945c8c8 HTTP/1.1" 200 None 2026-04-03 07:41:56,498 - lava_callback - ERROR - Error processing callback for node 69cccc408cd9cbcdc945c8c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40160 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:43:36,889 - lava_callback - INFO - Received callback for node 69cccc958cd9cbcdc945c8f0 INFO: 172.21.0.1:40166 - "POST /node/69cccc958cd9cbcdc945c8f0 HTTP/1.0" 202 Accepted 2026-04-03 07:43:36,897 - lava_callback - INFO - Starting async job submit for node 69cccc958cd9cbcdc945c8f0 2026-04-03 07:43:36,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:43:36,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccc958cd9cbcdc945c8f0 HTTP/1.1" 200 None 2026-04-03 07:43:36,917 - lava_callback - DEBUG - Processing node 69cccc958cd9cbcdc945c8f0: baseline-arm64-baylibre 2026-04-03 07:43:36,917 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccc958cd9cbcdc945c8f0 2026-04-03 07:43:36,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:43:36,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccc958cd9cbcdc945c8f0 HTTP/1.1" 200 None 2026-04-03 07:43:36,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:43:36,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:43:36,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:43:36,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccc958cd9cbcdc945c8f0 HTTP/1.1" 200 None 2026-04-03 07:43:36,981 - lava_callback - ERROR - Error processing callback for node 69cccc958cd9cbcdc945c8f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57188 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:44:56,506 - lava_callback - INFO - Received callback for node 69ccccec8cd9cbcdc945c912 INFO: 172.21.0.1:57190 - "POST /node/69ccccec8cd9cbcdc945c912 HTTP/1.0" 202 Accepted 2026-04-03 07:44:56,511 - lava_callback - INFO - Starting async job submit for node 69ccccec8cd9cbcdc945c912 2026-04-03 07:44:56,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:57204 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:44:56,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccccec8cd9cbcdc945c912 HTTP/1.1" 200 None 2026-04-03 07:44:56,530 - lava_callback - DEBUG - Processing node 69ccccec8cd9cbcdc945c912: baseline-arm64-baylibre 2026-04-03 07:44:56,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccccec8cd9cbcdc945c912 2026-04-03 07:44:56,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:44:56,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccccec8cd9cbcdc945c912 HTTP/1.1" 200 None 2026-04-03 07:44:56,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:44:56,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:44:56,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:44:56,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccccec8cd9cbcdc945c912 HTTP/1.1" 200 None 2026-04-03 07:44:56,589 - lava_callback - ERROR - Error processing callback for node 69ccccec8cd9cbcdc945c912: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52412 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:46:16,581 - lava_callback - INFO - Received callback for node 69cccd3f8cd9cbcdc945c92d INFO: 172.21.0.1:52416 - "POST /node/69cccd3f8cd9cbcdc945c92d HTTP/1.0" 202 Accepted 2026-04-03 07:46:16,590 - lava_callback - INFO - Starting async job submit for node 69cccd3f8cd9cbcdc945c92d 2026-04-03 07:46:16,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:46:16,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccd3f8cd9cbcdc945c92d HTTP/1.1" 200 None 2026-04-03 07:46:16,616 - lava_callback - DEBUG - Processing node 69cccd3f8cd9cbcdc945c92d: baseline-arm64-baylibre 2026-04-03 07:46:16,616 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccd3f8cd9cbcdc945c92d 2026-04-03 07:46:16,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:46:16,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccd3f8cd9cbcdc945c92d HTTP/1.1" 200 None 2026-04-03 07:46:16,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:46:16,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:46:16,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:46:16,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccd3f8cd9cbcdc945c92d HTTP/1.1" 200 None 2026-04-03 07:46:16,720 - lava_callback - ERROR - Error processing callback for node 69cccd3f8cd9cbcdc945c92d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48618 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:47:49,654 - lava_callback - INFO - Received callback for node 69cccda18cd9cbcdc945c95c INFO: 172.21.0.1:48626 - "POST /node/69cccda18cd9cbcdc945c95c HTTP/1.0" 202 Accepted 2026-04-03 07:47:49,659 - lava_callback - INFO - Starting async job submit for node 69cccda18cd9cbcdc945c95c 2026-04-03 07:47:49,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:47:49,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccda18cd9cbcdc945c95c HTTP/1.1" 200 None 2026-04-03 07:47:49,698 - lava_callback - DEBUG - Processing node 69cccda18cd9cbcdc945c95c: baseline-arm64-clabbe 2026-04-03 07:47:49,699 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccda18cd9cbcdc945c95c 2026-04-03 07:47:49,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:47:49,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccda18cd9cbcdc945c95c HTTP/1.1" 200 None 2026-04-03 07:47:49,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:47:49,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:47:49,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:47:49,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccda18cd9cbcdc945c95c HTTP/1.1" 200 None 2026-04-03 07:47:49,764 - lava_callback - ERROR - Error processing callback for node 69cccda18cd9cbcdc945c95c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35536 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:49:29,769 - lava_callback - INFO - Received callback for node 69cccdf98cd9cbcdc945c981 INFO: 172.21.0.1:42730 - "POST /node/69cccdf98cd9cbcdc945c981 HTTP/1.0" 202 Accepted 2026-04-03 07:49:29,774 - lava_callback - INFO - Starting async job submit for node 69cccdf98cd9cbcdc945c981 2026-04-03 07:49:29,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:49:29,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccdf98cd9cbcdc945c981 HTTP/1.1" 200 None 2026-04-03 07:49:29,790 - lava_callback - DEBUG - Processing node 69cccdf98cd9cbcdc945c981: baseline-arm64-clabbe 2026-04-03 07:49:29,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccdf98cd9cbcdc945c981 2026-04-03 07:49:29,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:49:29,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccdf98cd9cbcdc945c981 HTTP/1.1" 200 None 2026-04-03 07:49:29,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:49:29,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:49:29,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:49:29,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccdf98cd9cbcdc945c981 HTTP/1.1" 200 None 2026-04-03 07:49:29,851 - lava_callback - ERROR - Error processing callback for node 69cccdf98cd9cbcdc945c981: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57776 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:50:36,865 - lava_callback - INFO - Received callback for node 69ccce458cd9cbcdc945c99e INFO: 172.21.0.1:57786 - "POST /node/69ccce458cd9cbcdc945c99e HTTP/1.0" 202 Accepted 2026-04-03 07:50:36,871 - lava_callback - INFO - Starting async job submit for node 69ccce458cd9cbcdc945c99e 2026-04-03 07:50:36,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:50:36,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccce458cd9cbcdc945c99e HTTP/1.1" 200 None 2026-04-03 07:50:36,887 - lava_callback - DEBUG - Processing node 69ccce458cd9cbcdc945c99e: baseline-arm64-baylibre 2026-04-03 07:50:36,887 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccce458cd9cbcdc945c99e 2026-04-03 07:50:36,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:50:36,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccce458cd9cbcdc945c99e HTTP/1.1" 200 None 2026-04-03 07:50:36,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:50:36,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:50:36,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:50:36,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccce458cd9cbcdc945c99e HTTP/1.1" 200 None 2026-04-03 07:50:36,958 - lava_callback - ERROR - Error processing callback for node 69ccce458cd9cbcdc945c99e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41334 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34036 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:52:09,844 - lava_callback - INFO - Received callback for node 69cccea48cd9cbcdc945c9c4 INFO: 172.21.0.1:55758 - "POST /node/69cccea48cd9cbcdc945c9c4 HTTP/1.0" 202 Accepted 2026-04-03 07:52:09,850 - lava_callback - INFO - Starting async job submit for node 69cccea48cd9cbcdc945c9c4 2026-04-03 07:52:09,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:52:09,864 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccea48cd9cbcdc945c9c4 HTTP/1.1" 200 None 2026-04-03 07:52:09,865 - lava_callback - DEBUG - Processing node 69cccea48cd9cbcdc945c9c4: baseline-arm64-clabbe 2026-04-03 07:52:09,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccea48cd9cbcdc945c9c4 2026-04-03 07:52:09,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:52:09,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccea48cd9cbcdc945c9c4 HTTP/1.1" 200 None 2026-04-03 07:52:09,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:52:09,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:52:09,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:52:09,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccea48cd9cbcdc945c9c4 HTTP/1.1" 200 None 2026-04-03 07:52:09,928 - lava_callback - ERROR - Error processing callback for node 69cccea48cd9cbcdc945c9c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56022 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:53:36,905 - lava_callback - INFO - Received callback for node 69cccef18cd9cbcdc945c9e8 INFO: 172.21.0.1:56032 - "POST /node/69cccef18cd9cbcdc945c9e8 HTTP/1.0" 202 Accepted 2026-04-03 07:53:36,914 - lava_callback - INFO - Starting async job submit for node 69cccef18cd9cbcdc945c9e8 2026-04-03 07:53:36,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:53:36,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccef18cd9cbcdc945c9e8 HTTP/1.1" 200 None 2026-04-03 07:53:36,936 - lava_callback - DEBUG - Processing node 69cccef18cd9cbcdc945c9e8: baseline-arm64-baylibre 2026-04-03 07:53:36,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccef18cd9cbcdc945c9e8 2026-04-03 07:53:36,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:53:36,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccef18cd9cbcdc945c9e8 HTTP/1.1" 200 None 2026-04-03 07:53:36,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:53:36,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:53:36,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:53:37,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccef18cd9cbcdc945c9e8 HTTP/1.1" 200 None 2026-04-03 07:53:37,015 - lava_callback - ERROR - Error processing callback for node 69cccef18cd9cbcdc945c9e8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58332 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:55:09,936 - lava_callback - INFO - Received callback for node 69cccf518cd9cbcdc945ca0d INFO: 172.21.0.1:47550 - "POST /node/69cccf518cd9cbcdc945ca0d HTTP/1.0" 202 Accepted 2026-04-03 07:55:09,942 - lava_callback - INFO - Starting async job submit for node 69cccf518cd9cbcdc945ca0d 2026-04-03 07:55:09,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:55:09,965 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccf518cd9cbcdc945ca0d HTTP/1.1" 200 None 2026-04-03 07:55:09,966 - lava_callback - DEBUG - Processing node 69cccf518cd9cbcdc945ca0d: baseline-arm64-clabbe 2026-04-03 07:55:09,966 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccf518cd9cbcdc945ca0d 2026-04-03 07:55:09,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:55:09,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccf518cd9cbcdc945ca0d HTTP/1.1" 200 None 2026-04-03 07:55:09,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:55:09,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:55:09,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:55:10,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccf518cd9cbcdc945ca0d HTTP/1.1" 200 None 2026-04-03 07:55:10,017 - lava_callback - ERROR - Error processing callback for node 69cccf518cd9cbcdc945ca0d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36748 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56684 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:56:37,148 - lava_callback - INFO - Received callback for node 69cccfa28cd9cbcdc945ca2a INFO: 172.21.0.1:56694 - "POST /node/69cccfa28cd9cbcdc945ca2a HTTP/1.0" 202 Accepted 2026-04-03 07:56:37,155 - lava_callback - INFO - Starting async job submit for node 69cccfa28cd9cbcdc945ca2a 2026-04-03 07:56:37,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:56:37,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccfa28cd9cbcdc945ca2a HTTP/1.1" 200 None 2026-04-03 07:56:37,172 - lava_callback - DEBUG - Processing node 69cccfa28cd9cbcdc945ca2a: baseline-arm64-baylibre 2026-04-03 07:56:37,172 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cccfa28cd9cbcdc945ca2a 2026-04-03 07:56:37,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:56:37,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cccfa28cd9cbcdc945ca2a HTTP/1.1" 200 None 2026-04-03 07:56:37,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:56:37,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:56:37,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:56:37,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cccfa28cd9cbcdc945ca2a HTTP/1.1" 200 None 2026-04-03 07:56:37,246 - lava_callback - ERROR - Error processing callback for node 69cccfa28cd9cbcdc945ca2a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41650 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51314 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:58:10,109 - lava_callback - INFO - Received callback for node 69ccd0008cd9cbcdc945ca44 INFO: 172.21.0.1:36632 - "POST /node/69ccd0008cd9cbcdc945ca44 HTTP/1.0" 202 Accepted 2026-04-03 07:58:10,115 - lava_callback - INFO - Starting async job submit for node 69ccd0008cd9cbcdc945ca44 2026-04-03 07:58:10,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:58:10,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd0008cd9cbcdc945ca44 HTTP/1.1" 200 None 2026-04-03 07:58:10,131 - lava_callback - DEBUG - Processing node 69ccd0008cd9cbcdc945ca44: baseline-arm64-clabbe 2026-04-03 07:58:10,131 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd0008cd9cbcdc945ca44 2026-04-03 07:58:10,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:58:10,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd0008cd9cbcdc945ca44 HTTP/1.1" 200 None 2026-04-03 07:58:10,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:58:10,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:58:10,161 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:58:10,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd0008cd9cbcdc945ca44 HTTP/1.1" 200 None 2026-04-03 07:58:10,187 - lava_callback - ERROR - Error processing callback for node 69ccd0008cd9cbcdc945ca44: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60248 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60262 - "GET / HTTP/1.0" 200 OK 2026-04-03 07:59:17,227 - lava_callback - INFO - Received callback for node 69ccd0518cd9cbcdc945ca69 INFO: 172.21.0.1:60264 - "POST /node/69ccd0518cd9cbcdc945ca69 HTTP/1.0" 202 Accepted 2026-04-03 07:59:17,233 - lava_callback - INFO - Starting async job submit for node 69ccd0518cd9cbcdc945ca69 2026-04-03 07:59:17,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:59:17,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd0518cd9cbcdc945ca69 HTTP/1.1" 200 None 2026-04-03 07:59:17,250 - lava_callback - DEBUG - Processing node 69ccd0518cd9cbcdc945ca69: baseline-arm64-baylibre 2026-04-03 07:59:17,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd0518cd9cbcdc945ca69 2026-04-03 07:59:17,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:59:17,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd0518cd9cbcdc945ca69 HTTP/1.1" 200 None 2026-04-03 07:59:17,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:59:17,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 07:59:17,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 07:59:17,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd0518cd9cbcdc945ca69 HTTP/1.1" 200 None 2026-04-03 07:59:17,311 - lava_callback - ERROR - Error processing callback for node 69ccd0518cd9cbcdc945ca69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46766 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:00:50,194 - lava_callback - INFO - Received callback for node 69ccd0af8cd9cbcdc945ca91 INFO: 172.21.0.1:36858 - "POST /node/69ccd0af8cd9cbcdc945ca91 HTTP/1.0" 202 Accepted 2026-04-03 08:00:50,199 - lava_callback - INFO - Starting async job submit for node 69ccd0af8cd9cbcdc945ca91 2026-04-03 08:00:50,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:00:50,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd0af8cd9cbcdc945ca91 HTTP/1.1" 200 None 2026-04-03 08:00:50,215 - lava_callback - DEBUG - Processing node 69ccd0af8cd9cbcdc945ca91: baseline-arm64-clabbe 2026-04-03 08:00:50,216 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd0af8cd9cbcdc945ca91 2026-04-03 08:00:50,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:00:50,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd0af8cd9cbcdc945ca91 HTTP/1.1" 200 None 2026-04-03 08:00:50,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:00:50,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 08:00:50,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:00:50,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd0af8cd9cbcdc945ca91 HTTP/1.1" 200 None 2026-04-03 08:00:50,282 - lava_callback - ERROR - Error processing callback for node 69ccd0af8cd9cbcdc945ca91: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40290 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:03:37,563 - lava_callback - INFO - Received callback for node 69ccd1548cd9cbcdc945cad3 INFO: 172.21.0.1:40302 - "POST /node/69ccd1548cd9cbcdc945cad3 HTTP/1.0" 202 Accepted 2026-04-03 08:03:37,568 - lava_callback - INFO - Starting async job submit for node 69ccd1548cd9cbcdc945cad3 2026-04-03 08:03:37,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:03:37,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd1548cd9cbcdc945cad3 HTTP/1.1" 200 None 2026-04-03 08:03:37,591 - lava_callback - DEBUG - Processing node 69ccd1548cd9cbcdc945cad3: baseline-arm64-baylibre 2026-04-03 08:03:37,591 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd1548cd9cbcdc945cad3 2026-04-03 08:03:37,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:03:37,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd1548cd9cbcdc945cad3 HTTP/1.1" 200 None 2026-04-03 08:03:37,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:03:37,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 08:03:37,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:03:37,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd1548cd9cbcdc945cad3 HTTP/1.1" 200 None 2026-04-03 08:03:37,647 - lava_callback - ERROR - Error processing callback for node 69ccd1548cd9cbcdc945cad3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39132 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:05:30,456 - lava_callback - INFO - Received callback for node 69ccd1b78cd9cbcdc945caf8 INFO: 172.21.0.1:52032 - "POST /node/69ccd1b78cd9cbcdc945caf8 HTTP/1.0" 202 Accepted 2026-04-03 08:05:30,461 - lava_callback - INFO - Starting async job submit for node 69ccd1b78cd9cbcdc945caf8 2026-04-03 08:05:30,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:05:30,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd1b78cd9cbcdc945caf8 HTTP/1.1" 200 None 2026-04-03 08:05:30,478 - lava_callback - DEBUG - Processing node 69ccd1b78cd9cbcdc945caf8: baseline-arm64-clabbe 2026-04-03 08:05:30,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd1b78cd9cbcdc945caf8 2026-04-03 08:05:30,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:05:30,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd1b78cd9cbcdc945caf8 HTTP/1.1" 200 None 2026-04-03 08:05:30,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:05:30,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 08:05:30,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:05:30,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd1b78cd9cbcdc945caf8 HTTP/1.1" 200 None 2026-04-03 08:05:30,545 - lava_callback - ERROR - Error processing callback for node 69ccd1b78cd9cbcdc945caf8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44200 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52476 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37556 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:08:10,531 - lava_callback - INFO - Received callback for node 69ccd2618cd9cbcdc945cb45 INFO: 172.21.0.1:33460 - "POST /node/69ccd2618cd9cbcdc945cb45 HTTP/1.0" 202 Accepted 2026-04-03 08:08:10,537 - lava_callback - INFO - Starting async job submit for node 69ccd2618cd9cbcdc945cb45 2026-04-03 08:08:10,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:08:10,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd2618cd9cbcdc945cb45 HTTP/1.1" 200 None 2026-04-03 08:08:10,552 - lava_callback - DEBUG - Processing node 69ccd2618cd9cbcdc945cb45: baseline-arm64-clabbe 2026-04-03 08:08:10,552 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd2618cd9cbcdc945cb45 2026-04-03 08:08:10,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:08:10,566 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd2618cd9cbcdc945cb45 HTTP/1.1" 200 None 2026-04-03 08:08:10,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:08:10,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 08:08:10,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:08:10,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd2618cd9cbcdc945cb45 HTTP/1.1" 200 None 2026-04-03 08:08:10,603 - lava_callback - ERROR - Error processing callback for node 69ccd2618cd9cbcdc945cb45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52100 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:09:37,722 - lava_callback - INFO - Received callback for node 69ccd2ae8cd9cbcdc945cb6c INFO: 172.21.0.1:52110 - "POST /node/69ccd2ae8cd9cbcdc945cb6c HTTP/1.0" 202 Accepted 2026-04-03 08:09:37,728 - lava_callback - INFO - Starting async job submit for node 69ccd2ae8cd9cbcdc945cb6c 2026-04-03 08:09:37,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:09:37,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd2ae8cd9cbcdc945cb6c HTTP/1.1" 200 None 2026-04-03 08:09:37,744 - lava_callback - DEBUG - Processing node 69ccd2ae8cd9cbcdc945cb6c: baseline-arm64-baylibre 2026-04-03 08:09:37,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd2ae8cd9cbcdc945cb6c 2026-04-03 08:09:37,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:09:37,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd2ae8cd9cbcdc945cb6c HTTP/1.1" 200 None 2026-04-03 08:09:37,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:09:37,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-04-03 08:09:37,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:09:37,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd2ae8cd9cbcdc945cb6c HTTP/1.1" 200 None 2026-04-03 08:09:37,801 - lava_callback - ERROR - Error processing callback for node 69ccd2ae8cd9cbcdc945cb6c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44998 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:11:10,917 - lava_callback - INFO - Received callback for node 69ccd30f8cd9cbcdc945cb86 INFO: 172.21.0.1:36508 - "POST /node/69ccd30f8cd9cbcdc945cb86 HTTP/1.0" 202 Accepted 2026-04-03 08:11:10,923 - lava_callback - INFO - Starting async job submit for node 69ccd30f8cd9cbcdc945cb86 2026-04-03 08:11:10,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:11:10,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd30f8cd9cbcdc945cb86 HTTP/1.1" 200 None 2026-04-03 08:11:10,939 - lava_callback - DEBUG - Processing node 69ccd30f8cd9cbcdc945cb86: baseline-arm64-clabbe 2026-04-03 08:11:10,939 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd30f8cd9cbcdc945cb86 2026-04-03 08:11:10,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:11:10,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd30f8cd9cbcdc945cb86 HTTP/1.1" 200 None 2026-04-03 08:11:10,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:11:10,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 08:11:10,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:11:10,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd30f8cd9cbcdc945cb86 HTTP/1.1" 200 None 2026-04-03 08:11:10,998 - lava_callback - ERROR - Error processing callback for node 69ccd30f8cd9cbcdc945cb86: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48578 - "GET / HTTP/1.0" 200 OK 2026-04-03 08:12:30,772 - lava_callback - INFO - Received callback for node 69ccd3658cd9cbcdc945cba5 INFO: 172.21.0.1:45064 - "POST /node/69ccd3658cd9cbcdc945cba5 HTTP/1.0" 202 Accepted 2026-04-03 08:12:30,777 - lava_callback - INFO - Starting async job submit for node 69ccd3658cd9cbcdc945cba5 2026-04-03 08:12:30,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:12:30,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd3658cd9cbcdc945cba5 HTTP/1.1" 200 None 2026-04-03 08:12:30,793 - lava_callback - DEBUG - Processing node 69ccd3658cd9cbcdc945cba5: baseline-arm64-clabbe 2026-04-03 08:12:30,793 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ccd3658cd9cbcdc945cba5 2026-04-03 08:12:30,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:12:30,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd3658cd9cbcdc945cba5 HTTP/1.1" 200 None 2026-04-03 08:12:30,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:12:30,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c9c1283c3aafdab4be7756 HTTP/1.1" 200 None 2026-04-03 08:12:30,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-03 08:12:30,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ccd3658cd9cbcdc945cba5 HTTP/1.1" 200 None 2026-04-03 08:12:30,849 - lava_callback - ERROR - Error processing callback for node 69ccd3658cd9cbcdc945cba5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37462 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]