INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:49050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39716 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:17:04,684 - lava_callback - INFO - Received callback for node 69c47aec590270ad7b6d6974 2026-03-28 00:17:04,694 - lava_callback - INFO - Starting async job submit for node 69c47aec590270ad7b6d6974 INFO: 172.21.0.1:54576 - "POST /node/69c47aec590270ad7b6d6974 HTTP/1.0" 202 Accepted 2026-03-28 00:17:04,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:17:04,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47aec590270ad7b6d6974 HTTP/1.1" 200 None 2026-03-28 00:17:04,737 - lava_callback - DEBUG - Processing node 69c47aec590270ad7b6d6974: baseline-arm64-baylibre 2026-03-28 00:17:04,743 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47aec590270ad7b6d6974 2026-03-28 00:17:04,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:17:04,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47aec590270ad7b6d6974 HTTP/1.1" 200 None 2026-03-28 00:17:04,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:17:04,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:17:04,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:17:04,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47aec590270ad7b6d6974 HTTP/1.1" 200 None 2026-03-28 00:17:04,871 - lava_callback - ERROR - Error processing callback for node 69c47aec590270ad7b6d6974: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55330 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:18:24,434 - lava_callback - INFO - Received callback for node 69c47b3e590270ad7b6d6990 INFO: 172.21.0.1:41098 - "POST /node/69c47b3e590270ad7b6d6990 HTTP/1.0" 202 Accepted 2026-03-28 00:18:24,440 - lava_callback - INFO - Starting async job submit for node 69c47b3e590270ad7b6d6990 2026-03-28 00:18:24,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:18:24,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3e590270ad7b6d6990 HTTP/1.1" 200 None 2026-03-28 00:18:24,457 - lava_callback - DEBUG - Processing node 69c47b3e590270ad7b6d6990: baseline-arm64-baylibre 2026-03-28 00:18:24,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47b3e590270ad7b6d6990 2026-03-28 00:18:24,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:18:24,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3e590270ad7b6d6990 HTTP/1.1" 200 None 2026-03-28 00:18:24,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:18:24,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:18:24,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:18:24,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47b3e590270ad7b6d6990 HTTP/1.1" 200 None 2026-03-28 00:18:24,509 - lava_callback - ERROR - Error processing callback for node 69c47b3e590270ad7b6d6990: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55790 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:20:04,539 - lava_callback - INFO - Received callback for node 69c47ba1590270ad7b6d69d4 INFO: 172.21.0.1:40662 - "POST /node/69c47ba1590270ad7b6d69d4 HTTP/1.0" 202 Accepted 2026-03-28 00:20:04,545 - lava_callback - INFO - Starting async job submit for node 69c47ba1590270ad7b6d69d4 2026-03-28 00:20:04,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:20:04,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47ba1590270ad7b6d69d4 HTTP/1.1" 200 None 2026-03-28 00:20:04,561 - lava_callback - DEBUG - Processing node 69c47ba1590270ad7b6d69d4: baseline-arm64-baylibre 2026-03-28 00:20:04,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47ba1590270ad7b6d69d4 2026-03-28 00:20:04,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:20:04,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47ba1590270ad7b6d69d4 HTTP/1.1" 200 None 2026-03-28 00:20:04,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:20:04,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:20:04,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:20:04,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47ba1590270ad7b6d69d4 HTTP/1.1" 200 None 2026-03-28 00:20:04,618 - lava_callback - ERROR - Error processing callback for node 69c47ba1590270ad7b6d69d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:21:24,603 - lava_callback - INFO - Received callback for node 69c47bf5590270ad7b6d69e3 INFO: 172.21.0.1:59576 - "POST /node/69c47bf5590270ad7b6d69e3 HTTP/1.0" 202 Accepted 2026-03-28 00:21:24,608 - lava_callback - INFO - Starting async job submit for node 69c47bf5590270ad7b6d69e3 2026-03-28 00:21:24,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:21:24,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47bf5590270ad7b6d69e3 HTTP/1.1" 200 None 2026-03-28 00:21:24,625 - lava_callback - DEBUG - Processing node 69c47bf5590270ad7b6d69e3: baseline-arm64-baylibre 2026-03-28 00:21:24,625 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47bf5590270ad7b6d69e3 2026-03-28 00:21:24,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:21:24,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47bf5590270ad7b6d69e3 HTTP/1.1" 200 None 2026-03-28 00:21:24,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:21:24,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:21:24,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:21:24,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47bf5590270ad7b6d69e3 HTTP/1.1" 200 None 2026-03-28 00:21:24,739 - lava_callback - ERROR - Error processing callback for node 69c47bf5590270ad7b6d69e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43086 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:22:44,669 - lava_callback - INFO - Received callback for node 69c47c46590270ad7b6d69f3 INFO: 172.21.0.1:42872 - "POST /node/69c47c46590270ad7b6d69f3 HTTP/1.0" 202 Accepted 2026-03-28 00:22:44,675 - lava_callback - INFO - Starting async job submit for node 69c47c46590270ad7b6d69f3 2026-03-28 00:22:44,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:22:44,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47c46590270ad7b6d69f3 HTTP/1.1" 200 None 2026-03-28 00:22:44,702 - lava_callback - DEBUG - Processing node 69c47c46590270ad7b6d69f3: baseline-arm64-baylibre 2026-03-28 00:22:44,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47c46590270ad7b6d69f3 2026-03-28 00:22:44,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:22:44,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47c46590270ad7b6d69f3 HTTP/1.1" 200 None 2026-03-28 00:22:44,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:22:44,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:22:44,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:22:44,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47c46590270ad7b6d69f3 HTTP/1.1" 200 None 2026-03-28 00:22:44,761 - lava_callback - ERROR - Error processing callback for node 69c47c46590270ad7b6d69f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52052 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:24:04,723 - lava_callback - INFO - Received callback for node 69c47c98590270ad7b6d69fe INFO: 172.21.0.1:57702 - "POST /node/69c47c98590270ad7b6d69fe HTTP/1.0" 202 Accepted 2026-03-28 00:24:04,729 - lava_callback - INFO - Starting async job submit for node 69c47c98590270ad7b6d69fe 2026-03-28 00:24:04,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:24:04,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47c98590270ad7b6d69fe HTTP/1.1" 200 None 2026-03-28 00:24:04,745 - lava_callback - DEBUG - Processing node 69c47c98590270ad7b6d69fe: baseline-arm64-baylibre 2026-03-28 00:24:04,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47c98590270ad7b6d69fe 2026-03-28 00:24:04,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:24:04,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47c98590270ad7b6d69fe HTTP/1.1" 200 None 2026-03-28 00:24:04,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:24:04,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:24:04,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:24:04,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47c98590270ad7b6d69fe HTTP/1.1" 200 None 2026-03-28 00:24:04,803 - lava_callback - ERROR - Error processing callback for node 69c47c98590270ad7b6d69fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50276 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:25:41,806 - lava_callback - INFO - Received callback for node 69c47cf0590270ad7b6d6a14 INFO: 172.21.0.1:39308 - "POST /node/69c47cf0590270ad7b6d6a14 HTTP/1.0" 202 Accepted 2026-03-28 00:25:41,811 - lava_callback - INFO - Starting async job submit for node 69c47cf0590270ad7b6d6a14 2026-03-28 00:25:41,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:25:41,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47cf0590270ad7b6d6a14 HTTP/1.1" 200 None 2026-03-28 00:25:41,842 - lava_callback - DEBUG - Processing node 69c47cf0590270ad7b6d6a14: baseline-arm64-clabbe 2026-03-28 00:25:41,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47cf0590270ad7b6d6a14 2026-03-28 00:25:41,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:25:41,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47cf0590270ad7b6d6a14 HTTP/1.1" 200 None 2026-03-28 00:25:41,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:25:41,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:25:41,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:25:41,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47cf0590270ad7b6d6a14 HTTP/1.1" 200 None 2026-03-28 00:25:41,902 - lava_callback - ERROR - Error processing callback for node 69c47cf0590270ad7b6d6a14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47436 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:27:01,780 - lava_callback - INFO - Received callback for node 69c47d4f590270ad7b6d6a22 INFO: 172.21.0.1:46310 - "POST /node/69c47d4f590270ad7b6d6a22 HTTP/1.0" 202 Accepted 2026-03-28 00:27:01,786 - lava_callback - INFO - Starting async job submit for node 69c47d4f590270ad7b6d6a22 2026-03-28 00:27:01,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:27:01,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47d4f590270ad7b6d6a22 HTTP/1.1" 200 None 2026-03-28 00:27:01,802 - lava_callback - DEBUG - Processing node 69c47d4f590270ad7b6d6a22: baseline-arm64-clabbe 2026-03-28 00:27:01,802 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47d4f590270ad7b6d6a22 2026-03-28 00:27:01,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:27:01,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47d4f590270ad7b6d6a22 HTTP/1.1" 200 None 2026-03-28 00:27:01,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:27:01,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:27:01,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:27:01,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47d4f590270ad7b6d6a22 HTTP/1.1" 200 None 2026-03-28 00:27:01,868 - lava_callback - ERROR - Error processing callback for node 69c47d4f590270ad7b6d6a22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33202 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:28:21,871 - lava_callback - INFO - Received callback for node 69c47da1590270ad7b6d6a34 INFO: 172.21.0.1:39882 - "POST /node/69c47da1590270ad7b6d6a34 HTTP/1.0" 202 Accepted 2026-03-28 00:28:21,877 - lava_callback - INFO - Starting async job submit for node 69c47da1590270ad7b6d6a34 2026-03-28 00:28:21,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:28:21,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47da1590270ad7b6d6a34 HTTP/1.1" 200 None 2026-03-28 00:28:21,918 - lava_callback - DEBUG - Processing node 69c47da1590270ad7b6d6a34: baseline-arm64-clabbe 2026-03-28 00:28:21,918 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47da1590270ad7b6d6a34 2026-03-28 00:28:21,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:28:21,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47da1590270ad7b6d6a34 HTTP/1.1" 200 None 2026-03-28 00:28:21,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:28:21,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:28:21,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:28:21,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47da1590270ad7b6d6a34 HTTP/1.1" 200 None 2026-03-28 00:28:21,982 - lava_callback - ERROR - Error processing callback for node 69c47da1590270ad7b6d6a34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41778 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:30:01,890 - lava_callback - INFO - Received callback for node 69c47df5590270ad7b6d8fdc INFO: 172.21.0.1:43460 - "POST /node/69c47df5590270ad7b6d8fdc HTTP/1.0" 202 Accepted 2026-03-28 00:30:01,895 - lava_callback - INFO - Starting async job submit for node 69c47df5590270ad7b6d8fdc 2026-03-28 00:30:01,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:30:01,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47df5590270ad7b6d8fdc HTTP/1.1" 200 None 2026-03-28 00:30:01,914 - lava_callback - DEBUG - Processing node 69c47df5590270ad7b6d8fdc: baseline-arm64-clabbe 2026-03-28 00:30:01,914 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47df5590270ad7b6d8fdc 2026-03-28 00:30:01,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:30:01,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47df5590270ad7b6d8fdc HTTP/1.1" 200 None 2026-03-28 00:30:01,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:30:01,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:30:01,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:30:02,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47df5590270ad7b6d8fdc HTTP/1.1" 200 None 2026-03-28 00:30:02,146 - lava_callback - ERROR - Error processing callback for node 69c47df5590270ad7b6d8fdc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57928 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:31:21,961 - lava_callback - INFO - Received callback for node 69c47e4a590270ad7b6d8ff5 INFO: 172.21.0.1:41034 - "POST /node/69c47e4a590270ad7b6d8ff5 HTTP/1.0" 202 Accepted 2026-03-28 00:31:21,967 - lava_callback - INFO - Starting async job submit for node 69c47e4a590270ad7b6d8ff5 2026-03-28 00:31:21,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:31:21,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e4a590270ad7b6d8ff5 HTTP/1.1" 200 None 2026-03-28 00:31:21,983 - lava_callback - DEBUG - Processing node 69c47e4a590270ad7b6d8ff5: baseline-arm64-clabbe 2026-03-28 00:31:21,983 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47e4a590270ad7b6d8ff5 2026-03-28 00:31:21,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:31:21,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e4a590270ad7b6d8ff5 HTTP/1.1" 200 None 2026-03-28 00:31:21,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:31:22,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:31:22,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:31:22,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47e4a590270ad7b6d8ff5 HTTP/1.1" 200 None 2026-03-28 00:31:22,048 - lava_callback - ERROR - Error processing callback for node 69c47e4a590270ad7b6d8ff5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50750 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:32:41,983 - lava_callback - INFO - Received callback for node 69c47e9d590270ad7b6d900d INFO: 172.21.0.1:37132 - "POST /node/69c47e9d590270ad7b6d900d HTTP/1.0" 202 Accepted 2026-03-28 00:32:41,989 - lava_callback - INFO - Starting async job submit for node 69c47e9d590270ad7b6d900d 2026-03-28 00:32:41,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:32:42,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e9d590270ad7b6d900d HTTP/1.1" 200 None 2026-03-28 00:32:42,006 - lava_callback - DEBUG - Processing node 69c47e9d590270ad7b6d900d: baseline-arm64-clabbe 2026-03-28 00:32:42,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47e9d590270ad7b6d900d 2026-03-28 00:32:42,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:32:42,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e9d590270ad7b6d900d HTTP/1.1" 200 None 2026-03-28 00:32:42,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:32:42,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:32:42,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:32:42,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47e9d590270ad7b6d900d HTTP/1.1" 200 None 2026-03-28 00:32:42,062 - lava_callback - ERROR - Error processing callback for node 69c47e9d590270ad7b6d900d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37654 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:37:05,561 - lava_callback - INFO - Received callback for node 69c47fa4590270ad7b6d902b INFO: 172.21.0.1:45762 - "POST /node/69c47fa4590270ad7b6d902b HTTP/1.0" 202 Accepted 2026-03-28 00:37:05,570 - lava_callback - INFO - Starting async job submit for node 69c47fa4590270ad7b6d902b 2026-03-28 00:37:05,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:05,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47fa4590270ad7b6d902b HTTP/1.1" 200 None 2026-03-28 00:37:05,600 - lava_callback - DEBUG - Processing node 69c47fa4590270ad7b6d902b: baseline-arm-baylibre 2026-03-28 00:37:05,600 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47fa4590270ad7b6d902b 2026-03-28 00:37:05,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:05,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47fa4590270ad7b6d902b HTTP/1.1" 200 None 2026-03-28 00:37:05,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:05,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:37:05,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:05,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47fa4590270ad7b6d902b HTTP/1.1" 200 None 2026-03-28 00:37:05,692 - lava_callback - ERROR - Error processing callback for node 69c47fa4590270ad7b6d902b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39708 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:37:28,979 - lava_callback - INFO - Received callback for node 69c722a829015d0f9598f82e INFO: 172.21.0.1:39716 - "POST /node/69c722a829015d0f9598f82e HTTP/1.0" 202 Accepted 2026-03-28 00:37:28,988 - lava_callback - INFO - Starting async job submit for node 69c722a829015d0f9598f82e 2026-03-28 00:37:29,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:29,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722a829015d0f9598f82e HTTP/1.1" 200 None 2026-03-28 00:37:29,052 - lava_callback - DEBUG - Processing node 69c722a829015d0f9598f82e: baseline-x86 2026-03-28 00:37:29,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:29,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:29,781 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a829015d0f9598f82e/log.txt.gz 2026-03-28 00:37:29,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:30,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:30,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a829015d0f9598f82e/lava-logs.yaml 2026-03-28 00:37:30,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:31,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:31,131 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a829015d0f9598f82e/lava_callback.json.gz 2026-03-28 00:37:31,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:31,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722a829015d0f9598f82e HTTP/1.1" 200 None 2026-03-28 00:37:31,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:31,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 00:37:31,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:31,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c722a829015d0f9598f82e HTTP/1.1" 200 None 2026-03-28 00:37:31,266 - lava_callback - INFO - Completed processing callback for node 69c722a829015d0f9598f82e 2026-03-28 00:37:34,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:34,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:37:39,640 - lava_callback - INFO - Received callback for node 69c722ab29015d0f9598f837 INFO: 172.21.0.1:43380 - "POST /node/69c722ab29015d0f9598f837 HTTP/1.0" 202 Accepted 2026-03-28 00:37:39,647 - lava_callback - INFO - Starting async job submit for node 69c722ab29015d0f9598f837 2026-03-28 00:37:39,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:39,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722ab29015d0f9598f837 HTTP/1.1" 200 None 2026-03-28 00:37:39,681 - lava_callback - DEBUG - Processing node 69c722ab29015d0f9598f837: baseline-x86 2026-03-28 00:37:39,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:39,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:39,965 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c722ab29015d0f9598f837/log.txt.gz 2026-03-28 00:37:40,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:40,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:40,238 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c722ab29015d0f9598f837/lava-logs.yaml 2026-03-28 00:37:40,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:40,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:40,373 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c722ab29015d0f9598f837/lava_callback.json.gz 2026-03-28 00:37:40,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:40,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722ab29015d0f9598f837 HTTP/1.1" 200 None 2026-03-28 00:37:40,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:40,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 00:37:40,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:40,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c722ab29015d0f9598f837 HTTP/1.1" 200 None 2026-03-28 00:37:40,452 - lava_callback - INFO - Completed processing callback for node 69c722ab29015d0f9598f837 2026-03-28 00:37:45,475 - lava_callback - INFO - Received callback for node 69c47fcb590270ad7b6d903e INFO: 172.21.0.1:35054 - "POST /node/69c47fcb590270ad7b6d903e HTTP/1.0" 202 Accepted 2026-03-28 00:37:45,481 - lava_callback - INFO - Starting async job submit for node 69c47fcb590270ad7b6d903e 2026-03-28 00:37:45,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:45,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47fcb590270ad7b6d903e HTTP/1.1" 200 None 2026-03-28 00:37:45,497 - lava_callback - DEBUG - Processing node 69c47fcb590270ad7b6d903e: baseline-arm-baylibre 2026-03-28 00:37:45,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47fcb590270ad7b6d903e 2026-03-28 00:37:45,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:45,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47fcb590270ad7b6d903e HTTP/1.1" 200 None 2026-03-28 00:37:45,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:45,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:37:45,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:45,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47fcb590270ad7b6d903e HTTP/1.1" 200 None 2026-03-28 00:37:45,550 - lava_callback - ERROR - Error processing callback for node 69c47fcb590270ad7b6d903e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59470 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:38:04,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:04,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47962 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:38:25,559 - lava_callback - INFO - Received callback for node 69c47ff1590270ad7b6d9049 INFO: 172.21.0.1:54416 - "POST /node/69c47ff1590270ad7b6d9049 HTTP/1.0" 202 Accepted 2026-03-28 00:38:25,566 - lava_callback - INFO - Starting async job submit for node 69c47ff1590270ad7b6d9049 2026-03-28 00:38:25,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:25,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47ff1590270ad7b6d9049 HTTP/1.1" 200 None 2026-03-28 00:38:25,609 - lava_callback - DEBUG - Processing node 69c47ff1590270ad7b6d9049: baseline-arm-baylibre 2026-03-28 00:38:25,610 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47ff1590270ad7b6d9049 2026-03-28 00:38:25,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:25,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47ff1590270ad7b6d9049 HTTP/1.1" 200 None 2026-03-28 00:38:25,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:25,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:38:25,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:25,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47ff1590270ad7b6d9049 HTTP/1.1" 200 None 2026-03-28 00:38:25,696 - lava_callback - ERROR - Error processing callback for node 69c47ff1590270ad7b6d9049: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43564 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:38:36,786 - lava_callback - INFO - Received callback for node 69c722a929015d0f9598f831 INFO: 172.21.0.1:43578 - "POST /node/69c722a929015d0f9598f831 HTTP/1.0" 202 Accepted 2026-03-28 00:38:36,795 - lava_callback - INFO - Starting async job submit for node 69c722a929015d0f9598f831 2026-03-28 00:38:36,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:36,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722a929015d0f9598f831 HTTP/1.1" 200 None 2026-03-28 00:38:36,862 - lava_callback - DEBUG - Processing node 69c722a929015d0f9598f831: baseline-x86 2026-03-28 00:38:37,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:38:37,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:38:37,669 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a929015d0f9598f831/log.txt.gz 2026-03-28 00:38:37,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:38:38,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:38:38,375 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a929015d0f9598f831/lava-logs.yaml 2026-03-28 00:38:38,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:38:38,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:38:38,569 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a929015d0f9598f831/lava_callback.json.gz 2026-03-28 00:38:38,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:38,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722a929015d0f9598f831 HTTP/1.1" 200 None 2026-03-28 00:38:38,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:38,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 00:38:38,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:38,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c722a929015d0f9598f831 HTTP/1.1" 200 None 2026-03-28 00:38:38,661 - lava_callback - INFO - Completed processing callback for node 69c722a929015d0f9598f831 INFO: 172.21.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34862 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:39:04,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:04,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 00:39:05,527 - lava_callback - INFO - Received callback for node 69c48017590270ad7b6d9055 INFO: 172.21.0.1:55452 - "POST /node/69c48017590270ad7b6d9055 HTTP/1.0" 202 Accepted 2026-03-28 00:39:05,533 - lava_callback - INFO - Starting async job submit for node 69c48017590270ad7b6d9055 2026-03-28 00:39:05,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:05,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48017590270ad7b6d9055 HTTP/1.1" 200 None 2026-03-28 00:39:05,549 - lava_callback - DEBUG - Processing node 69c48017590270ad7b6d9055: baseline-arm-baylibre 2026-03-28 00:39:05,549 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48017590270ad7b6d9055 2026-03-28 00:39:05,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:05,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48017590270ad7b6d9055 HTTP/1.1" 200 None 2026-03-28 00:39:05,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:05,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:39:05,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:05,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48017590270ad7b6d9055 HTTP/1.1" 200 None 2026-03-28 00:39:05,606 - lava_callback - ERROR - Error processing callback for node 69c48017590270ad7b6d9055: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43830 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:39:45,930 - lava_callback - INFO - Received callback for node 69c4803d590270ad7b6d9064 INFO: 172.21.0.1:56868 - "POST /node/69c4803d590270ad7b6d9064 HTTP/1.0" 202 Accepted 2026-03-28 00:39:45,936 - lava_callback - INFO - Starting async job submit for node 69c4803d590270ad7b6d9064 2026-03-28 00:39:45,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:45,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4803d590270ad7b6d9064 HTTP/1.1" 200 None 2026-03-28 00:39:45,956 - lava_callback - DEBUG - Processing node 69c4803d590270ad7b6d9064: baseline-arm-baylibre 2026-03-28 00:39:45,956 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4803d590270ad7b6d9064 2026-03-28 00:39:45,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:45,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4803d590270ad7b6d9064 HTTP/1.1" 200 None 2026-03-28 00:39:45,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:45,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:39:45,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:46,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4803d590270ad7b6d9064 HTTP/1.1" 200 None 2026-03-28 00:39:46,008 - lava_callback - ERROR - Error processing callback for node 69c4803d590270ad7b6d9064: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60396 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:40:05,612 - lava_callback - INFO - Received callback for node 69c48062590270ad7b6d9071 INFO: 172.21.0.1:42430 - "POST /node/69c48062590270ad7b6d9071 HTTP/1.0" 202 Accepted 2026-03-28 00:40:05,618 - lava_callback - INFO - Starting async job submit for node 69c48062590270ad7b6d9071 2026-03-28 00:40:05,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:05,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48062590270ad7b6d9071 HTTP/1.1" 200 None 2026-03-28 00:40:05,634 - lava_callback - DEBUG - Processing node 69c48062590270ad7b6d9071: baseline-arm-baylibre 2026-03-28 00:40:05,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48062590270ad7b6d9071 2026-03-28 00:40:05,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:05,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48062590270ad7b6d9071 HTTP/1.1" 200 None 2026-03-28 00:40:05,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:05,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:40:05,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:05,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48062590270ad7b6d9071 HTTP/1.1" 200 None 2026-03-28 00:40:05,690 - lava_callback - ERROR - Error processing callback for node 69c48062590270ad7b6d9071: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43632 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:40:25,591 - lava_callback - INFO - Received callback for node 69c48075590270ad7b6d909d INFO: 172.21.0.1:59354 - "POST /node/69c48075590270ad7b6d909d HTTP/1.0" 202 Accepted 2026-03-28 00:40:25,597 - lava_callback - INFO - Starting async job submit for node 69c48075590270ad7b6d909d 2026-03-28 00:40:25,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:25,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48075590270ad7b6d909d HTTP/1.1" 200 None 2026-03-28 00:40:25,619 - lava_callback - DEBUG - Processing node 69c48075590270ad7b6d909d: baseline-arm-baylibre 2026-03-28 00:40:25,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48075590270ad7b6d909d 2026-03-28 00:40:25,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:25,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48075590270ad7b6d909d HTTP/1.1" 200 None 2026-03-28 00:40:25,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:25,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:25,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:25,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48075590270ad7b6d909d HTTP/1.1" 200 None 2026-03-28 00:40:25,683 - lava_callback - ERROR - Error processing callback for node 69c48075590270ad7b6d909d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:40:26,275 - lava_callback - INFO - Received callback for node 69c48077590270ad7b6d90a0 INFO: 172.21.0.1:59370 - "POST /node/69c48077590270ad7b6d90a0 HTTP/1.0" 202 Accepted 2026-03-28 00:40:26,281 - lava_callback - INFO - Starting async job submit for node 69c48077590270ad7b6d90a0 2026-03-28 00:40:26,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:26,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48077590270ad7b6d90a0 HTTP/1.1" 200 None 2026-03-28 00:40:26,297 - lava_callback - DEBUG - Processing node 69c48077590270ad7b6d90a0: baseline-arm-baylibre 2026-03-28 00:40:26,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48077590270ad7b6d90a0 2026-03-28 00:40:26,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:26,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48077590270ad7b6d90a0 HTTP/1.1" 200 None 2026-03-28 00:40:26,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:26,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:26,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:26,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48077590270ad7b6d90a0 HTTP/1.1" 200 None 2026-03-28 00:40:26,350 - lava_callback - ERROR - Error processing callback for node 69c48077590270ad7b6d90a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56542 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:40:42,476 - lava_callback - INFO - Received callback for node 69c48085590270ad7b6d90e6 INFO: 172.21.0.1:42796 - "POST /node/69c48085590270ad7b6d90e6 HTTP/1.0" 202 Accepted 2026-03-28 00:40:42,482 - lava_callback - INFO - Starting async job submit for node 69c48085590270ad7b6d90e6 2026-03-28 00:40:42,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:42,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48085590270ad7b6d90e6 HTTP/1.1" 200 None 2026-03-28 00:40:42,500 - lava_callback - DEBUG - Processing node 69c48085590270ad7b6d90e6: baseline-arm-clabbe 2026-03-28 00:40:42,500 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48085590270ad7b6d90e6 2026-03-28 00:40:42,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:42,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48085590270ad7b6d90e6 HTTP/1.1" 200 None 2026-03-28 00:40:42,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:42,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:42,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:42,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48085590270ad7b6d90e6 HTTP/1.1" 200 None 2026-03-28 00:40:42,558 - lava_callback - ERROR - Error processing callback for node 69c48085590270ad7b6d90e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:40:45,597 - lava_callback - INFO - Received callback for node 69c48078590270ad7b6d90a3 INFO: 172.21.0.1:42800 - "POST /node/69c48078590270ad7b6d90a3 HTTP/1.0" 202 Accepted 2026-03-28 00:40:45,602 - lava_callback - INFO - Starting async job submit for node 69c48078590270ad7b6d90a3 2026-03-28 00:40:45,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:45,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48078590270ad7b6d90a3 HTTP/1.1" 200 None 2026-03-28 00:40:45,633 - lava_callback - DEBUG - Processing node 69c48078590270ad7b6d90a3: baseline-arm-baylibre 2026-03-28 00:40:45,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48078590270ad7b6d90a3 2026-03-28 00:40:45,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:45,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48078590270ad7b6d90a3 HTTP/1.1" 200 None 2026-03-28 00:40:45,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:45,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:45,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:45,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48078590270ad7b6d90a3 HTTP/1.1" 200 None 2026-03-28 00:40:45,690 - lava_callback - ERROR - Error processing callback for node 69c48078590270ad7b6d90a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:40:46,263 - lava_callback - INFO - Received callback for node 69c48079590270ad7b6d90a6 INFO: 172.21.0.1:42802 - "POST /node/69c48079590270ad7b6d90a6 HTTP/1.0" 202 Accepted 2026-03-28 00:40:46,268 - lava_callback - INFO - Starting async job submit for node 69c48079590270ad7b6d90a6 2026-03-28 00:40:46,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:46,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48079590270ad7b6d90a6 HTTP/1.1" 200 None 2026-03-28 00:40:46,286 - lava_callback - DEBUG - Processing node 69c48079590270ad7b6d90a6: baseline-arm-baylibre 2026-03-28 00:40:46,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48079590270ad7b6d90a6 2026-03-28 00:40:46,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:46,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48079590270ad7b6d90a6 HTTP/1.1" 200 None 2026-03-28 00:40:46,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:46,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:46,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:46,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48079590270ad7b6d90a6 HTTP/1.1" 200 None 2026-03-28 00:40:46,375 - lava_callback - ERROR - Error processing callback for node 69c48079590270ad7b6d90a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42810 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:40:47,006 - lava_callback - INFO - Received callback for node 69c4807c590270ad7b6d90cb INFO: 172.21.0.1:42822 - "POST /node/69c4807c590270ad7b6d90cb HTTP/1.0" 202 Accepted 2026-03-28 00:40:47,011 - lava_callback - INFO - Starting async job submit for node 69c4807c590270ad7b6d90cb 2026-03-28 00:40:47,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4807c590270ad7b6d90cb HTTP/1.1" 200 None 2026-03-28 00:40:47,027 - lava_callback - DEBUG - Processing node 69c4807c590270ad7b6d90cb: baseline-arm-baylibre 2026-03-28 00:40:47,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4807c590270ad7b6d90cb 2026-03-28 00:40:47,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4807c590270ad7b6d90cb HTTP/1.1" 200 None 2026-03-28 00:40:47,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:47,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4807c590270ad7b6d90cb HTTP/1.1" 200 None 2026-03-28 00:40:47,080 - lava_callback - ERROR - Error processing callback for node 69c4807c590270ad7b6d90cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:40:47,656 - lava_callback - INFO - Received callback for node 69c4807e590270ad7b6d90ce INFO: 172.21.0.1:42830 - "POST /node/69c4807e590270ad7b6d90ce HTTP/1.0" 202 Accepted 2026-03-28 00:40:47,662 - lava_callback - INFO - Starting async job submit for node 69c4807e590270ad7b6d90ce 2026-03-28 00:40:47,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4807e590270ad7b6d90ce HTTP/1.1" 200 None 2026-03-28 00:40:47,681 - lava_callback - DEBUG - Processing node 69c4807e590270ad7b6d90ce: baseline-arm-baylibre 2026-03-28 00:40:47,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4807e590270ad7b6d90ce 2026-03-28 00:40:47,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4807e590270ad7b6d90ce HTTP/1.1" 200 None 2026-03-28 00:40:47,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:47,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4807e590270ad7b6d90ce HTTP/1.1" 200 None 2026-03-28 00:40:47,736 - lava_callback - ERROR - Error processing callback for node 69c4807e590270ad7b6d90ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:40:48,755 - lava_callback - INFO - Received callback for node 69c7231029015d0f9598f86f INFO: 172.21.0.1:42838 - "POST /node/69c7231029015d0f9598f86f HTTP/1.0" 202 Accepted 2026-03-28 00:40:48,761 - lava_callback - INFO - Starting async job submit for node 69c7231029015d0f9598f86f 2026-03-28 00:40:48,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:48,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7231029015d0f9598f86f HTTP/1.1" 200 None 2026-03-28 00:40:48,804 - lava_callback - DEBUG - Processing node 69c7231029015d0f9598f86f: baseline-arm 2026-03-28 00:40:48,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:40:49,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:40:49,174 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7231029015d0f9598f86f/log.txt.gz 2026-03-28 00:40:49,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:40:49,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:40:49,510 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7231029015d0f9598f86f/lava-logs.yaml 2026-03-28 00:40:49,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:40:49,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:40:49,681 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7231029015d0f9598f86f/lava_callback.json.gz 2026-03-28 00:40:49,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:49,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7231029015d0f9598f86f HTTP/1.1" 200 None 2026-03-28 00:40:49,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:49,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4229015d0f9598f742 HTTP/1.1" 200 None 2026-03-28 00:40:49,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:49,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7231029015d0f9598f86f HTTP/1.1" 200 None 2026-03-28 00:40:49,766 - lava_callback - INFO - Completed processing callback for node 69c7231029015d0f9598f86f INFO: 172.21.0.1:40152 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:41:02,364 - lava_callback - INFO - Received callback for node 69c48088590270ad7b6d90f3 INFO: 172.21.0.1:53828 - "POST /node/69c48088590270ad7b6d90f3 HTTP/1.0" 202 Accepted 2026-03-28 00:41:02,369 - lava_callback - INFO - Starting async job submit for node 69c48088590270ad7b6d90f3 2026-03-28 00:41:02,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:02,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48088590270ad7b6d90f3 HTTP/1.1" 200 None 2026-03-28 00:41:02,392 - lava_callback - DEBUG - Processing node 69c48088590270ad7b6d90f3: baseline-arm-clabbe 2026-03-28 00:41:02,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48088590270ad7b6d90f3 2026-03-28 00:41:02,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:02,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48088590270ad7b6d90f3 HTTP/1.1" 200 None 2026-03-28 00:41:02,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:02,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:41:02,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:02,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48088590270ad7b6d90f3 HTTP/1.1" 200 None 2026-03-28 00:41:02,446 - lava_callback - ERROR - Error processing callback for node 69c48088590270ad7b6d90f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:41:03,046 - lava_callback - INFO - Received callback for node 69c4808a590270ad7b6d90fc INFO: 172.21.0.1:53834 - "POST /node/69c4808a590270ad7b6d90fc HTTP/1.0" 202 Accepted 2026-03-28 00:41:03,052 - lava_callback - INFO - Starting async job submit for node 69c4808a590270ad7b6d90fc 2026-03-28 00:41:03,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808a590270ad7b6d90fc HTTP/1.1" 200 None 2026-03-28 00:41:03,068 - lava_callback - DEBUG - Processing node 69c4808a590270ad7b6d90fc: baseline-arm-clabbe 2026-03-28 00:41:03,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4808a590270ad7b6d90fc 2026-03-28 00:41:03,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808a590270ad7b6d90fc HTTP/1.1" 200 None 2026-03-28 00:41:03,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:41:03,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4808a590270ad7b6d90fc HTTP/1.1" 200 None 2026-03-28 00:41:03,120 - lava_callback - ERROR - Error processing callback for node 69c4808a590270ad7b6d90fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:41:03,716 - lava_callback - INFO - Received callback for node 69c4808b590270ad7b6d90ff INFO: 172.21.0.1:53850 - "POST /node/69c4808b590270ad7b6d90ff HTTP/1.0" 202 Accepted 2026-03-28 00:41:03,722 - lava_callback - INFO - Starting async job submit for node 69c4808b590270ad7b6d90ff 2026-03-28 00:41:03,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808b590270ad7b6d90ff HTTP/1.1" 200 None 2026-03-28 00:41:03,740 - lava_callback - DEBUG - Processing node 69c4808b590270ad7b6d90ff: baseline-arm-clabbe 2026-03-28 00:41:03,740 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4808b590270ad7b6d90ff 2026-03-28 00:41:03,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808b590270ad7b6d90ff HTTP/1.1" 200 None 2026-03-28 00:41:03,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:41:03,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4808b590270ad7b6d90ff HTTP/1.1" 200 None 2026-03-28 00:41:03,796 - lava_callback - ERROR - Error processing callback for node 69c4808b590270ad7b6d90ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:41:04,395 - lava_callback - INFO - Received callback for node 69c4808c590270ad7b6d9102 INFO: 172.21.0.1:53854 - "POST /node/69c4808c590270ad7b6d9102 HTTP/1.0" 202 Accepted 2026-03-28 00:41:04,401 - lava_callback - INFO - Starting async job submit for node 69c4808c590270ad7b6d9102 2026-03-28 00:41:04,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:04,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808c590270ad7b6d9102 HTTP/1.1" 200 None 2026-03-28 00:41:04,417 - lava_callback - DEBUG - Processing node 69c4808c590270ad7b6d9102: baseline-arm-clabbe 2026-03-28 00:41:04,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4808c590270ad7b6d9102 2026-03-28 00:41:04,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:04,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808c590270ad7b6d9102 HTTP/1.1" 200 None 2026-03-28 00:41:04,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:04,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:41:04,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:04,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4808c590270ad7b6d9102 HTTP/1.1" 200 None 2026-03-28 00:41:04,476 - lava_callback - ERROR - Error processing callback for node 69c4808c590270ad7b6d9102: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:41:04,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:04,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 00:41:05,078 - lava_callback - INFO - Received callback for node 69c4808f590270ad7b6d910b INFO: 172.21.0.1:53860 - "POST /node/69c4808f590270ad7b6d910b HTTP/1.0" 202 Accepted 2026-03-28 00:41:05,084 - lava_callback - INFO - Starting async job submit for node 69c4808f590270ad7b6d910b 2026-03-28 00:41:05,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808f590270ad7b6d910b HTTP/1.1" 200 None 2026-03-28 00:41:05,099 - lava_callback - DEBUG - Processing node 69c4808f590270ad7b6d910b: baseline-arm-clabbe 2026-03-28 00:41:05,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4808f590270ad7b6d910b 2026-03-28 00:41:05,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808f590270ad7b6d910b HTTP/1.1" 200 None 2026-03-28 00:41:05,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:41:05,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4808f590270ad7b6d910b HTTP/1.1" 200 None 2026-03-28 00:41:05,170 - lava_callback - ERROR - Error processing callback for node 69c4808f590270ad7b6d910b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:41:05,779 - lava_callback - INFO - Received callback for node 69c48090590270ad7b6d910e INFO: 172.21.0.1:53866 - "POST /node/69c48090590270ad7b6d910e HTTP/1.0" 202 Accepted 2026-03-28 00:41:05,784 - lava_callback - INFO - Starting async job submit for node 69c48090590270ad7b6d910e 2026-03-28 00:41:05,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48090590270ad7b6d910e HTTP/1.1" 200 None 2026-03-28 00:41:05,823 - lava_callback - DEBUG - Processing node 69c48090590270ad7b6d910e: baseline-arm-kontron 2026-03-28 00:41:05,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48090590270ad7b6d910e 2026-03-28 00:41:05,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48090590270ad7b6d910e HTTP/1.1" 200 None 2026-03-28 00:41:05,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:41:05,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48090590270ad7b6d910e HTTP/1.1" 200 None 2026-03-28 00:41:05,879 - lava_callback - ERROR - Error processing callback for node 69c48090590270ad7b6d910e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46388 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:41:22,354 - lava_callback - INFO - Received callback for node 69c480ad590270ad7b6d916d INFO: 172.21.0.1:56302 - "POST /node/69c480ad590270ad7b6d916d HTTP/1.0" 202 Accepted 2026-03-28 00:41:22,360 - lava_callback - INFO - Starting async job submit for node 69c480ad590270ad7b6d916d 2026-03-28 00:41:22,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:22,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480ad590270ad7b6d916d HTTP/1.1" 200 None 2026-03-28 00:41:22,376 - lava_callback - DEBUG - Processing node 69c480ad590270ad7b6d916d: baseline-arm-clabbe 2026-03-28 00:41:22,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c480ad590270ad7b6d916d 2026-03-28 00:41:22,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:22,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480ad590270ad7b6d916d HTTP/1.1" 200 None 2026-03-28 00:41:22,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:22,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:41:22,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:22,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480ad590270ad7b6d916d HTTP/1.1" 200 None 2026-03-28 00:41:22,431 - lava_callback - ERROR - Error processing callback for node 69c480ad590270ad7b6d916d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43646 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:42:02,392 - lava_callback - INFO - Received callback for node 69c480d4590270ad7b6d91bd INFO: 172.21.0.1:37764 - "POST /node/69c480d4590270ad7b6d91bd HTTP/1.0" 202 Accepted 2026-03-28 00:42:02,398 - lava_callback - INFO - Starting async job submit for node 69c480d4590270ad7b6d91bd 2026-03-28 00:42:02,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:02,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480d4590270ad7b6d91bd HTTP/1.1" 200 None 2026-03-28 00:42:02,414 - lava_callback - DEBUG - Processing node 69c480d4590270ad7b6d91bd: baseline-arm-clabbe 2026-03-28 00:42:02,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c480d4590270ad7b6d91bd 2026-03-28 00:42:02,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:02,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480d4590270ad7b6d91bd HTTP/1.1" 200 None 2026-03-28 00:42:02,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:02,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:42:02,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:02,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480d4590270ad7b6d91bd HTTP/1.1" 200 None 2026-03-28 00:42:02,477 - lava_callback - ERROR - Error processing callback for node 69c480d4590270ad7b6d91bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47744 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:42:42,412 - lava_callback - INFO - Received callback for node 69c480fb590270ad7b6d91f8 INFO: 172.21.0.1:45518 - "POST /node/69c480fb590270ad7b6d91f8 HTTP/1.0" 202 Accepted 2026-03-28 00:42:42,418 - lava_callback - INFO - Starting async job submit for node 69c480fb590270ad7b6d91f8 2026-03-28 00:42:42,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:42,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480fb590270ad7b6d91f8 HTTP/1.1" 200 None 2026-03-28 00:42:42,434 - lava_callback - DEBUG - Processing node 69c480fb590270ad7b6d91f8: baseline-arm-clabbe 2026-03-28 00:42:42,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c480fb590270ad7b6d91f8 2026-03-28 00:42:42,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:42,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480fb590270ad7b6d91f8 HTTP/1.1" 200 None 2026-03-28 00:42:42,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:42,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:42:42,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:42,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480fb590270ad7b6d91f8 HTTP/1.1" 200 None 2026-03-28 00:42:42,509 - lava_callback - ERROR - Error processing callback for node 69c480fb590270ad7b6d91f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57842 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:43:22,487 - lava_callback - INFO - Received callback for node 69c48121590270ad7b6d9217 INFO: 172.21.0.1:48998 - "POST /node/69c48121590270ad7b6d9217 HTTP/1.0" 202 Accepted 2026-03-28 00:43:22,493 - lava_callback - INFO - Starting async job submit for node 69c48121590270ad7b6d9217 2026-03-28 00:43:22,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:43:22,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48121590270ad7b6d9217 HTTP/1.1" 200 None 2026-03-28 00:43:22,509 - lava_callback - DEBUG - Processing node 69c48121590270ad7b6d9217: baseline-arm-clabbe 2026-03-28 00:43:22,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48121590270ad7b6d9217 2026-03-28 00:43:22,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:43:22,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48121590270ad7b6d9217 HTTP/1.1" 200 None 2026-03-28 00:43:22,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:43:22,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:43:22,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:43:22,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48121590270ad7b6d9217 HTTP/1.1" 200 None 2026-03-28 00:43:22,585 - lava_callback - ERROR - Error processing callback for node 69c48121590270ad7b6d9217: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40256 - "GET /node/694271c01b61f1c6005f7244%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:39634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52858 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:48:44,274 - lava_callback - INFO - Received callback for node 69c7256529015d0f9598f935 INFO: 172.21.0.1:49826 - "POST /node/69c7256529015d0f9598f935 HTTP/1.0" 202 Accepted 2026-03-28 00:48:44,281 - lava_callback - INFO - Starting async job submit for node 69c7256529015d0f9598f935 2026-03-28 00:48:44,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:48:44,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256529015d0f9598f935 HTTP/1.1" 200 None 2026-03-28 00:48:44,298 - lava_callback - DEBUG - Processing node 69c7256529015d0f9598f935: baseline-arm 2026-03-28 00:48:44,303 - lava_callback - WARNING - Failed to upload log 2026-03-28 00:48:44,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:48:44,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:48:44,449 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256529015d0f9598f935/lava-logs.yaml 2026-03-28 00:48:44,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:48:44,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:48:44,638 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256529015d0f9598f935/lava_callback.json.gz 2026-03-28 00:48:44,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:48:44,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256529015d0f9598f935 HTTP/1.1" 200 None 2026-03-28 00:48:44,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:48:44,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:48:44,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:48:44,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256529015d0f9598f935 HTTP/1.1" 200 None 2026-03-28 00:48:44,706 - lava_callback - INFO - Completed processing callback for node 69c7256529015d0f9598f935 INFO: 172.21.0.1:49840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47948 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:49:04,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:05,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 00:49:06,137 - lava_callback - INFO - Received callback for node 69c7256629015d0f9598f938 INFO: 172.21.0.1:54616 - "POST /node/69c7256629015d0f9598f938 HTTP/1.0" 202 Accepted 2026-03-28 00:49:06,186 - lava_callback - INFO - Starting async job submit for node 69c7256629015d0f9598f938 2026-03-28 00:49:06,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:06,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256629015d0f9598f938 HTTP/1.1" 200 None 2026-03-28 00:49:06,243 - lava_callback - DEBUG - Processing node 69c7256629015d0f9598f938: baseline-arm 2026-03-28 00:49:06,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:06,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:06,517 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f938/log.txt.gz INFO: 172.21.0.1:54620 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:49:06,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:06,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:06,888 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f938/lava-logs.yaml 2026-03-28 00:49:06,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:07,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:07,004 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f938/lava_callback.json.gz 2026-03-28 00:49:07,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:07,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256629015d0f9598f938 HTTP/1.1" 200 None 2026-03-28 00:49:07,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:07,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:49:07,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:07,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256629015d0f9598f938 HTTP/1.1" 200 None 2026-03-28 00:49:07,082 - lava_callback - INFO - Completed processing callback for node 69c7256629015d0f9598f938 INFO: 172.21.0.1:35910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35916 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:49:17,132 - lava_callback - INFO - Received callback for node 69c7256829015d0f9598f942 INFO: 172.21.0.1:35928 - "POST /node/69c7256829015d0f9598f942 HTTP/1.0" 202 Accepted 2026-03-28 00:49:17,185 - lava_callback - INFO - Starting async job submit for node 69c7256829015d0f9598f942 2026-03-28 00:49:17,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:17,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256829015d0f9598f942 HTTP/1.1" 200 None 2026-03-28 00:49:17,237 - lava_callback - DEBUG - Processing node 69c7256829015d0f9598f942: baseline-arm 2026-03-28 00:49:17,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:17,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:17,574 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256829015d0f9598f942/log.txt.gz 2026-03-28 00:49:17,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:17,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:17,819 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256829015d0f9598f942/lava-logs.yaml 2026-03-28 00:49:17,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:17,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:17,953 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256829015d0f9598f942/lava_callback.json.gz 2026-03-28 00:49:17,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:17,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256829015d0f9598f942 HTTP/1.1" 200 None 2026-03-28 00:49:17,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:18,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:49:18,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:18,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256829015d0f9598f942 HTTP/1.1" 200 None 2026-03-28 00:49:18,036 - lava_callback - INFO - Completed processing callback for node 69c7256829015d0f9598f942 INFO: 172.21.0.1:37462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37476 - "GET /node/6986faf86e077e4126a7d395%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-28 00:49:35,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:35,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49414 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:50:00,057 - lava_callback - INFO - Received callback for node 69c7256729015d0f9598f93f INFO: 172.21.0.1:45854 - "POST /node/69c7256729015d0f9598f93f HTTP/1.0" 202 Accepted 2026-03-28 00:50:00,063 - lava_callback - INFO - Starting async job submit for node 69c7256729015d0f9598f93f 2026-03-28 00:50:00,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:00,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256729015d0f9598f93f HTTP/1.1" 200 None 2026-03-28 00:50:00,100 - lava_callback - DEBUG - Processing node 69c7256729015d0f9598f93f: baseline-arm 2026-03-28 00:50:00,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:00,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:00,545 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256729015d0f9598f93f/log.txt.gz 2026-03-28 00:50:00,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:00,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:00,878 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256729015d0f9598f93f/lava-logs.yaml 2026-03-28 00:50:00,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:01,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:01,016 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256729015d0f9598f93f/lava_callback.json.gz 2026-03-28 00:50:01,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:01,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256729015d0f9598f93f HTTP/1.1" 200 None 2026-03-28 00:50:01,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:01,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:01,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:01,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256729015d0f9598f93f HTTP/1.1" 200 None 2026-03-28 00:50:01,102 - lava_callback - INFO - Completed processing callback for node 69c7256729015d0f9598f93f 2026-03-28 00:50:05,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:05,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45870 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:50:09,003 - lava_callback - INFO - Received callback for node 69c7256429015d0f9598f932 INFO: 172.21.0.1:45886 - "POST /node/69c7256429015d0f9598f932 HTTP/1.0" 202 Accepted 2026-03-28 00:50:09,009 - lava_callback - INFO - Starting async job submit for node 69c7256429015d0f9598f932 2026-03-28 00:50:09,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:09,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256429015d0f9598f932 HTTP/1.1" 200 None 2026-03-28 00:50:09,051 - lava_callback - DEBUG - Processing node 69c7256429015d0f9598f932: baseline-arm 2026-03-28 00:50:09,267 - lava_callback - INFO - Received callback for node 69c7257729015d0f9598f974 2026-03-28 00:50:09,273 - lava_callback - INFO - Starting async job submit for node 69c7257729015d0f9598f974 INFO: 172.21.0.1:45894 - "POST /node/69c7257729015d0f9598f974 HTTP/1.0" 202 Accepted 2026-03-28 00:50:09,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:09,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:09,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257729015d0f9598f974 HTTP/1.1" 200 None 2026-03-28 00:50:09,320 - lava_callback - DEBUG - Processing node 69c7257729015d0f9598f974: baseline-arm-broonie 2026-03-28 00:50:09,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:09,489 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256429015d0f9598f932/log.txt.gz 2026-03-28 00:50:09,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:09,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:09,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:09,788 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257729015d0f9598f974/log.txt.gz 2026-03-28 00:50:09,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:09,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:09,959 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256429015d0f9598f932/lava-logs.yaml 2026-03-28 00:50:09,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:10,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:10,173 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256429015d0f9598f932/lava_callback.json.gz 2026-03-28 00:50:10,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:10,199 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257729015d0f9598f974/lava-logs.yaml 2026-03-28 00:50:10,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:10,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256429015d0f9598f932 HTTP/1.1" 200 None 2026-03-28 00:50:10,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:10,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256429015d0f9598f932 HTTP/1.1" 200 None 2026-03-28 00:50:10,255 - lava_callback - INFO - Completed processing callback for node 69c7256429015d0f9598f932 2026-03-28 00:50:10,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:10,381 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257729015d0f9598f974/lava_callback.json.gz 2026-03-28 00:50:10,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257729015d0f9598f974 HTTP/1.1" 200 None 2026-03-28 00:50:10,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:10,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257729015d0f9598f974 HTTP/1.1" 200 None 2026-03-28 00:50:10,504 - lava_callback - INFO - Completed processing callback for node 69c7257729015d0f9598f974 2026-03-28 00:50:10,825 - lava_callback - INFO - Received callback for node 69c7257829015d0f9598f977 INFO: 172.21.0.1:39902 - "POST /node/69c7257829015d0f9598f977 HTTP/1.0" 202 Accepted 2026-03-28 00:50:10,831 - lava_callback - INFO - Starting async job submit for node 69c7257829015d0f9598f977 2026-03-28 00:50:10,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257829015d0f9598f977 HTTP/1.1" 200 None 2026-03-28 00:50:10,874 - lava_callback - DEBUG - Processing node 69c7257829015d0f9598f977: baseline-arm-broonie 2026-03-28 00:50:11,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:11,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:11,246 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257829015d0f9598f977/log.txt.gz 2026-03-28 00:50:11,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:11,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:11,579 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257829015d0f9598f977/lava-logs.yaml 2026-03-28 00:50:11,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:11,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:11,738 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257829015d0f9598f977/lava_callback.json.gz 2026-03-28 00:50:11,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:11,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257829015d0f9598f977 HTTP/1.1" 200 None 2026-03-28 00:50:11,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:11,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:11,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:11,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257829015d0f9598f977 HTTP/1.1" 200 None 2026-03-28 00:50:11,833 - lava_callback - INFO - Completed processing callback for node 69c7257829015d0f9598f977 INFO: 172.21.0.1:39912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38086 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:50:28,918 - lava_callback - INFO - Received callback for node 69c7258829015d0f9598f9d8 INFO: 172.21.0.1:38094 - "POST /node/69c7258829015d0f9598f9d8 HTTP/1.0" 202 Accepted 2026-03-28 00:50:28,924 - lava_callback - INFO - Starting async job submit for node 69c7258829015d0f9598f9d8 2026-03-28 00:50:28,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:28,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258829015d0f9598f9d8 HTTP/1.1" 200 None 2026-03-28 00:50:28,962 - lava_callback - DEBUG - Processing node 69c7258829015d0f9598f9d8: baseline-arm-pengutronix 2026-03-28 00:50:29,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:29,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:29,221 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258829015d0f9598f9d8/log.txt.gz 2026-03-28 00:50:29,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:29,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:29,477 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258829015d0f9598f9d8/lava-logs.yaml 2026-03-28 00:50:29,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:29,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:29,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258829015d0f9598f9d8/lava_callback.json.gz 2026-03-28 00:50:29,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:29,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258829015d0f9598f9d8 HTTP/1.1" 200 None 2026-03-28 00:50:29,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:29,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:29,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:29,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258829015d0f9598f9d8 HTTP/1.1" 200 None 2026-03-28 00:50:29,712 - lava_callback - INFO - Completed processing callback for node 69c7258829015d0f9598f9d8 2026-03-28 00:50:31,590 - lava_callback - INFO - Received callback for node 69c7258929015d0f9598f9dc INFO: 172.21.0.1:41070 - "POST /node/69c7258929015d0f9598f9dc HTTP/1.0" 202 Accepted 2026-03-28 00:50:31,599 - lava_callback - INFO - Starting async job submit for node 69c7258929015d0f9598f9dc 2026-03-28 00:50:31,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:31,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258929015d0f9598f9dc HTTP/1.1" 200 None 2026-03-28 00:50:31,652 - lava_callback - DEBUG - Processing node 69c7258929015d0f9598f9dc: baseline-arm-pengutronix 2026-03-28 00:50:31,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:31,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:31,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258929015d0f9598f9dc/log.txt.gz 2026-03-28 00:50:32,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:32,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:32,185 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258929015d0f9598f9dc/lava-logs.yaml 2026-03-28 00:50:32,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:32,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:32,318 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258929015d0f9598f9dc/lava_callback.json.gz 2026-03-28 00:50:32,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:32,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258929015d0f9598f9dc HTTP/1.1" 200 None 2026-03-28 00:50:32,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:32,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:32,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:32,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258929015d0f9598f9dc HTTP/1.1" 200 None 2026-03-28 00:50:32,399 - lava_callback - INFO - Completed processing callback for node 69c7258929015d0f9598f9dc 2026-03-28 00:50:35,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:35,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:41076 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:50:35,457 - lava_callback - INFO - Received callback for node 69c7258c29015d0f9598f9e4 INFO: 172.21.0.1:41092 - "POST /node/69c7258c29015d0f9598f9e4 HTTP/1.0" 202 Accepted 2026-03-28 00:50:35,508 - lava_callback - INFO - Starting async job submit for node 69c7258c29015d0f9598f9e4 2026-03-28 00:50:35,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:35,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258c29015d0f9598f9e4 HTTP/1.1" 200 None 2026-03-28 00:50:35,579 - lava_callback - DEBUG - Processing node 69c7258c29015d0f9598f9e4: baseline-arm-pengutronix 2026-03-28 00:50:35,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:36,066 - lava_callback - INFO - Received callback for node 69c7257e29015d0f9598f98b INFO: 172.21.0.1:41104 - "POST /node/69c7257e29015d0f9598f98b HTTP/1.0" 202 Accepted 2026-03-28 00:50:36,077 - lava_callback - INFO - Starting async job submit for node 69c7257e29015d0f9598f98b 2026-03-28 00:50:36,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:36,118 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258c29015d0f9598f9e4/log.txt.gz 2026-03-28 00:50:36,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:36,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257e29015d0f9598f98b HTTP/1.1" 200 None 2026-03-28 00:50:36,208 - lava_callback - DEBUG - Processing node 69c7257e29015d0f9598f98b: baseline-arm-clabbe 2026-03-28 00:50:36,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:41118 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:50:36,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:36,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:36,839 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258c29015d0f9598f9e4/lava-logs.yaml 2026-03-28 00:50:36,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:37,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:37,049 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257e29015d0f9598f98b/log.txt.gz 2026-03-28 00:50:37,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:37,184 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258c29015d0f9598f9e4/lava_callback.json.gz 2026-03-28 00:50:37,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:37,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258c29015d0f9598f9e4 HTTP/1.1" 200 None 2026-03-28 00:50:37,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:37,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:37,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:37,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:37,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258c29015d0f9598f9e4 HTTP/1.1" 200 None 2026-03-28 00:50:37,408 - lava_callback - INFO - Completed processing callback for node 69c7258c29015d0f9598f9e4 2026-03-28 00:50:37,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:37,945 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257e29015d0f9598f98b/lava-logs.yaml 2026-03-28 00:50:37,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:38,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:38,383 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257e29015d0f9598f98b/lava_callback.json.gz 2026-03-28 00:50:38,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:38,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257e29015d0f9598f98b HTTP/1.1" 200 None 2026-03-28 00:50:38,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:38,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:38,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:38,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257e29015d0f9598f98b HTTP/1.1" 200 None 2026-03-28 00:50:38,529 - lava_callback - INFO - Completed processing callback for node 69c7257e29015d0f9598f98b INFO: 172.21.0.1:53372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36052 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:51:05,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:05,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41298 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:51:25,047 - lava_callback - INFO - Received callback for node 69c725a429015d0f9598fa03 INFO: 172.21.0.1:40478 - "POST /node/69c725a429015d0f9598fa03 HTTP/1.0" 202 Accepted 2026-03-28 00:51:25,055 - lava_callback - INFO - Starting async job submit for node 69c725a429015d0f9598fa03 2026-03-28 00:51:25,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:25,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a429015d0f9598fa03 HTTP/1.1" 200 None 2026-03-28 00:51:25,104 - lava_callback - DEBUG - Processing node 69c725a429015d0f9598fa03: ltp-dio 2026-03-28 00:51:25,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:51:26,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:51:26,613 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c725a429015d0f9598fa03/log.txt.gz 2026-03-28 00:51:26,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:51:27,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:51:27,743 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c725a429015d0f9598fa03/lava-logs.yaml 2026-03-28 00:51:27,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:51:28,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:51:28,661 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c725a429015d0f9598fa03/lava_callback.json.gz 2026-03-28 00:51:28,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:28,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a429015d0f9598fa03 HTTP/1.1" 200 None 2026-03-28 00:51:28,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:28,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:51:28,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:28,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725a429015d0f9598fa03 HTTP/1.1" 200 None 2026-03-28 00:51:28,962 - lava_callback - INFO - Completed processing callback for node 69c725a429015d0f9598fa03 2026-03-28 00:51:35,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:35,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33476 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:52:07,144 - lava_callback - INFO - Received callback for node 69c7257f29015d0f9598f990 INFO: 172.21.0.1:33482 - "POST /node/69c7257f29015d0f9598f990 HTTP/1.0" 202 Accepted 2026-03-28 00:52:07,154 - lava_callback - INFO - Starting async job submit for node 69c7257f29015d0f9598f990 2026-03-28 00:52:07,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:07,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257f29015d0f9598f990 HTTP/1.1" 200 None 2026-03-28 00:52:07,214 - lava_callback - DEBUG - Processing node 69c7257f29015d0f9598f990: baseline-arm-clabbe 2026-03-28 00:52:07,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:08,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:08,915 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257f29015d0f9598f990/log.txt.gz 2026-03-28 00:52:09,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:10,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:10,576 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257f29015d0f9598f990/lava-logs.yaml 2026-03-28 00:52:10,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:11,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:11,937 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257f29015d0f9598f990/lava_callback.json.gz 2026-03-28 00:52:11,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:11,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257f29015d0f9598f990 HTTP/1.1" 200 None 2026-03-28 00:52:11,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:12,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:52:12,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:12,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257f29015d0f9598f990 HTTP/1.1" 200 None 2026-03-28 00:52:12,054 - lava_callback - INFO - Completed processing callback for node 69c7257f29015d0f9598f990 INFO: 172.21.0.1:42368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33158 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:52:35,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:35,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40302 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:52:43,458 - lava_callback - INFO - Received callback for node 69c722aa29015d0f9598f834 INFO: 172.21.0.1:60074 - "POST /node/69c722aa29015d0f9598f834 HTTP/1.0" 202 Accepted 2026-03-28 00:52:43,473 - lava_callback - INFO - Starting async job submit for node 69c722aa29015d0f9598f834 2026-03-28 00:52:43,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:43,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722aa29015d0f9598f834 HTTP/1.1" 200 None 2026-03-28 00:52:43,558 - lava_callback - DEBUG - Processing node 69c722aa29015d0f9598f834: baseline-x86 2026-03-28 00:52:44,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:46,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:46,088 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c722aa29015d0f9598f834/log.txt.gz INFO: 172.21.0.1:60076 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:52:46,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:48,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:48,141 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c722aa29015d0f9598f834/lava-logs.yaml 2026-03-28 00:52:48,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:49,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:49,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c722aa29015d0f9598f834/lava_callback.json.gz DEBUG: Setup failed for 69c722aa29015d0f9598f834. Transitting job node result: incomplete -> fail 2026-03-28 00:52:49,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:49,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722aa29015d0f9598f834 HTTP/1.1" 200 None 2026-03-28 00:52:49,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:49,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 00:52:49,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:49,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c722aa29015d0f9598f834 HTTP/1.1" 200 None 2026-03-28 00:52:49,435 - lava_callback - INFO - Completed processing callback for node 69c722aa29015d0f9598f834 INFO: 172.21.0.1:60276 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:53:05,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:53:05,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54102 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:53:54,307 - lava_callback - INFO - Received callback for node 69c7258229015d0f9598f99d INFO: 172.21.0.1:46830 - "POST /node/69c7258229015d0f9598f99d HTTP/1.0" 202 Accepted 2026-03-28 00:53:54,317 - lava_callback - INFO - Starting async job submit for node 69c7258229015d0f9598f99d 2026-03-28 00:53:54,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:53:54,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258229015d0f9598f99d HTTP/1.1" 200 None 2026-03-28 00:53:54,366 - lava_callback - DEBUG - Processing node 69c7258229015d0f9598f99d: baseline-arm-clabbe 2026-03-28 00:53:54,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:53:55,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:53:55,599 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258229015d0f9598f99d/log.txt.gz 2026-03-28 00:53:55,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:53:56,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:53:56,793 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258229015d0f9598f99d/lava-logs.yaml 2026-03-28 00:53:56,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:53:57,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:53:57,797 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258229015d0f9598f99d/lava_callback.json.gz 2026-03-28 00:53:57,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:53:57,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258229015d0f9598f99d HTTP/1.1" 200 None 2026-03-28 00:53:57,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:53:57,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:53:57,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:53:57,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258229015d0f9598f99d HTTP/1.1" 200 None 2026-03-28 00:53:57,921 - lava_callback - INFO - Completed processing callback for node 69c7258229015d0f9598f99d 2026-03-28 00:54:05,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:05,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60656 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:54:23,131 - lava_callback - INFO - Received callback for node 69c483b6590270ad7b6d9383 INFO: 172.21.0.1:37266 - "POST /node/69c483b6590270ad7b6d9383 HTTP/1.0" 202 Accepted 2026-03-28 00:54:23,139 - lava_callback - INFO - Starting async job submit for node 69c483b6590270ad7b6d9383 2026-03-28 00:54:23,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:23,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c483b6590270ad7b6d9383 HTTP/1.1" 200 None 2026-03-28 00:54:23,158 - lava_callback - DEBUG - Processing node 69c483b6590270ad7b6d9383: baseline-arm-clabbe 2026-03-28 00:54:23,158 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c483b6590270ad7b6d9383 2026-03-28 00:54:23,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:23,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c483b6590270ad7b6d9383 HTTP/1.1" 200 None 2026-03-28 00:54:23,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:23,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 00:54:23,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:23,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c483b6590270ad7b6d9383 HTTP/1.1" 200 None 2026-03-28 00:54:23,239 - lava_callback - ERROR - Error processing callback for node 69c483b6590270ad7b6d9383: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60196 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:54:38,167 - lava_callback - INFO - Received callback for node 69c7257a29015d0f9598f97a INFO: 172.21.0.1:60206 - "POST /node/69c7257a29015d0f9598f97a HTTP/1.0" 202 Accepted 2026-03-28 00:54:38,173 - lava_callback - INFO - Starting async job submit for node 69c7257a29015d0f9598f97a 2026-03-28 00:54:38,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:38,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257a29015d0f9598f97a HTTP/1.1" 200 None 2026-03-28 00:54:38,210 - lava_callback - DEBUG - Processing node 69c7257a29015d0f9598f97a: baseline-arm-broonie 2026-03-28 00:54:38,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:54:38,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:54:38,562 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257a29015d0f9598f97a/log.txt.gz 2026-03-28 00:54:38,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:54:38,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:54:38,865 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257a29015d0f9598f97a/lava-logs.yaml 2026-03-28 00:54:38,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:54:39,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:54:39,009 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257a29015d0f9598f97a/lava_callback.json.gz 2026-03-28 00:54:39,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:39,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257a29015d0f9598f97a HTTP/1.1" 200 None 2026-03-28 00:54:39,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:39,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:54:39,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:39,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257a29015d0f9598f97a HTTP/1.1" 200 None 2026-03-28 00:54:39,094 - lava_callback - INFO - Completed processing callback for node 69c7257a29015d0f9598f97a INFO: 172.21.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39706 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:55:05,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:05,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42890 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:55:29,360 - lava_callback - INFO - Received callback for node 69c7258329015d0f9598f9a4 INFO: 172.21.0.1:42898 - "POST /node/69c7258329015d0f9598f9a4 HTTP/1.0" 202 Accepted 2026-03-28 00:55:29,366 - lava_callback - INFO - Starting async job submit for node 69c7258329015d0f9598f9a4 2026-03-28 00:55:29,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:29,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258329015d0f9598f9a4 HTTP/1.1" 200 None 2026-03-28 00:55:29,411 - lava_callback - DEBUG - Processing node 69c7258329015d0f9598f9a4: baseline-arm-clabbe 2026-03-28 00:55:29,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:55:29,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:55:29,754 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258329015d0f9598f9a4/log.txt.gz 2026-03-28 00:55:29,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:55:30,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:55:30,112 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258329015d0f9598f9a4/lava-logs.yaml 2026-03-28 00:55:30,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:55:30,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:55:30,286 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258329015d0f9598f9a4/lava_callback.json.gz 2026-03-28 00:55:30,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:30,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258329015d0f9598f9a4 HTTP/1.1" 200 None 2026-03-28 00:55:30,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:30,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:55:30,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:30,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258329015d0f9598f9a4 HTTP/1.1" 200 None 2026-03-28 00:55:30,375 - lava_callback - INFO - Completed processing callback for node 69c7258329015d0f9598f9a4 2026-03-28 00:55:35,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:35,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48994 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:55:43,381 - lava_callback - INFO - Received callback for node 69c483fe590270ad7b6d93ac INFO: 172.21.0.1:33102 - "POST /node/69c483fe590270ad7b6d93ac HTTP/1.0" 202 Accepted 2026-03-28 00:55:43,390 - lava_callback - INFO - Starting async job submit for node 69c483fe590270ad7b6d93ac 2026-03-28 00:55:43,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:43,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c483fe590270ad7b6d93ac HTTP/1.1" 200 None 2026-03-28 00:55:43,410 - lava_callback - DEBUG - Processing node 69c483fe590270ad7b6d93ac: baseline-arm-clabbe 2026-03-28 00:55:43,411 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c483fe590270ad7b6d93ac 2026-03-28 00:55:43,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:43,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c483fe590270ad7b6d93ac HTTP/1.1" 200 None 2026-03-28 00:55:43,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:43,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 00:55:43,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:43,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c483fe590270ad7b6d93ac HTTP/1.1" 200 None 2026-03-28 00:55:43,495 - lava_callback - ERROR - Error processing callback for node 69c483fe590270ad7b6d93ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39740 - "GET /node/6999066ad17b46e99fad2aa5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49042 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:56:43,224 - lava_callback - INFO - Received callback for node 69c48444590270ad7b6d93cb INFO: 172.21.0.1:52704 - "POST /node/69c48444590270ad7b6d93cb HTTP/1.0" 202 Accepted 2026-03-28 00:56:43,230 - lava_callback - INFO - Starting async job submit for node 69c48444590270ad7b6d93cb 2026-03-28 00:56:43,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:56:43,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48444590270ad7b6d93cb HTTP/1.1" 200 None 2026-03-28 00:56:43,256 - lava_callback - DEBUG - Processing node 69c48444590270ad7b6d93cb: baseline-arm-clabbe 2026-03-28 00:56:43,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48444590270ad7b6d93cb 2026-03-28 00:56:43,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:56:43,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48444590270ad7b6d93cb HTTP/1.1" 200 None 2026-03-28 00:56:43,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:56:43,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 00:56:43,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:56:43,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48444590270ad7b6d93cb HTTP/1.1" 200 None 2026-03-28 00:56:43,329 - lava_callback - ERROR - Error processing callback for node 69c48444590270ad7b6d93cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51328 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:57:23,190 - lava_callback - INFO - Received callback for node 69c4846b590270ad7b6d93dd INFO: 172.21.0.1:37102 - "POST /node/69c4846b590270ad7b6d93dd HTTP/1.0" 202 Accepted 2026-03-28 00:57:23,197 - lava_callback - INFO - Starting async job submit for node 69c4846b590270ad7b6d93dd 2026-03-28 00:57:23,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:57:23,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4846b590270ad7b6d93dd HTTP/1.1" 200 None 2026-03-28 00:57:23,220 - lava_callback - DEBUG - Processing node 69c4846b590270ad7b6d93dd: baseline-arm-clabbe 2026-03-28 00:57:23,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4846b590270ad7b6d93dd 2026-03-28 00:57:23,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:57:23,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4846b590270ad7b6d93dd HTTP/1.1" 200 None 2026-03-28 00:57:23,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:57:23,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 00:57:23,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:57:23,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4846b590270ad7b6d93dd HTTP/1.1" 200 None 2026-03-28 00:57:23,287 - lava_callback - ERROR - Error processing callback for node 69c4846b590270ad7b6d93dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54530 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:58:09,792 - lava_callback - INFO - Received callback for node 69c7231129015d0f9598f877 INFO: 172.21.0.1:58442 - "POST /node/69c7231129015d0f9598f877 HTTP/1.0" 202 Accepted 2026-03-28 00:58:09,799 - lava_callback - INFO - Starting async job submit for node 69c7231129015d0f9598f877 2026-03-28 00:58:09,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:09,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7231129015d0f9598f877 HTTP/1.1" 200 None 2026-03-28 00:58:09,893 - lava_callback - DEBUG - Processing node 69c7231129015d0f9598f877: ltp-smoke 2026-03-28 00:58:10,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:10,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:10,726 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7231129015d0f9598f877/log.txt.gz 2026-03-28 00:58:11,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:11,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:11,328 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7231129015d0f9598f877/lava-logs.yaml 2026-03-28 00:58:11,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:11,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:11,541 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7231129015d0f9598f877/lava_callback.json.gz 2026-03-28 00:58:11,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:11,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7231129015d0f9598f877 HTTP/1.1" 200 None 2026-03-28 00:58:11,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:11,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4229015d0f9598f742 HTTP/1.1" 200 None 2026-03-28 00:58:11,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:11,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7231129015d0f9598f877 HTTP/1.1" 200 None 2026-03-28 00:58:11,895 - lava_callback - INFO - Completed processing callback for node 69c7231129015d0f9598f877 INFO: 172.21.0.1:58458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58468 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:58:19,835 - lava_callback - INFO - Received callback for node 69c725b329015d0f9598fa2d INFO: 172.21.0.1:50454 - "POST /node/69c725b329015d0f9598fa2d HTTP/1.0" 202 Accepted 2026-03-28 00:58:19,843 - lava_callback - INFO - Starting async job submit for node 69c725b329015d0f9598fa2d 2026-03-28 00:58:19,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:19,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b329015d0f9598fa2d HTTP/1.1" 200 None 2026-03-28 00:58:19,898 - lava_callback - DEBUG - Processing node 69c725b329015d0f9598fa2d: ltp-syscalls 2026-03-28 00:58:20,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:21,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:21,003 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c725b329015d0f9598fa2d/log.txt.gz 2026-03-28 00:58:21,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:21,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:21,827 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c725b329015d0f9598fa2d/lava-logs.yaml 2026-03-28 00:58:21,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:22,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:22,026 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c725b329015d0f9598fa2d/lava_callback.json.gz 2026-03-28 00:58:22,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:22,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b329015d0f9598fa2d HTTP/1.1" 200 None 2026-03-28 00:58:22,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:22,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:58:22,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:22,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725b329015d0f9598fa2d HTTP/1.1" 200 None 2026-03-28 00:58:22,128 - lava_callback - INFO - Completed processing callback for node 69c725b329015d0f9598fa2d INFO: 172.21.0.1:50470 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:58:35,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:35,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47206 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:58:43,275 - lava_callback - INFO - Received callback for node 69c484b2590270ad7b6d93f5 INFO: 172.21.0.1:47858 - "POST /node/69c484b2590270ad7b6d93f5 HTTP/1.0" 202 Accepted 2026-03-28 00:58:43,283 - lava_callback - INFO - Starting async job submit for node 69c484b2590270ad7b6d93f5 2026-03-28 00:58:43,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:43,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c484b2590270ad7b6d93f5 HTTP/1.1" 200 None 2026-03-28 00:58:43,302 - lava_callback - DEBUG - Processing node 69c484b2590270ad7b6d93f5: baseline-arm-clabbe 2026-03-28 00:58:43,302 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c484b2590270ad7b6d93f5 2026-03-28 00:58:43,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:43,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c484b2590270ad7b6d93f5 HTTP/1.1" 200 None 2026-03-28 00:58:43,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:43,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 00:58:43,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:43,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c484b2590270ad7b6d93f5 HTTP/1.1" 200 None 2026-03-28 00:58:43,366 - lava_callback - ERROR - Error processing callback for node 69c484b2590270ad7b6d93f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33546 - "GET /node/69876ee4e31b58f834f1a37b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:39424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42132 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:01:43,447 - lava_callback - INFO - Received callback for node 69c4856a590270ad7b6d9437 INFO: 172.21.0.1:55074 - "POST /node/69c4856a590270ad7b6d9437 HTTP/1.0" 202 Accepted 2026-03-28 01:01:43,452 - lava_callback - INFO - Starting async job submit for node 69c4856a590270ad7b6d9437 2026-03-28 01:01:43,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:01:43,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4856a590270ad7b6d9437 HTTP/1.1" 200 None 2026-03-28 01:01:43,468 - lava_callback - DEBUG - Processing node 69c4856a590270ad7b6d9437: baseline-arm-clabbe 2026-03-28 01:01:43,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4856a590270ad7b6d9437 2026-03-28 01:01:43,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:01:43,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4856a590270ad7b6d9437 HTTP/1.1" 200 None 2026-03-28 01:01:43,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:01:43,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-28 01:01:43,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:01:43,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4856a590270ad7b6d9437 HTTP/1.1" 200 None 2026-03-28 01:01:43,526 - lava_callback - ERROR - Error processing callback for node 69c4856a590270ad7b6d9437: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60496 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:02:23,430 - lava_callback - INFO - Received callback for node 69c48591590270ad7b6d943e INFO: 172.21.0.1:55668 - "POST /node/69c48591590270ad7b6d943e HTTP/1.0" 202 Accepted 2026-03-28 01:02:23,435 - lava_callback - INFO - Starting async job submit for node 69c48591590270ad7b6d943e 2026-03-28 01:02:23,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:02:23,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48591590270ad7b6d943e HTTP/1.1" 200 None 2026-03-28 01:02:23,451 - lava_callback - DEBUG - Processing node 69c48591590270ad7b6d943e: baseline-arm-clabbe 2026-03-28 01:02:23,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48591590270ad7b6d943e 2026-03-28 01:02:23,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:02:23,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48591590270ad7b6d943e HTTP/1.1" 200 None 2026-03-28 01:02:23,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:02:23,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-28 01:02:23,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:02:23,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48591590270ad7b6d943e HTTP/1.1" 200 None 2026-03-28 01:02:23,517 - lava_callback - ERROR - Error processing callback for node 69c48591590270ad7b6d943e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57886 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:03:03,501 - lava_callback - INFO - Received callback for node 69c485b8590270ad7b6d9448 INFO: 172.21.0.1:55200 - "POST /node/69c485b8590270ad7b6d9448 HTTP/1.0" 202 Accepted 2026-03-28 01:03:03,506 - lava_callback - INFO - Starting async job submit for node 69c485b8590270ad7b6d9448 2026-03-28 01:03:03,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:03,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c485b8590270ad7b6d9448 HTTP/1.1" 200 None 2026-03-28 01:03:03,525 - lava_callback - DEBUG - Processing node 69c485b8590270ad7b6d9448: baseline-arm-clabbe 2026-03-28 01:03:03,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c485b8590270ad7b6d9448 2026-03-28 01:03:03,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:03,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c485b8590270ad7b6d9448 HTTP/1.1" 200 None 2026-03-28 01:03:03,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:03,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-28 01:03:03,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:03,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c485b8590270ad7b6d9448 HTTP/1.1" 200 None 2026-03-28 01:03:03,620 - lava_callback - ERROR - Error processing callback for node 69c485b8590270ad7b6d9448: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58248 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:03:43,611 - lava_callback - INFO - Received callback for node 69c485dd590270ad7b6d9460 INFO: 172.21.0.1:38842 - "POST /node/69c485dd590270ad7b6d9460 HTTP/1.0" 202 Accepted 2026-03-28 01:03:43,616 - lava_callback - INFO - Starting async job submit for node 69c485dd590270ad7b6d9460 2026-03-28 01:03:43,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:43,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c485dd590270ad7b6d9460 HTTP/1.1" 200 None 2026-03-28 01:03:43,633 - lava_callback - DEBUG - Processing node 69c485dd590270ad7b6d9460: baseline-arm-clabbe 2026-03-28 01:03:43,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c485dd590270ad7b6d9460 2026-03-28 01:03:43,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:43,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c485dd590270ad7b6d9460 HTTP/1.1" 200 None 2026-03-28 01:03:43,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:43,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-28 01:03:43,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:43,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c485dd590270ad7b6d9460 HTTP/1.1" 200 None 2026-03-28 01:03:43,683 - lava_callback - ERROR - Error processing callback for node 69c485dd590270ad7b6d9460: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37408 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:04:23,610 - lava_callback - INFO - Received callback for node 69c48603590270ad7b6d946e INFO: 172.21.0.1:55432 - "POST /node/69c48603590270ad7b6d946e HTTP/1.0" 202 Accepted 2026-03-28 01:04:23,620 - lava_callback - INFO - Starting async job submit for node 69c48603590270ad7b6d946e 2026-03-28 01:04:23,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:04:23,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48603590270ad7b6d946e HTTP/1.1" 200 None 2026-03-28 01:04:23,655 - lava_callback - DEBUG - Processing node 69c48603590270ad7b6d946e: baseline-arm-clabbe 2026-03-28 01:04:23,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48603590270ad7b6d946e 2026-03-28 01:04:23,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:04:23,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48603590270ad7b6d946e HTTP/1.1" 200 None 2026-03-28 01:04:23,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:04:23,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-28 01:04:23,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:04:23,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48603590270ad7b6d946e HTTP/1.1" 200 None 2026-03-28 01:04:23,737 - lava_callback - ERROR - Error processing callback for node 69c48603590270ad7b6d946e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33048 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:06:54,239 - lava_callback - INFO - Received callback for node 69c7256629015d0f9598f93c INFO: 172.21.0.1:47258 - "POST /node/69c7256629015d0f9598f93c HTTP/1.0" 202 Accepted 2026-03-28 01:06:54,246 - lava_callback - INFO - Starting async job submit for node 69c7256629015d0f9598f93c 2026-03-28 01:06:54,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:54,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256629015d0f9598f93c HTTP/1.1" 200 None 2026-03-28 01:06:54,322 - lava_callback - DEBUG - Processing node 69c7256629015d0f9598f93c: baseline-arm 2026-03-28 01:06:54,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:06:54,989 - lava_callback - INFO - Received callback for node 69c7298f29015d0f95992185 INFO: 172.21.0.1:47268 - "POST /node/69c7298f29015d0f95992185 HTTP/1.0" 202 Accepted 2026-03-28 01:06:54,995 - lava_callback - INFO - Starting async job submit for node 69c7298f29015d0f95992185 2026-03-28 01:06:54,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:55,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7298f29015d0f95992185 HTTP/1.1" 200 None 2026-03-28 01:06:55,032 - lava_callback - DEBUG - Processing node 69c7298f29015d0f95992185: baseline-arm 2026-03-28 01:06:55,038 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:06:55,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:06:55,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:06:55,095 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f93c/log.txt.gz 2026-03-28 01:06:55,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:06:55,234 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7298f29015d0f95992185/lava-logs.yaml 2026-03-28 01:06:55,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:06:55,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:06:55,500 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7298f29015d0f95992185/lava_callback.json.gz 2026-03-28 01:06:55,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:55,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:06:55,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7298f29015d0f95992185 HTTP/1.1" 200 None 2026-03-28 01:06:55,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:55,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:06:55,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:55,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7298f29015d0f95992185 HTTP/1.1" 200 None 2026-03-28 01:06:55,569 - lava_callback - INFO - Completed processing callback for node 69c7298f29015d0f95992185 2026-03-28 01:06:55,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:06:55,826 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f93c/lava-logs.yaml 2026-03-28 01:06:55,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:06:56,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:06:56,006 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f93c/lava_callback.json.gz DEBUG: Setup failed for 69c7256629015d0f9598f93c. Transitting job node result: incomplete -> fail 2026-03-28 01:06:56,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:56,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256629015d0f9598f93c HTTP/1.1" 200 None 2026-03-28 01:06:56,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:56,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:06:56,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:56,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256629015d0f9598f93c HTTP/1.1" 200 None 2026-03-28 01:06:56,302 - lava_callback - INFO - Completed processing callback for node 69c7256629015d0f9598f93c INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:07:05,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:07:05,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:07:07,202 - lava_callback - INFO - Received callback for node 69c486b7590270ad7b6d9495 INFO: 172.21.0.1:45918 - "POST /node/69c486b7590270ad7b6d9495 HTTP/1.0" 202 Accepted 2026-03-28 01:07:07,208 - lava_callback - INFO - Starting async job submit for node 69c486b7590270ad7b6d9495 2026-03-28 01:07:07,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:07:07,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c486b7590270ad7b6d9495 HTTP/1.1" 200 None 2026-03-28 01:07:07,225 - lava_callback - DEBUG - Processing node 69c486b7590270ad7b6d9495: baseline-arm64-baylibre 2026-03-28 01:07:07,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c486b7590270ad7b6d9495 2026-03-28 01:07:07,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:07:07,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c486b7590270ad7b6d9495 HTTP/1.1" 200 None 2026-03-28 01:07:07,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:07:07,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:07:07,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:07:07,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c486b7590270ad7b6d9495 HTTP/1.1" 200 None 2026-03-28 01:07:07,276 - lava_callback - ERROR - Error processing callback for node 69c486b7590270ad7b6d9495: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49284 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:08:47,153 - lava_callback - INFO - Received callback for node 69c4870a590270ad7b6d94b7 INFO: 172.21.0.1:49296 - "POST /node/69c4870a590270ad7b6d94b7 HTTP/1.0" 202 Accepted 2026-03-28 01:08:47,159 - lava_callback - INFO - Starting async job submit for node 69c4870a590270ad7b6d94b7 2026-03-28 01:08:47,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:08:47,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4870a590270ad7b6d94b7 HTTP/1.1" 200 None 2026-03-28 01:08:47,330 - lava_callback - DEBUG - Processing node 69c4870a590270ad7b6d94b7: baseline-arm64-baylibre 2026-03-28 01:08:47,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4870a590270ad7b6d94b7 2026-03-28 01:08:47,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:08:47,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4870a590270ad7b6d94b7 HTTP/1.1" 200 None 2026-03-28 01:08:47,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:08:47,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:08:47,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:08:47,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4870a590270ad7b6d94b7 HTTP/1.1" 200 None 2026-03-28 01:08:47,392 - lava_callback - ERROR - Error processing callback for node 69c4870a590270ad7b6d94b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35620 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:09:38,556 - lava_callback - INFO - Received callback for node 69c7257529015d0f9598f971 INFO: 172.21.0.1:35634 - "POST /node/69c7257529015d0f9598f971 HTTP/1.0" 202 Accepted 2026-03-28 01:09:38,565 - lava_callback - INFO - Starting async job submit for node 69c7257529015d0f9598f971 2026-03-28 01:09:38,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:09:38,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257529015d0f9598f971 HTTP/1.1" 200 None 2026-03-28 01:09:38,611 - lava_callback - DEBUG - Processing node 69c7257529015d0f9598f971: baseline-arm-broonie 2026-03-28 01:09:38,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:09:39,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:09:39,102 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257529015d0f9598f971/log.txt.gz 2026-03-28 01:09:39,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:09:39,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:09:39,554 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257529015d0f9598f971/lava-logs.yaml 2026-03-28 01:09:39,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:09:39,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:09:39,730 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257529015d0f9598f971/lava_callback.json.gz 2026-03-28 01:09:39,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:09:39,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257529015d0f9598f971 HTTP/1.1" 200 None 2026-03-28 01:09:39,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:09:39,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:09:39,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:09:39,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257529015d0f9598f971 HTTP/1.1" 200 None 2026-03-28 01:09:39,844 - lava_callback - INFO - Completed processing callback for node 69c7257529015d0f9598f971 INFO: 172.21.0.1:33918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53222 - "GET /node/698b677b0a16aea5a0dc75a5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:53232 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:10:05,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:10:05,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33964 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:10:07,264 - lava_callback - INFO - Received callback for node 69c4875a590270ad7b6d94c4 INFO: 172.21.0.1:33974 - "POST /node/69c4875a590270ad7b6d94c4 HTTP/1.0" 202 Accepted 2026-03-28 01:10:07,272 - lava_callback - INFO - Starting async job submit for node 69c4875a590270ad7b6d94c4 2026-03-28 01:10:07,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:10:07,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4875a590270ad7b6d94c4 HTTP/1.1" 200 None 2026-03-28 01:10:07,290 - lava_callback - DEBUG - Processing node 69c4875a590270ad7b6d94c4: baseline-arm64-baylibre 2026-03-28 01:10:07,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4875a590270ad7b6d94c4 2026-03-28 01:10:07,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:10:07,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4875a590270ad7b6d94c4 HTTP/1.1" 200 None 2026-03-28 01:10:07,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:10:07,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:10:07,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:10:07,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4875a590270ad7b6d94c4 HTTP/1.1" 200 None 2026-03-28 01:10:07,361 - lava_callback - ERROR - Error processing callback for node 69c4875a590270ad7b6d94c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40722 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:11:27,298 - lava_callback - INFO - Received callback for node 69c487b1590270ad7b6d94cf INFO: 172.21.0.1:40734 - "POST /node/69c487b1590270ad7b6d94cf HTTP/1.0" 202 Accepted 2026-03-28 01:11:27,307 - lava_callback - INFO - Starting async job submit for node 69c487b1590270ad7b6d94cf 2026-03-28 01:11:27,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:11:27,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c487b1590270ad7b6d94cf HTTP/1.1" 200 None 2026-03-28 01:11:27,337 - lava_callback - DEBUG - Processing node 69c487b1590270ad7b6d94cf: baseline-arm64-baylibre 2026-03-28 01:11:27,337 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c487b1590270ad7b6d94cf 2026-03-28 01:11:27,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:11:27,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c487b1590270ad7b6d94cf HTTP/1.1" 200 None 2026-03-28 01:11:27,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:11:27,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:11:27,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:11:27,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c487b1590270ad7b6d94cf HTTP/1.1" 200 None 2026-03-28 01:11:27,453 - lava_callback - ERROR - Error processing callback for node 69c487b1590270ad7b6d94cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45240 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:12:47,390 - lava_callback - INFO - Received callback for node 69c48804590270ad7b6d94db INFO: 172.21.0.1:45256 - "POST /node/69c48804590270ad7b6d94db HTTP/1.0" 202 Accepted 2026-03-28 01:12:47,396 - lava_callback - INFO - Starting async job submit for node 69c48804590270ad7b6d94db 2026-03-28 01:12:47,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:12:47,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48804590270ad7b6d94db HTTP/1.1" 200 None 2026-03-28 01:12:47,415 - lava_callback - DEBUG - Processing node 69c48804590270ad7b6d94db: baseline-arm64-baylibre 2026-03-28 01:12:47,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48804590270ad7b6d94db 2026-03-28 01:12:47,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:12:47,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48804590270ad7b6d94db HTTP/1.1" 200 None 2026-03-28 01:12:47,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:12:47,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:12:47,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:12:47,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48804590270ad7b6d94db HTTP/1.1" 200 None 2026-03-28 01:12:47,478 - lava_callback - ERROR - Error processing callback for node 69c48804590270ad7b6d94db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49546 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:14:07,479 - lava_callback - INFO - Received callback for node 69c48855590270ad7b6d9523 INFO: 172.21.0.1:49560 - "POST /node/69c48855590270ad7b6d9523 HTTP/1.0" 202 Accepted 2026-03-28 01:14:07,488 - lava_callback - INFO - Starting async job submit for node 69c48855590270ad7b6d9523 2026-03-28 01:14:07,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:07,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48855590270ad7b6d9523 HTTP/1.1" 200 None 2026-03-28 01:14:07,514 - lava_callback - DEBUG - Processing node 69c48855590270ad7b6d9523: baseline-arm64-baylibre 2026-03-28 01:14:07,514 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48855590270ad7b6d9523 2026-03-28 01:14:07,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:07,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48855590270ad7b6d9523 HTTP/1.1" 200 None 2026-03-28 01:14:07,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:07,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:14:07,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:07,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48855590270ad7b6d9523 HTTP/1.1" 200 None 2026-03-28 01:14:07,581 - lava_callback - ERROR - Error processing callback for node 69c48855590270ad7b6d9523: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:14:27,426 - lava_callback - INFO - Received callback for node 69c4886c590270ad7b6d9575 INFO: 172.21.0.1:35024 - "POST /node/69c4886c590270ad7b6d9575 HTTP/1.0" 202 Accepted 2026-03-28 01:14:27,431 - lava_callback - INFO - Starting async job submit for node 69c4886c590270ad7b6d9575 2026-03-28 01:14:27,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:27,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4886c590270ad7b6d9575 HTTP/1.1" 200 None 2026-03-28 01:14:27,462 - lava_callback - DEBUG - Processing node 69c4886c590270ad7b6d9575: baseline-arm64-baylibre 2026-03-28 01:14:27,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4886c590270ad7b6d9575 2026-03-28 01:14:27,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:27,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4886c590270ad7b6d9575 HTTP/1.1" 200 None 2026-03-28 01:14:27,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:27,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:14:27,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:27,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4886c590270ad7b6d9575 HTTP/1.1" 200 None 2026-03-28 01:14:27,516 - lava_callback - ERROR - Error processing callback for node 69c4886c590270ad7b6d9575: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39176 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:14:43,972 - lava_callback - INFO - Received callback for node 69c4887e590270ad7b6d95bd INFO: 172.21.0.1:52784 - "POST /node/69c4887e590270ad7b6d95bd HTTP/1.0" 202 Accepted 2026-03-28 01:14:43,978 - lava_callback - INFO - Starting async job submit for node 69c4887e590270ad7b6d95bd 2026-03-28 01:14:43,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4887e590270ad7b6d95bd HTTP/1.1" 200 None 2026-03-28 01:14:44,005 - lava_callback - DEBUG - Processing node 69c4887e590270ad7b6d95bd: baseline-arm64-clabbe 2026-03-28 01:14:44,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4887e590270ad7b6d95bd 2026-03-28 01:14:44,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4887e590270ad7b6d95bd HTTP/1.1" 200 None 2026-03-28 01:14:44,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:14:44,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4887e590270ad7b6d95bd HTTP/1.1" 200 None 2026-03-28 01:14:44,057 - lava_callback - ERROR - Error processing callback for node 69c4887e590270ad7b6d95bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:14:44,658 - lava_callback - INFO - Received callback for node 69c48880590270ad7b6d95c0 INFO: 172.21.0.1:52792 - "POST /node/69c48880590270ad7b6d95c0 HTTP/1.0" 202 Accepted 2026-03-28 01:14:44,665 - lava_callback - INFO - Starting async job submit for node 69c48880590270ad7b6d95c0 2026-03-28 01:14:44,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48880590270ad7b6d95c0 HTTP/1.1" 200 None 2026-03-28 01:14:44,686 - lava_callback - DEBUG - Processing node 69c48880590270ad7b6d95c0: baseline-arm64-clabbe 2026-03-28 01:14:44,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48880590270ad7b6d95c0 2026-03-28 01:14:44,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48880590270ad7b6d95c0 HTTP/1.1" 200 None 2026-03-28 01:14:44,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:14:44,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48880590270ad7b6d95c0 HTTP/1.1" 200 None 2026-03-28 01:14:44,744 - lava_callback - ERROR - Error processing callback for node 69c48880590270ad7b6d95c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:14:45,334 - lava_callback - INFO - Received callback for node 69c48881590270ad7b6d95c3 INFO: 172.21.0.1:52804 - "POST /node/69c48881590270ad7b6d95c3 HTTP/1.0" 202 Accepted 2026-03-28 01:14:45,339 - lava_callback - INFO - Starting async job submit for node 69c48881590270ad7b6d95c3 2026-03-28 01:14:45,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:45,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48881590270ad7b6d95c3 HTTP/1.1" 200 None 2026-03-28 01:14:45,355 - lava_callback - DEBUG - Processing node 69c48881590270ad7b6d95c3: baseline-arm64-clabbe 2026-03-28 01:14:45,356 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48881590270ad7b6d95c3 2026-03-28 01:14:45,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:45,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48881590270ad7b6d95c3 HTTP/1.1" 200 None 2026-03-28 01:14:45,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:45,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:14:45,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:45,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48881590270ad7b6d95c3 HTTP/1.1" 200 None 2026-03-28 01:14:45,419 - lava_callback - ERROR - Error processing callback for node 69c48881590270ad7b6d95c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:14:46,029 - lava_callback - INFO - Received callback for node 69c48882590270ad7b6d95c6 INFO: 172.21.0.1:52810 - "POST /node/69c48882590270ad7b6d95c6 HTTP/1.0" 202 Accepted 2026-03-28 01:14:46,034 - lava_callback - INFO - Starting async job submit for node 69c48882590270ad7b6d95c6 2026-03-28 01:14:46,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:46,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48882590270ad7b6d95c6 HTTP/1.1" 200 None 2026-03-28 01:14:46,057 - lava_callback - DEBUG - Processing node 69c48882590270ad7b6d95c6: baseline-arm64-clabbe 2026-03-28 01:14:46,057 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48882590270ad7b6d95c6 2026-03-28 01:14:46,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:46,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48882590270ad7b6d95c6 HTTP/1.1" 200 None 2026-03-28 01:14:46,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:46,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:14:46,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:46,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48882590270ad7b6d95c6 HTTP/1.1" 200 None 2026-03-28 01:14:46,114 - lava_callback - ERROR - Error processing callback for node 69c48882590270ad7b6d95c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52826 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:14:50,553 - lava_callback - INFO - Received callback for node 69c7259d29015d0f9598f9ef INFO: 172.21.0.1:59312 - "POST /node/69c7259d29015d0f9598f9ef HTTP/1.0" 202 Accepted 2026-03-28 01:14:50,560 - lava_callback - INFO - Starting async job submit for node 69c7259d29015d0f9598f9ef 2026-03-28 01:14:50,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:50,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7259d29015d0f9598f9ef HTTP/1.1" 200 None 2026-03-28 01:14:50,600 - lava_callback - DEBUG - Processing node 69c7259d29015d0f9598f9ef: ltp-cap_bounds 2026-03-28 01:14:50,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:14:51,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:14:51,115 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c7259d29015d0f9598f9ef/log.txt.gz 2026-03-28 01:14:51,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:14:51,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:14:51,565 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c7259d29015d0f9598f9ef/lava-logs.yaml 2026-03-28 01:14:51,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:14:51,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:14:51,733 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c7259d29015d0f9598f9ef/lava_callback.json.gz 2026-03-28 01:14:51,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:51,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7259d29015d0f9598f9ef HTTP/1.1" 200 None 2026-03-28 01:14:51,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:51,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:14:51,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:51,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7259d29015d0f9598f9ef HTTP/1.1" 200 None 2026-03-28 01:14:51,822 - lava_callback - INFO - Completed processing callback for node 69c7259d29015d0f9598f9ef INFO: 172.21.0.1:59328 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:15:03,958 - lava_callback - INFO - Received callback for node 69c48884590270ad7b6d95cc INFO: 172.21.0.1:54014 - "POST /node/69c48884590270ad7b6d95cc HTTP/1.0" 202 Accepted 2026-03-28 01:15:03,964 - lava_callback - INFO - Starting async job submit for node 69c48884590270ad7b6d95cc 2026-03-28 01:15:03,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:03,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48884590270ad7b6d95cc HTTP/1.1" 200 None 2026-03-28 01:15:03,980 - lava_callback - DEBUG - Processing node 69c48884590270ad7b6d95cc: baseline-arm64-clabbe 2026-03-28 01:15:03,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48884590270ad7b6d95cc 2026-03-28 01:15:03,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:03,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48884590270ad7b6d95cc HTTP/1.1" 200 None 2026-03-28 01:15:03,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:15:04,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48884590270ad7b6d95cc HTTP/1.1" 200 None 2026-03-28 01:15:04,043 - lava_callback - ERROR - Error processing callback for node 69c48884590270ad7b6d95cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:15:04,666 - lava_callback - INFO - Received callback for node 69c48885590270ad7b6d95cf INFO: 172.21.0.1:54026 - "POST /node/69c48885590270ad7b6d95cf HTTP/1.0" 202 Accepted 2026-03-28 01:15:04,672 - lava_callback - INFO - Starting async job submit for node 69c48885590270ad7b6d95cf 2026-03-28 01:15:04,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48885590270ad7b6d95cf HTTP/1.1" 200 None 2026-03-28 01:15:04,688 - lava_callback - DEBUG - Processing node 69c48885590270ad7b6d95cf: baseline-arm64-clabbe 2026-03-28 01:15:04,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48885590270ad7b6d95cf 2026-03-28 01:15:04,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48885590270ad7b6d95cf HTTP/1.1" 200 None 2026-03-28 01:15:04,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:15:04,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48885590270ad7b6d95cf HTTP/1.1" 200 None 2026-03-28 01:15:04,751 - lava_callback - ERROR - Error processing callback for node 69c48885590270ad7b6d95cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:15:05,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:05,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52584 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:15:39,495 - lava_callback - INFO - Received callback for node 69c7259f29015d0f9598f9f8 INFO: 172.21.0.1:52592 - "POST /node/69c7259f29015d0f9598f9f8 HTTP/1.0" 202 Accepted 2026-03-28 01:15:39,502 - lava_callback - INFO - Starting async job submit for node 69c7259f29015d0f9598f9f8 2026-03-28 01:15:39,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:39,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7259f29015d0f9598f9f8 HTTP/1.1" 200 None 2026-03-28 01:15:39,542 - lava_callback - DEBUG - Processing node 69c7259f29015d0f9598f9f8: ltp-containers 2026-03-28 01:15:39,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:15:40,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:15:40,113 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c7259f29015d0f9598f9f8/log.txt.gz 2026-03-28 01:15:40,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:15:40,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:15:40,533 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c7259f29015d0f9598f9f8/lava-logs.yaml 2026-03-28 01:15:40,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:15:40,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:15:40,714 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c7259f29015d0f9598f9f8/lava_callback.json.gz 2026-03-28 01:15:40,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:40,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7259f29015d0f9598f9f8 HTTP/1.1" 200 None 2026-03-28 01:15:40,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:40,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:15:40,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:40,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7259f29015d0f9598f9f8 HTTP/1.1" 200 None 2026-03-28 01:15:40,809 - lava_callback - INFO - Completed processing callback for node 69c7259f29015d0f9598f9f8 2026-03-28 01:15:43,988 - lava_callback - INFO - Received callback for node 69c488ad590270ad7b6d9628 INFO: 172.21.0.1:45962 - "POST /node/69c488ad590270ad7b6d9628 HTTP/1.0" 202 Accepted 2026-03-28 01:15:43,993 - lava_callback - INFO - Starting async job submit for node 69c488ad590270ad7b6d9628 2026-03-28 01:15:43,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:44,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c488ad590270ad7b6d9628 HTTP/1.1" 200 None 2026-03-28 01:15:44,009 - lava_callback - DEBUG - Processing node 69c488ad590270ad7b6d9628: baseline-arm64-clabbe 2026-03-28 01:15:44,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c488ad590270ad7b6d9628 2026-03-28 01:15:44,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:44,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c488ad590270ad7b6d9628 HTTP/1.1" 200 None 2026-03-28 01:15:44,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:44,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:15:44,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:44,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c488ad590270ad7b6d9628 HTTP/1.1" 200 None 2026-03-28 01:15:44,068 - lava_callback - ERROR - Error processing callback for node 69c488ad590270ad7b6d9628: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55604 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:15:59,125 - lava_callback - INFO - Received callback for node 69c725a029015d0f9598f9fb INFO: 172.21.0.1:55612 - "POST /node/69c725a029015d0f9598f9fb HTTP/1.0" 202 Accepted 2026-03-28 01:15:59,132 - lava_callback - INFO - Starting async job submit for node 69c725a029015d0f9598f9fb 2026-03-28 01:15:59,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:59,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a029015d0f9598f9fb HTTP/1.1" 200 None 2026-03-28 01:15:59,172 - lava_callback - DEBUG - Processing node 69c725a029015d0f9598f9fb: ltp-controllers 2026-03-28 01:15:59,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:15:59,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:15:59,776 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c725a029015d0f9598f9fb/log.txt.gz 2026-03-28 01:16:00,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:16:00,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:16:00,267 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c725a029015d0f9598f9fb/lava-logs.yaml 2026-03-28 01:16:00,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:16:00,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:16:00,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c725a029015d0f9598f9fb/lava_callback.json.gz 2026-03-28 01:16:00,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:16:00,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a029015d0f9598f9fb HTTP/1.1" 200 None 2026-03-28 01:16:00,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:16:00,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:16:00,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:16:00,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725a029015d0f9598f9fb HTTP/1.1" 200 None 2026-03-28 01:16:00,560 - lava_callback - INFO - Completed processing callback for node 69c725a029015d0f9598f9fb 2026-03-28 01:16:05,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:16:05,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45662 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:17:04,094 - lava_callback - INFO - Received callback for node 69c488ff590270ad7b6d96cd INFO: 172.21.0.1:40072 - "POST /node/69c488ff590270ad7b6d96cd HTTP/1.0" 202 Accepted 2026-03-28 01:17:04,099 - lava_callback - INFO - Starting async job submit for node 69c488ff590270ad7b6d96cd 2026-03-28 01:17:04,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:04,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c488ff590270ad7b6d96cd HTTP/1.1" 200 None 2026-03-28 01:17:04,118 - lava_callback - DEBUG - Processing node 69c488ff590270ad7b6d96cd: baseline-arm64-clabbe 2026-03-28 01:17:04,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c488ff590270ad7b6d96cd 2026-03-28 01:17:04,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:04,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c488ff590270ad7b6d96cd HTTP/1.1" 200 None 2026-03-28 01:17:04,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:04,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:17:04,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:04,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c488ff590270ad7b6d96cd HTTP/1.1" 200 None 2026-03-28 01:17:04,183 - lava_callback - ERROR - Error processing callback for node 69c488ff590270ad7b6d96cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35518 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:17:42,568 - lava_callback - INFO - Received callback for node 69c72bde29015d0f95992257 INFO: 172.21.0.1:50438 - "POST /node/69c72bde29015d0f95992257 HTTP/1.0" 202 Accepted 2026-03-28 01:17:42,574 - lava_callback - INFO - Starting async job submit for node 69c72bde29015d0f95992257 2026-03-28 01:17:42,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:42,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72bde29015d0f95992257 HTTP/1.1" 200 None 2026-03-28 01:17:42,608 - lava_callback - DEBUG - Processing node 69c72bde29015d0f95992257: baseline-x86 2026-03-28 01:17:42,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:17:43,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:17:43,024 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c72bde29015d0f95992257/log.txt.gz 2026-03-28 01:17:43,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:17:43,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:17:43,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c72bde29015d0f95992257/lava-logs.yaml 2026-03-28 01:17:43,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:17:43,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:17:43,547 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c72bde29015d0f95992257/lava_callback.json.gz 2026-03-28 01:17:43,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:43,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72bde29015d0f95992257 HTTP/1.1" 200 None 2026-03-28 01:17:43,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:43,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 01:17:43,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:43,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72bde29015d0f95992257 HTTP/1.1" 200 None 2026-03-28 01:17:43,639 - lava_callback - INFO - Completed processing callback for node 69c72bde29015d0f95992257 INFO: 172.21.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59962 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:18:05,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:18:05,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43684 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:18:24,159 - lava_callback - INFO - Received callback for node 69c48951590270ad7b6d96f1 INFO: 172.21.0.1:59398 - "POST /node/69c48951590270ad7b6d96f1 HTTP/1.0" 202 Accepted 2026-03-28 01:18:24,171 - lava_callback - INFO - Starting async job submit for node 69c48951590270ad7b6d96f1 2026-03-28 01:18:24,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:18:24,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48951590270ad7b6d96f1 HTTP/1.1" 200 None 2026-03-28 01:18:24,198 - lava_callback - DEBUG - Processing node 69c48951590270ad7b6d96f1: baseline-arm64-clabbe 2026-03-28 01:18:24,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48951590270ad7b6d96f1 2026-03-28 01:18:24,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:18:24,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48951590270ad7b6d96f1 HTTP/1.1" 200 None 2026-03-28 01:18:24,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:18:24,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:18:24,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:18:24,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48951590270ad7b6d96f1 HTTP/1.1" 200 None 2026-03-28 01:18:24,324 - lava_callback - ERROR - Error processing callback for node 69c48951590270ad7b6d96f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60182 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:19:44,282 - lava_callback - INFO - Received callback for node 69c489a1590270ad7b6d9711 INFO: 172.21.0.1:44874 - "POST /node/69c489a1590270ad7b6d9711 HTTP/1.0" 202 Accepted 2026-03-28 01:19:44,291 - lava_callback - INFO - Starting async job submit for node 69c489a1590270ad7b6d9711 2026-03-28 01:19:44,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:44,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c489a1590270ad7b6d9711 HTTP/1.1" 200 None 2026-03-28 01:19:44,319 - lava_callback - DEBUG - Processing node 69c489a1590270ad7b6d9711: baseline-arm64-clabbe 2026-03-28 01:19:44,319 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c489a1590270ad7b6d9711 2026-03-28 01:19:44,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:44,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c489a1590270ad7b6d9711 HTTP/1.1" 200 None 2026-03-28 01:19:44,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:44,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:19:44,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:44,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c489a1590270ad7b6d9711 HTTP/1.1" 200 None 2026-03-28 01:19:44,404 - lava_callback - ERROR - Error processing callback for node 69c489a1590270ad7b6d9711: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45722 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:19:58,832 - lava_callback - INFO - Received callback for node 69c72ca929015d0f959922b8 INFO: 172.21.0.1:45736 - "POST /node/69c72ca929015d0f959922b8 HTTP/1.0" 202 Accepted 2026-03-28 01:19:58,842 - lava_callback - INFO - Starting async job submit for node 69c72ca929015d0f959922b8 2026-03-28 01:19:58,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:58,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ca929015d0f959922b8 HTTP/1.1" 200 None 2026-03-28 01:19:58,887 - lava_callback - DEBUG - Processing node 69c72ca929015d0f959922b8: baseline-arm 2026-03-28 01:19:58,895 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:19:58,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:19:59,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:19:59,121 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c72ca929015d0f959922b8/lava-logs.yaml 2026-03-28 01:19:59,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:19:59,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:19:59,237 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c72ca929015d0f959922b8/lava_callback.json.gz 2026-03-28 01:19:59,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:59,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ca929015d0f959922b8 HTTP/1.1" 200 None 2026-03-28 01:19:59,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:59,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:19:59,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:59,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ca929015d0f959922b8 HTTP/1.1" 200 None 2026-03-28 01:19:59,339 - lava_callback - INFO - Completed processing callback for node 69c72ca929015d0f959922b8 2026-03-28 01:20:05,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:05,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54778 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:20:57,592 - lava_callback - INFO - Received callback for node 69c72ccf29015d0f959922d9 INFO: 172.21.0.1:54794 - "POST /node/69c72ccf29015d0f959922d9 HTTP/1.0" 202 Accepted 2026-03-28 01:20:57,599 - lava_callback - INFO - Starting async job submit for node 69c72ccf29015d0f959922d9 2026-03-28 01:20:57,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:57,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ccf29015d0f959922d9 HTTP/1.1" 200 None 2026-03-28 01:20:57,677 - lava_callback - DEBUG - Processing node 69c72ccf29015d0f959922d9: baseline-arm64 2026-03-28 01:20:57,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:20:58,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:20:58,011 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922d9/log.txt.gz 2026-03-28 01:20:58,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:20:58,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:20:58,288 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922d9/lava-logs.yaml 2026-03-28 01:20:58,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:20:58,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:20:58,482 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922d9/lava_callback.json.gz 2026-03-28 01:20:58,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:58,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ccf29015d0f959922d9 HTTP/1.1" 200 None 2026-03-28 01:20:58,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:58,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:20:58,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:58,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ccf29015d0f959922d9 HTTP/1.1" 200 None 2026-03-28 01:20:58,624 - lava_callback - INFO - Completed processing callback for node 69c72ccf29015d0f959922d9 2026-03-28 01:20:59,449 - lava_callback - INFO - Received callback for node 69c72cee29015d0f9599234a INFO: 172.21.0.1:54802 - "POST /node/69c72cee29015d0f9599234a HTTP/1.0" 202 Accepted 2026-03-28 01:20:59,455 - lava_callback - INFO - Starting async job submit for node 69c72cee29015d0f9599234a 2026-03-28 01:20:59,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:59,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cee29015d0f9599234a HTTP/1.1" 200 None 2026-03-28 01:20:59,476 - lava_callback - DEBUG - Processing node 69c72cee29015d0f9599234a: baseline-nfs-arm64 2026-03-28 01:20:59,478 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:20:59,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:20:59,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:20:59,604 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cee29015d0f9599234a/lava-logs.yaml 2026-03-28 01:20:59,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:20:59,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:20:59,731 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cee29015d0f9599234a/lava_callback.json.gz 2026-03-28 01:20:59,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:59,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cee29015d0f9599234a HTTP/1.1" 200 None 2026-03-28 01:20:59,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:59,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:20:59,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:59,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cee29015d0f9599234a HTTP/1.1" 200 None 2026-03-28 01:20:59,790 - lava_callback - INFO - Completed processing callback for node 69c72cee29015d0f9599234a 2026-03-28 01:21:04,265 - lava_callback - INFO - Received callback for node 69c489f1590270ad7b6d9727 INFO: 172.21.0.1:59454 - "POST /node/69c489f1590270ad7b6d9727 HTTP/1.0" 202 Accepted 2026-03-28 01:21:04,274 - lava_callback - INFO - Starting async job submit for node 69c489f1590270ad7b6d9727 2026-03-28 01:21:04,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:04,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c489f1590270ad7b6d9727 HTTP/1.1" 200 None 2026-03-28 01:21:04,298 - lava_callback - DEBUG - Processing node 69c489f1590270ad7b6d9727: baseline-arm64-clabbe 2026-03-28 01:21:04,298 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c489f1590270ad7b6d9727 2026-03-28 01:21:04,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:04,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c489f1590270ad7b6d9727 HTTP/1.1" 200 None 2026-03-28 01:21:04,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:04,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:21:04,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:04,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c489f1590270ad7b6d9727 HTTP/1.1" 200 None 2026-03-28 01:21:04,355 - lava_callback - ERROR - Error processing callback for node 69c489f1590270ad7b6d9727: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:21:05,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:05,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59468 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:21:06,633 - lava_callback - INFO - Received callback for node 69c72cdc29015d0f959922fd INFO: 172.21.0.1:59476 - "POST /node/69c72cdc29015d0f959922fd HTTP/1.0" 202 Accepted 2026-03-28 01:21:06,640 - lava_callback - INFO - Starting async job submit for node 69c72cdc29015d0f959922fd 2026-03-28 01:21:06,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:06,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdc29015d0f959922fd HTTP/1.1" 200 None 2026-03-28 01:21:06,674 - lava_callback - DEBUG - Processing node 69c72cdc29015d0f959922fd: baseline-arm64 2026-03-28 01:21:06,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:06,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:06,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdc29015d0f959922fd/log.txt.gz 2026-03-28 01:21:07,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:07,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:07,253 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdc29015d0f959922fd/lava-logs.yaml 2026-03-28 01:21:07,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:07,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:07,441 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdc29015d0f959922fd/lava_callback.json.gz 2026-03-28 01:21:07,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:07,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdc29015d0f959922fd HTTP/1.1" 200 None 2026-03-28 01:21:07,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:07,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:07,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:07,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cdc29015d0f959922fd HTTP/1.1" 200 None 2026-03-28 01:21:07,539 - lava_callback - INFO - Completed processing callback for node 69c72cdc29015d0f959922fd INFO: 172.21.0.1:46368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46382 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:21:23,895 - lava_callback - INFO - Received callback for node 69c72cd329015d0f959922e8 INFO: 172.21.0.1:46250 - "POST /node/69c72cd329015d0f959922e8 HTTP/1.0" 202 Accepted 2026-03-28 01:21:23,907 - lava_callback - INFO - Starting async job submit for node 69c72cd329015d0f959922e8 2026-03-28 01:21:23,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:24,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd329015d0f959922e8 HTTP/1.1" 200 None 2026-03-28 01:21:24,012 - lava_callback - DEBUG - Processing node 69c72cd329015d0f959922e8: baseline-arm64 2026-03-28 01:21:24,155 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:21:24,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46264 - "GET /node/69936dcdcebe99dd8e0680d4%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-28 01:21:24,394 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:24,395 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd329015d0f959922e8/lava-logs.yaml 2026-03-28 01:21:24,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:24,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:24,507 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd329015d0f959922e8/lava_callback.json.gz 2026-03-28 01:21:24,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:24,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd329015d0f959922e8 HTTP/1.1" 200 None 2026-03-28 01:21:24,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:24,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:24,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:24,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd329015d0f959922e8 HTTP/1.1" 200 None 2026-03-28 01:21:24,659 - lava_callback - INFO - Completed processing callback for node 69c72cd329015d0f959922e8 INFO: 172.21.0.1:46278 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:21:33,277 - lava_callback - INFO - Received callback for node 69c72ce229015d0f95992309 INFO: 172.21.0.1:48852 - "POST /node/69c72ce229015d0f95992309 HTTP/1.0" 202 Accepted 2026-03-28 01:21:33,287 - lava_callback - INFO - Starting async job submit for node 69c72ce229015d0f95992309 2026-03-28 01:21:33,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:33,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce229015d0f95992309 HTTP/1.1" 200 None 2026-03-28 01:21:33,332 - lava_callback - DEBUG - Processing node 69c72ce229015d0f95992309: baseline-arm64 2026-03-28 01:21:33,579 - lava_callback - INFO - Received callback for node 69c72cd429015d0f959922eb INFO: 172.21.0.1:48864 - "POST /node/69c72cd429015d0f959922eb HTTP/1.0" 202 Accepted 2026-03-28 01:21:33,611 - lava_callback - INFO - Starting async job submit for node 69c72cd429015d0f959922eb 2026-03-28 01:21:33,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:33,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:33,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd429015d0f959922eb HTTP/1.1" 200 None 2026-03-28 01:21:33,743 - lava_callback - DEBUG - Processing node 69c72cd429015d0f959922eb: baseline-arm64 2026-03-28 01:21:33,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:33,895 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce229015d0f95992309/log.txt.gz 2026-03-28 01:21:33,920 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:21:34,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:34,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:34,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:34,396 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce229015d0f95992309/lava-logs.yaml 2026-03-28 01:21:34,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:34,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:34,410 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd429015d0f959922eb/lava-logs.yaml 2026-03-28 01:21:34,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:34,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:34,611 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce229015d0f95992309/lava_callback.json.gz 2026-03-28 01:21:34,637 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:34,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,639 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd429015d0f959922eb/lava_callback.json.gz 2026-03-28 01:21:34,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce229015d0f95992309 HTTP/1.1" 200 None 2026-03-28 01:21:34,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd429015d0f959922eb HTTP/1.1" 200 None 2026-03-28 01:21:34,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:34,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:34,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd429015d0f959922eb HTTP/1.1" 200 None 2026-03-28 01:21:34,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce229015d0f95992309 HTTP/1.1" 200 None 2026-03-28 01:21:34,781 - lava_callback - INFO - Completed processing callback for node 69c72ce229015d0f95992309 2026-03-28 01:21:34,831 - lava_callback - INFO - Completed processing callback for node 69c72cd429015d0f959922eb 2026-03-28 01:21:35,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:35,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48876 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:21:40,622 - lava_callback - INFO - Received callback for node 69c72cd729015d0f959922f1 INFO: 172.21.0.1:38752 - "POST /node/69c72cd729015d0f959922f1 HTTP/1.0" 202 Accepted 2026-03-28 01:21:40,628 - lava_callback - INFO - Starting async job submit for node 69c72cd729015d0f959922f1 2026-03-28 01:21:40,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:40,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd729015d0f959922f1 HTTP/1.1" 200 None 2026-03-28 01:21:40,666 - lava_callback - DEBUG - Processing node 69c72cd729015d0f959922f1: baseline-arm64 2026-03-28 01:21:40,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:41,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:41,040 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd729015d0f959922f1/log.txt.gz 2026-03-28 01:21:41,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:41,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:41,400 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd729015d0f959922f1/lava-logs.yaml 2026-03-28 01:21:41,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:41,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:41,551 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd729015d0f959922f1/lava_callback.json.gz 2026-03-28 01:21:41,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:41,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd729015d0f959922f1 HTTP/1.1" 200 None 2026-03-28 01:21:41,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:41,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:41,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:41,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd729015d0f959922f1 HTTP/1.1" 200 None 2026-03-28 01:21:41,639 - lava_callback - INFO - Completed processing callback for node 69c72cd729015d0f959922f1 2026-03-28 01:21:41,666 - lava_callback - INFO - Received callback for node 69c72cd029015d0f959922e1 INFO: 172.21.0.1:38754 - "POST /node/69c72cd029015d0f959922e1 HTTP/1.0" 202 Accepted 2026-03-28 01:21:41,681 - lava_callback - INFO - Starting async job submit for node 69c72cd029015d0f959922e1 2026-03-28 01:21:41,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:41,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd029015d0f959922e1 HTTP/1.1" 200 None 2026-03-28 01:21:41,748 - lava_callback - DEBUG - Processing node 69c72cd029015d0f959922e1: baseline-arm64 2026-03-28 01:21:42,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:42,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:42,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd029015d0f959922e1/log.txt.gz 2026-03-28 01:21:42,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:42,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:42,605 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd029015d0f959922e1/lava-logs.yaml 2026-03-28 01:21:42,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:42,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:42,753 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd029015d0f959922e1/lava_callback.json.gz 2026-03-28 01:21:42,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:42,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd029015d0f959922e1 HTTP/1.1" 200 None 2026-03-28 01:21:42,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:42,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:42,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:42,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd029015d0f959922e1 HTTP/1.1" 200 None 2026-03-28 01:21:42,849 - lava_callback - INFO - Completed processing callback for node 69c72cd029015d0f959922e1 INFO: 172.21.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60546 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:21:58,400 - lava_callback - INFO - Received callback for node 69c72cf429015d0f95992356 INFO: 172.21.0.1:60554 - "POST /node/69c72cf429015d0f95992356 HTTP/1.0" 202 Accepted 2026-03-28 01:21:58,407 - lava_callback - INFO - Starting async job submit for node 69c72cf429015d0f95992356 2026-03-28 01:21:58,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:58,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf429015d0f95992356 HTTP/1.1" 200 None 2026-03-28 01:21:58,453 - lava_callback - DEBUG - Processing node 69c72cf429015d0f95992356: baseline-nfs-arm64 2026-03-28 01:21:58,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:58,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:58,947 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf429015d0f95992356/log.txt.gz 2026-03-28 01:21:59,100 - lava_callback - INFO - Received callback for node 69c725a329015d0f9598fa00 INFO: 172.21.0.1:60570 - "POST /node/69c725a329015d0f9598fa00 HTTP/1.0" 202 Accepted 2026-03-28 01:21:59,121 - lava_callback - INFO - Starting async job submit for node 69c725a329015d0f9598fa00 2026-03-28 01:21:59,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:59,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:59,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a329015d0f9598fa00 HTTP/1.1" 200 None 2026-03-28 01:21:59,226 - lava_callback - DEBUG - Processing node 69c725a329015d0f9598fa00: ltp-cve 2026-03-28 01:21:59,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:59,492 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf429015d0f95992356/lava-logs.yaml 2026-03-28 01:21:59,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:59,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:59,693 - lava_callback - INFO - Received callback for node 69c72cde29015d0f95992300 2026-03-28 01:21:59,702 - lava_callback - INFO - Starting async job submit for node 69c72cde29015d0f95992300 INFO: 172.21.0.1:60578 - "POST /node/69c72cde29015d0f95992300 HTTP/1.0" 202 Accepted 2026-03-28 01:21:59,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:59,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cde29015d0f95992300 HTTP/1.1" 200 None 2026-03-28 01:21:59,770 - lava_callback - DEBUG - Processing node 69c72cde29015d0f95992300: baseline-arm64 2026-03-28 01:21:59,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:59,927 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c725a329015d0f9598fa00/log.txt.gz 2026-03-28 01:22:00,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:00,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:00,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:00,349 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cde29015d0f95992300/log.txt.gz 2026-03-28 01:22:00,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:00,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:00,504 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c725a329015d0f9598fa00/lava-logs.yaml 2026-03-28 01:22:00,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:00,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:00,678 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cde29015d0f95992300/lava-logs.yaml 2026-03-28 01:22:00,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:00,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:00,726 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c725a329015d0f9598fa00/lava_callback.json.gz 2026-03-28 01:22:00,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:00,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a329015d0f9598fa00 HTTP/1.1" 200 None 2026-03-28 01:22:00,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:00,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:22:00,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:00,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725a329015d0f9598fa00 HTTP/1.1" 200 None 2026-03-28 01:22:00,854 - lava_callback - INFO - Completed processing callback for node 69c725a329015d0f9598fa00 2026-03-28 01:22:00,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:00,891 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cde29015d0f95992300/lava_callback.json.gz 2026-03-28 01:22:00,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:00,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cde29015d0f95992300 HTTP/1.1" 200 None 2026-03-28 01:22:00,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:00,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:00,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:01,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cde29015d0f95992300 HTTP/1.1" 200 None 2026-03-28 01:22:01,056 - lava_callback - INFO - Completed processing callback for node 69c72cde29015d0f95992300 2026-03-28 01:22:04,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:04,228 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf429015d0f95992356/lava_callback.json.gz 2026-03-28 01:22:04,240 - lava_callback - INFO - Received callback for node 69c72ccf29015d0f959922dc INFO: 172.21.0.1:36676 - "POST /node/69c72ccf29015d0f959922dc HTTP/1.0" 202 Accepted 2026-03-28 01:22:04,257 - lava_callback - INFO - Starting async job submit for node 69c72ccf29015d0f959922dc 2026-03-28 01:22:04,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:04,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:04,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ccf29015d0f959922dc HTTP/1.1" 200 None 2026-03-28 01:22:04,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf429015d0f95992356 HTTP/1.1" 200 None 2026-03-28 01:22:04,316 - lava_callback - DEBUG - Processing node 69c72ccf29015d0f959922dc: baseline-arm64 2026-03-28 01:22:04,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:04,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:04,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:04,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:04,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cf429015d0f95992356 HTTP/1.1" 200 None 2026-03-28 01:22:04,571 - lava_callback - INFO - Completed processing callback for node 69c72cf429015d0f95992356 2026-03-28 01:22:04,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:04,721 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922dc/log.txt.gz 2026-03-28 01:22:04,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:05,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:05,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922dc/lava-logs.yaml 2026-03-28 01:22:05,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:05,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:05,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922dc/lava_callback.json.gz 2026-03-28 01:22:05,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:05,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ccf29015d0f959922dc HTTP/1.1" 200 None 2026-03-28 01:22:05,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:05,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:05,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:05,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ccf29015d0f959922dc HTTP/1.1" 200 None 2026-03-28 01:22:05,361 - lava_callback - INFO - Completed processing callback for node 69c72ccf29015d0f959922dc 2026-03-28 01:22:05,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:05,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:36682 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:12,875 - lava_callback - INFO - Received callback for node 69c72ce929015d0f95992318 INFO: 172.21.0.1:39836 - "POST /node/69c72ce929015d0f95992318 HTTP/1.0" 202 Accepted 2026-03-28 01:22:12,882 - lava_callback - INFO - Starting async job submit for node 69c72ce929015d0f95992318 2026-03-28 01:22:12,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:12,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce929015d0f95992318 HTTP/1.1" 200 None 2026-03-28 01:22:12,920 - lava_callback - DEBUG - Processing node 69c72ce929015d0f95992318: baseline-nfs-arm64 2026-03-28 01:22:13,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:13,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:13,431 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce929015d0f95992318/log.txt.gz 2026-03-28 01:22:13,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:13,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:13,915 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce929015d0f95992318/lava-logs.yaml 2026-03-28 01:22:13,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:14,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:14,113 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce929015d0f95992318/lava_callback.json.gz 2026-03-28 01:22:14,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:14,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce929015d0f95992318 HTTP/1.1" 200 None 2026-03-28 01:22:14,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:14,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:14,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:14,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce929015d0f95992318 HTTP/1.1" 200 None 2026-03-28 01:22:14,218 - lava_callback - INFO - Completed processing callback for node 69c72ce929015d0f95992318 INFO: 172.21.0.1:39840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39850 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:19,130 - lava_callback - INFO - Received callback for node 69c725a629015d0f9598fa15 INFO: 172.21.0.1:39862 - "POST /node/69c725a629015d0f9598fa15 HTTP/1.0" 202 Accepted 2026-03-28 01:22:19,136 - lava_callback - INFO - Starting async job submit for node 69c725a629015d0f9598fa15 2026-03-28 01:22:19,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:19,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a629015d0f9598fa15 HTTP/1.1" 200 None 2026-03-28 01:22:19,176 - lava_callback - DEBUG - Processing node 69c725a629015d0f9598fa15: ltp-fs_bind 2026-03-28 01:22:19,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:19,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:19,715 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c725a629015d0f9598fa15/log.txt.gz 2026-03-28 01:22:19,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:20,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:20,147 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c725a629015d0f9598fa15/lava-logs.yaml 2026-03-28 01:22:20,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:20,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:20,299 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c725a629015d0f9598fa15/lava_callback.json.gz 2026-03-28 01:22:20,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:20,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a629015d0f9598fa15 HTTP/1.1" 200 None 2026-03-28 01:22:20,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:20,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:22:20,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:20,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725a629015d0f9598fa15 HTTP/1.1" 200 None 2026-03-28 01:22:20,395 - lava_callback - INFO - Completed processing callback for node 69c725a629015d0f9598fa15 2026-03-28 01:22:25,012 - lava_callback - INFO - Received callback for node 69c72d1929015d0f959923d9 INFO: 172.21.0.1:46326 - "POST /node/69c72d1929015d0f959923d9 HTTP/1.0" 202 Accepted 2026-03-28 01:22:25,062 - lava_callback - INFO - Starting async job submit for node 69c72d1929015d0f959923d9 2026-03-28 01:22:25,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:25,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1929015d0f959923d9 HTTP/1.1" 200 None 2026-03-28 01:22:25,099 - lava_callback - DEBUG - Processing node 69c72d1929015d0f959923d9: baseline-arm64-pengutronix 2026-03-28 01:22:25,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:25,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:25,397 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1929015d0f959923d9/log.txt.gz 2026-03-28 01:22:25,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:25,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:25,729 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1929015d0f959923d9/lava-logs.yaml 2026-03-28 01:22:25,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:25,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:25,869 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1929015d0f959923d9/lava_callback.json.gz 2026-03-28 01:22:25,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:25,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1929015d0f959923d9 HTTP/1.1" 200 None 2026-03-28 01:22:25,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:25,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:25,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:25,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1929015d0f959923d9 HTTP/1.1" 200 None 2026-03-28 01:22:25,955 - lava_callback - INFO - Completed processing callback for node 69c72d1929015d0f959923d9 INFO: 172.21.0.1:46336 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:26,718 - lava_callback - INFO - Received callback for node 69c72ce329015d0f9599230c INFO: 172.21.0.1:46352 - "POST /node/69c72ce329015d0f9599230c HTTP/1.0" 202 Accepted 2026-03-28 01:22:26,725 - lava_callback - INFO - Starting async job submit for node 69c72ce329015d0f9599230c 2026-03-28 01:22:26,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:26,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce329015d0f9599230c HTTP/1.1" 200 None 2026-03-28 01:22:26,767 - lava_callback - DEBUG - Processing node 69c72ce329015d0f9599230c: baseline-arm64 2026-03-28 01:22:27,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:27,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:27,390 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce329015d0f9599230c/log.txt.gz 2026-03-28 01:22:27,552 - lava_callback - INFO - Received callback for node 69c7258a29015d0f9598f9e1 INFO: 172.21.0.1:46368 - "POST /node/69c7258a29015d0f9598f9e1 HTTP/1.0" 202 Accepted 2026-03-28 01:22:27,593 - lava_callback - INFO - Starting async job submit for node 69c7258a29015d0f9598f9e1 2026-03-28 01:22:27,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:27,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:27,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258a29015d0f9598f9e1 HTTP/1.1" 200 None 2026-03-28 01:22:27,635 - lava_callback - DEBUG - Processing node 69c7258a29015d0f9598f9e1: baseline-arm-pengutronix 2026-03-28 01:22:27,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:27,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:27,856 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce329015d0f9599230c/lava-logs.yaml 2026-03-28 01:22:27,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:27,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:27,887 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258a29015d0f9598f9e1/log.txt.gz 2026-03-28 01:22:27,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:28,056 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:28,057 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce329015d0f9599230c/lava_callback.json.gz 2026-03-28 01:22:28,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,106 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:28,108 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258a29015d0f9598f9e1/lava-logs.yaml 2026-03-28 01:22:28,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:28,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce329015d0f9599230c HTTP/1.1" 200 None 2026-03-28 01:22:28,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:28,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce329015d0f9599230c HTTP/1.1" 200 None 2026-03-28 01:22:28,236 - lava_callback - INFO - Completed processing callback for node 69c72ce329015d0f9599230c 2026-03-28 01:22:28,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:28,313 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258a29015d0f9598f9e1/lava_callback.json.gz DEBUG: Setup failed for 69c7258a29015d0f9598f9e1. Transitting job node result: incomplete -> fail 2026-03-28 01:22:28,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258a29015d0f9598f9e1 HTTP/1.1" 200 None 2026-03-28 01:22:28,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:22:28,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258a29015d0f9598f9e1 HTTP/1.1" 200 None 2026-03-28 01:22:28,511 - lava_callback - INFO - Completed processing callback for node 69c7258a29015d0f9598f9e1 2026-03-28 01:22:35,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:35,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:40206 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:36,060 - lava_callback - INFO - Received callback for node 69c72d1d29015d0f959923e2 INFO: 172.21.0.1:40208 - "POST /node/69c72d1d29015d0f959923e2 HTTP/1.0" 202 Accepted 2026-03-28 01:22:36,066 - lava_callback - INFO - Starting async job submit for node 69c72d1d29015d0f959923e2 2026-03-28 01:22:36,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:36,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1d29015d0f959923e2 HTTP/1.1" 200 None 2026-03-28 01:22:36,104 - lava_callback - DEBUG - Processing node 69c72d1d29015d0f959923e2: baseline-arm64-foundriesio 2026-03-28 01:22:36,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:36,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:36,483 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e2/log.txt.gz INFO: 172.21.0.1:40220 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:36,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:36,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:36,770 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e2/lava-logs.yaml 2026-03-28 01:22:36,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:36,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:36,905 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e2/lava_callback.json.gz 2026-03-28 01:22:36,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:36,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1d29015d0f959923e2 HTTP/1.1" 200 None 2026-03-28 01:22:36,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:36,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:36,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:37,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1d29015d0f959923e2 HTTP/1.1" 200 None 2026-03-28 01:22:37,071 - lava_callback - INFO - Completed processing callback for node 69c72d1d29015d0f959923e2 2026-03-28 01:22:43,270 - lava_callback - INFO - Received callback for node 69c72cef29015d0f9599234d INFO: 172.21.0.1:55958 - "POST /node/69c72cef29015d0f9599234d HTTP/1.0" 202 Accepted 2026-03-28 01:22:43,277 - lava_callback - INFO - Starting async job submit for node 69c72cef29015d0f9599234d 2026-03-28 01:22:43,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:43,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cef29015d0f9599234d HTTP/1.1" 200 None 2026-03-28 01:22:43,316 - lava_callback - DEBUG - Processing node 69c72cef29015d0f9599234d: baseline-nfs-arm64 2026-03-28 01:22:43,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:43,740 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:43,741 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cef29015d0f9599234d/log.txt.gz 2026-03-28 01:22:43,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:44,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:44,121 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cef29015d0f9599234d/lava-logs.yaml 2026-03-28 01:22:44,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:44,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:44,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cef29015d0f9599234d/lava_callback.json.gz 2026-03-28 01:22:44,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:44,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cef29015d0f9599234d HTTP/1.1" 200 None 2026-03-28 01:22:44,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:44,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:44,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:44,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cef29015d0f9599234d HTTP/1.1" 200 None 2026-03-28 01:22:44,361 - lava_callback - INFO - Completed processing callback for node 69c72cef29015d0f9599234d INFO: 172.21.0.1:55966 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:47,695 - lava_callback - INFO - Received callback for node 69c72d1a29015d0f959923dc INFO: 172.21.0.1:55968 - "POST /node/69c72d1a29015d0f959923dc HTTP/1.0" 202 Accepted 2026-03-28 01:22:47,701 - lava_callback - INFO - Starting async job submit for node 69c72d1a29015d0f959923dc 2026-03-28 01:22:47,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:47,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1a29015d0f959923dc HTTP/1.1" 200 None 2026-03-28 01:22:47,737 - lava_callback - DEBUG - Processing node 69c72d1a29015d0f959923dc: baseline-arm64-pengutronix 2026-03-28 01:22:47,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:48,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:48,083 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1a29015d0f959923dc/log.txt.gz 2026-03-28 01:22:48,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:48,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:48,382 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1a29015d0f959923dc/lava-logs.yaml 2026-03-28 01:22:48,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:48,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:48,526 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1a29015d0f959923dc/lava_callback.json.gz 2026-03-28 01:22:48,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:48,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1a29015d0f959923dc HTTP/1.1" 200 None 2026-03-28 01:22:48,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:48,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:48,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:48,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1a29015d0f959923dc HTTP/1.1" 200 None 2026-03-28 01:22:48,621 - lava_callback - INFO - Completed processing callback for node 69c72d1a29015d0f959923dc 2026-03-28 01:22:53,289 - lava_callback - INFO - Received callback for node 69c72ce029015d0f95992306 INFO: 172.21.0.1:39156 - "POST /node/69c72ce029015d0f95992306 HTTP/1.0" 202 Accepted 2026-03-28 01:22:53,295 - lava_callback - INFO - Starting async job submit for node 69c72ce029015d0f95992306 2026-03-28 01:22:53,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:53,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce029015d0f95992306 HTTP/1.1" 200 None 2026-03-28 01:22:53,332 - lava_callback - DEBUG - Processing node 69c72ce029015d0f95992306: baseline-arm64 2026-03-28 01:22:53,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:53,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:53,714 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce029015d0f95992306/log.txt.gz 2026-03-28 01:22:53,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:54,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:54,043 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce029015d0f95992306/lava-logs.yaml 2026-03-28 01:22:54,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:54,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:54,185 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce029015d0f95992306/lava_callback.json.gz 2026-03-28 01:22:54,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:54,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce029015d0f95992306 HTTP/1.1" 200 None 2026-03-28 01:22:54,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:54,230 - lava_callback - INFO - Received callback for node 69c72d1d29015d0f959923e5 INFO: 172.21.0.1:39166 - "POST /node/69c72d1d29015d0f959923e5 HTTP/1.0" 202 Accepted 2026-03-28 01:22:54,238 - lava_callback - INFO - Starting async job submit for node 69c72d1d29015d0f959923e5 2026-03-28 01:22:54,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:54,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:54,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:54,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1d29015d0f959923e5 HTTP/1.1" 200 None 2026-03-28 01:22:54,284 - lava_callback - DEBUG - Processing node 69c72d1d29015d0f959923e5: baseline-arm64-foundriesio 2026-03-28 01:22:54,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce029015d0f95992306 HTTP/1.1" 200 None 2026-03-28 01:22:54,328 - lava_callback - INFO - Completed processing callback for node 69c72ce029015d0f95992306 2026-03-28 01:22:54,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:54,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:54,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e5/log.txt.gz 2026-03-28 01:22:55,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:55,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:55,414 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e5/lava-logs.yaml 2026-03-28 01:22:55,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:55,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:55,572 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e5/lava_callback.json.gz 2026-03-28 01:22:55,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:55,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1d29015d0f959923e5 HTTP/1.1" 200 None 2026-03-28 01:22:55,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:55,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:55,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:55,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1d29015d0f959923e5 HTTP/1.1" 200 None 2026-03-28 01:22:55,662 - lava_callback - INFO - Completed processing callback for node 69c72d1d29015d0f959923e5 2026-03-28 01:22:56,021 - lava_callback - INFO - Received callback for node 69c72d1b29015d0f959923df INFO: 172.21.0.1:39168 - "POST /node/69c72d1b29015d0f959923df HTTP/1.0" 202 Accepted 2026-03-28 01:22:56,028 - lava_callback - INFO - Starting async job submit for node 69c72d1b29015d0f959923df 2026-03-28 01:22:56,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:56,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1b29015d0f959923df HTTP/1.1" 200 None 2026-03-28 01:22:56,067 - lava_callback - DEBUG - Processing node 69c72d1b29015d0f959923df: baseline-arm64-pengutronix 2026-03-28 01:22:56,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:56,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:56,413 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1b29015d0f959923df/log.txt.gz INFO: 172.21.0.1:39172 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:56,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:56,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:56,725 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1b29015d0f959923df/lava-logs.yaml 2026-03-28 01:22:56,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:56,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:56,858 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1b29015d0f959923df/lava_callback.json.gz 2026-03-28 01:22:56,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:56,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1b29015d0f959923df HTTP/1.1" 200 None 2026-03-28 01:22:56,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:56,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:56,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:56,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1b29015d0f959923df HTTP/1.1" 200 None 2026-03-28 01:22:56,937 - lava_callback - INFO - Completed processing callback for node 69c72d1b29015d0f959923df 2026-03-28 01:23:00,687 - lava_callback - INFO - Received callback for node 69c72cf529015d0f95992359 INFO: 172.21.0.1:58156 - "POST /node/69c72cf529015d0f95992359 HTTP/1.0" 202 Accepted 2026-03-28 01:23:00,693 - lava_callback - INFO - Starting async job submit for node 69c72cf529015d0f95992359 2026-03-28 01:23:00,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:00,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf529015d0f95992359 HTTP/1.1" 200 None 2026-03-28 01:23:00,734 - lava_callback - DEBUG - Processing node 69c72cf529015d0f95992359: baseline-nfs-arm64 2026-03-28 01:23:01,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:01,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:01,300 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf529015d0f95992359/log.txt.gz 2026-03-28 01:23:01,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:01,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:01,736 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf529015d0f95992359/lava-logs.yaml 2026-03-28 01:23:01,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:01,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:01,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf529015d0f95992359/lava_callback.json.gz 2026-03-28 01:23:01,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:01,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf529015d0f95992359 HTTP/1.1" 200 None 2026-03-28 01:23:01,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:01,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:23:01,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:01,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cf529015d0f95992359 HTTP/1.1" 200 None 2026-03-28 01:23:01,993 - lava_callback - INFO - Completed processing callback for node 69c72cf529015d0f95992359 2026-03-28 01:23:04,333 - lava_callback - INFO - Received callback for node 69c48a67590270ad7b6d974d INFO: 172.21.0.1:58160 - "POST /node/69c48a67590270ad7b6d974d HTTP/1.0" 202 Accepted 2026-03-28 01:23:04,339 - lava_callback - INFO - Starting async job submit for node 69c48a67590270ad7b6d974d 2026-03-28 01:23:04,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:04,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48a67590270ad7b6d974d HTTP/1.1" 200 None 2026-03-28 01:23:04,361 - lava_callback - DEBUG - Processing node 69c48a67590270ad7b6d974d: baseline-arm64-clabbe 2026-03-28 01:23:04,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48a67590270ad7b6d974d 2026-03-28 01:23:04,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:04,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48a67590270ad7b6d974d HTTP/1.1" 200 None 2026-03-28 01:23:04,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:04,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:23:04,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:04,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48a67590270ad7b6d974d HTTP/1.1" 200 None 2026-03-28 01:23:04,414 - lava_callback - ERROR - Error processing callback for node 69c48a67590270ad7b6d974d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:23:05,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:05,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:58176 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:23:14,025 - lava_callback - INFO - Received callback for node 69c72d1429015d0f959923d0 INFO: 172.21.0.1:41206 - "POST /node/69c72d1429015d0f959923d0 HTTP/1.0" 202 Accepted 2026-03-28 01:23:14,032 - lava_callback - INFO - Starting async job submit for node 69c72d1429015d0f959923d0 2026-03-28 01:23:14,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:14,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1429015d0f959923d0 HTTP/1.1" 200 None 2026-03-28 01:23:14,072 - lava_callback - DEBUG - Processing node 69c72d1429015d0f959923d0: baseline-arm64-clabbe 2026-03-28 01:23:14,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:14,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:14,428 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c72d1429015d0f959923d0/log.txt.gz 2026-03-28 01:23:14,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:14,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:14,737 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c72d1429015d0f959923d0/lava-logs.yaml 2026-03-28 01:23:14,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:14,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:14,889 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c72d1429015d0f959923d0/lava_callback.json.gz 2026-03-28 01:23:14,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:14,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1429015d0f959923d0 HTTP/1.1" 200 None 2026-03-28 01:23:14,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:14,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:23:14,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:14,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1429015d0f959923d0 HTTP/1.1" 200 None 2026-03-28 01:23:14,991 - lava_callback - INFO - Completed processing callback for node 69c72d1429015d0f959923d0 INFO: 172.21.0.1:41210 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 01:23:15,319 - lava_callback - INFO - Received callback for node 69c72cf229015d0f95992353 INFO: 172.21.0.1:41216 - "POST /node/69c72cf229015d0f95992353 HTTP/1.0" 202 Accepted 2026-03-28 01:23:15,327 - lava_callback - INFO - Starting async job submit for node 69c72cf229015d0f95992353 2026-03-28 01:23:15,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:15,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf229015d0f95992353 HTTP/1.1" 200 None 2026-03-28 01:23:15,370 - lava_callback - DEBUG - Processing node 69c72cf229015d0f95992353: baseline-nfs-arm64 2026-03-28 01:23:15,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:15,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:15,908 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf229015d0f95992353/log.txt.gz 2026-03-28 01:23:16,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:16,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:16,408 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf229015d0f95992353/lava-logs.yaml 2026-03-28 01:23:16,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:41222 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:23:16,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:16,581 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf229015d0f95992353/lava_callback.json.gz 2026-03-28 01:23:16,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:16,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf229015d0f95992353 HTTP/1.1" 200 None 2026-03-28 01:23:16,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:16,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:23:16,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:16,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cf229015d0f95992353 HTTP/1.1" 200 None 2026-03-28 01:23:16,670 - lava_callback - INFO - Completed processing callback for node 69c72cf229015d0f95992353 INFO: 172.21.0.1:34860 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:23:35,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:35,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39614 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:23:54,027 - lava_callback - INFO - Received callback for node 69c72cd229015d0f959922e5 INFO: 172.21.0.1:53370 - "POST /node/69c72cd229015d0f959922e5 HTTP/1.0" 202 Accepted 2026-03-28 01:23:54,033 - lava_callback - INFO - Starting async job submit for node 69c72cd229015d0f959922e5 2026-03-28 01:23:54,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:54,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd229015d0f959922e5 HTTP/1.1" 200 None 2026-03-28 01:23:54,070 - lava_callback - DEBUG - Processing node 69c72cd229015d0f959922e5: baseline-arm64 2026-03-28 01:23:54,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:54,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:54,475 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd229015d0f959922e5/log.txt.gz 2026-03-28 01:23:54,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:54,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:54,804 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd229015d0f959922e5/lava-logs.yaml 2026-03-28 01:23:54,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:54,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:54,983 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd229015d0f959922e5/lava_callback.json.gz 2026-03-28 01:23:55,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:55,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd229015d0f959922e5 HTTP/1.1" 200 None 2026-03-28 01:23:55,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:55,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:23:55,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:55,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd229015d0f959922e5 HTTP/1.1" 200 None 2026-03-28 01:23:55,119 - lava_callback - INFO - Completed processing callback for node 69c72cd229015d0f959922e5 INFO: 172.21.0.1:53378 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:24:05,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:24:05,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41578 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:24:51,405 - lava_callback - INFO - Received callback for node 69c72ce629015d0f95992312 INFO: 172.21.0.1:51136 - "POST /node/69c72ce629015d0f95992312 HTTP/1.0" 202 Accepted 2026-03-28 01:24:51,411 - lava_callback - INFO - Starting async job submit for node 69c72ce629015d0f95992312 2026-03-28 01:24:51,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:24:51,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce629015d0f95992312 HTTP/1.1" 200 None 2026-03-28 01:24:51,450 - lava_callback - DEBUG - Processing node 69c72ce629015d0f95992312: baseline-nfs-arm64 2026-03-28 01:24:51,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:24:51,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:24:51,927 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce629015d0f95992312/log.txt.gz 2026-03-28 01:24:52,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:24:52,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:24:52,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce629015d0f95992312/lava-logs.yaml 2026-03-28 01:24:52,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:24:52,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:24:52,549 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce629015d0f95992312/lava_callback.json.gz 2026-03-28 01:24:52,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:24:52,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce629015d0f95992312 HTTP/1.1" 200 None 2026-03-28 01:24:52,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:24:52,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:24:52,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:24:52,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce629015d0f95992312 HTTP/1.1" 200 None 2026-03-28 01:24:52,645 - lava_callback - INFO - Completed processing callback for node 69c72ce629015d0f95992312 INFO: 172.21.0.1:51140 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:25:05,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:05,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38666 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:25:09,598 - lava_callback - INFO - Received callback for node 69c72cdb29015d0f959922fa INFO: 172.21.0.1:38680 - "POST /node/69c72cdb29015d0f959922fa HTTP/1.0" 202 Accepted 2026-03-28 01:25:09,605 - lava_callback - INFO - Starting async job submit for node 69c72cdb29015d0f959922fa 2026-03-28 01:25:09,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:09,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdb29015d0f959922fa HTTP/1.1" 200 None 2026-03-28 01:25:09,642 - lava_callback - DEBUG - Processing node 69c72cdb29015d0f959922fa: baseline-arm64 2026-03-28 01:25:10,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:10,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:10,204 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdb29015d0f959922fa/log.txt.gz 2026-03-28 01:25:10,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:10,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:10,928 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdb29015d0f959922fa/lava-logs.yaml 2026-03-28 01:25:10,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:11,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:11,116 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdb29015d0f959922fa/lava_callback.json.gz 2026-03-28 01:25:11,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:11,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdb29015d0f959922fa HTTP/1.1" 200 None 2026-03-28 01:25:11,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:11,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:25:11,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:11,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cdb29015d0f959922fa HTTP/1.1" 200 None 2026-03-28 01:25:11,299 - lava_callback - INFO - Completed processing callback for node 69c72cdb29015d0f959922fa INFO: 172.21.0.1:52610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44790 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:25:28,562 - lava_callback - INFO - Received callback for node 69c72ced29015d0f95992347 INFO: 172.21.0.1:44796 - "POST /node/69c72ced29015d0f95992347 HTTP/1.0" 202 Accepted 2026-03-28 01:25:28,569 - lava_callback - INFO - Starting async job submit for node 69c72ced29015d0f95992347 2026-03-28 01:25:28,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:28,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ced29015d0f95992347 HTTP/1.1" 200 None 2026-03-28 01:25:28,630 - lava_callback - DEBUG - Processing node 69c72ced29015d0f95992347: baseline-nfs-arm64 2026-03-28 01:25:29,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:29,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:29,270 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ced29015d0f95992347/log.txt.gz 2026-03-28 01:25:29,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:29,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:29,767 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ced29015d0f95992347/lava-logs.yaml 2026-03-28 01:25:29,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:29,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:29,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ced29015d0f95992347/lava_callback.json.gz 2026-03-28 01:25:29,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:29,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ced29015d0f95992347 HTTP/1.1" 200 None 2026-03-28 01:25:29,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:29,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:25:29,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:30,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ced29015d0f95992347 HTTP/1.1" 200 None 2026-03-28 01:25:30,025 - lava_callback - INFO - Completed processing callback for node 69c72ced29015d0f95992347 2026-03-28 01:25:35,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:35,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58164 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:26:33,371 - lava_callback - INFO - Received callback for node 69c72d0f29015d0f959923bf INFO: 172.21.0.1:35982 - "POST /node/69c72d0f29015d0f959923bf HTTP/1.0" 202 Accepted 2026-03-28 01:26:33,377 - lava_callback - INFO - Starting async job submit for node 69c72d0f29015d0f959923bf 2026-03-28 01:26:33,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:33,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0f29015d0f959923bf HTTP/1.1" 200 None 2026-03-28 01:26:33,419 - lava_callback - DEBUG - Processing node 69c72d0f29015d0f959923bf: baseline-arm64-broonie 2026-03-28 01:26:33,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:33,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:33,980 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0f29015d0f959923bf/log.txt.gz 2026-03-28 01:26:34,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:34,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:34,329 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0f29015d0f959923bf/lava-logs.yaml 2026-03-28 01:26:34,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:34,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:34,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0f29015d0f959923bf/lava_callback.json.gz 2026-03-28 01:26:34,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:34,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0f29015d0f959923bf HTTP/1.1" 200 None 2026-03-28 01:26:34,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:34,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:26:34,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:34,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0f29015d0f959923bf HTTP/1.1" 200 None 2026-03-28 01:26:34,545 - lava_callback - INFO - Completed processing callback for node 69c72d0f29015d0f959923bf 2026-03-28 01:26:35,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:35,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36002 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:26:39,343 - lava_callback - INFO - Received callback for node 69c725a829015d0f9598fa18 INFO: 172.21.0.1:36010 - "POST /node/69c725a829015d0f9598fa18 HTTP/1.0" 202 Accepted 2026-03-28 01:26:39,349 - lava_callback - INFO - Starting async job submit for node 69c725a829015d0f9598fa18 2026-03-28 01:26:39,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:39,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a829015d0f9598fa18 HTTP/1.1" 200 None 2026-03-28 01:26:39,391 - lava_callback - DEBUG - Processing node 69c725a829015d0f9598fa18: ltp-fsx 2026-03-28 01:26:39,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:39,929 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:39,931 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c725a829015d0f9598fa18/log.txt.gz 2026-03-28 01:26:40,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:40,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:40,378 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c725a829015d0f9598fa18/lava-logs.yaml 2026-03-28 01:26:40,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:40,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:40,555 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c725a829015d0f9598fa18/lava_callback.json.gz 2026-03-28 01:26:40,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:40,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a829015d0f9598fa18 HTTP/1.1" 200 None 2026-03-28 01:26:40,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:40,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:26:40,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:40,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725a829015d0f9598fa18 HTTP/1.1" 200 None 2026-03-28 01:26:40,653 - lava_callback - INFO - Completed processing callback for node 69c725a829015d0f9598fa18 2026-03-28 01:26:41,020 - lava_callback - INFO - Received callback for node 69c725aa29015d0f9598fa1b INFO: 172.21.0.1:48218 - "POST /node/69c725aa29015d0f9598fa1b HTTP/1.0" 202 Accepted 2026-03-28 01:26:41,027 - lava_callback - INFO - Starting async job submit for node 69c725aa29015d0f9598fa1b 2026-03-28 01:26:41,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:41,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725aa29015d0f9598fa1b HTTP/1.1" 200 None 2026-03-28 01:26:41,074 - lava_callback - DEBUG - Processing node 69c725aa29015d0f9598fa1b: ltp-input 2026-03-28 01:26:41,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:41,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:41,567 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c725aa29015d0f9598fa1b/log.txt.gz 2026-03-28 01:26:41,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:41,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:41,976 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c725aa29015d0f9598fa1b/lava-logs.yaml 2026-03-28 01:26:41,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:42,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:42,145 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c725aa29015d0f9598fa1b/lava_callback.json.gz 2026-03-28 01:26:42,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:42,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725aa29015d0f9598fa1b HTTP/1.1" 200 None 2026-03-28 01:26:42,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:42,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:26:42,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:42,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725aa29015d0f9598fa1b HTTP/1.1" 200 None 2026-03-28 01:26:42,238 - lava_callback - INFO - Completed processing callback for node 69c725aa29015d0f9598fa1b INFO: 172.21.0.1:48222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35876 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:27:05,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:05,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38146 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:27:16,864 - lava_callback - INFO - Received callback for node 69c72cda29015d0f959922f7 INFO: 172.21.0.1:38152 - "POST /node/69c72cda29015d0f959922f7 HTTP/1.0" 202 Accepted 2026-03-28 01:27:16,871 - lava_callback - INFO - Starting async job submit for node 69c72cda29015d0f959922f7 2026-03-28 01:27:16,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:16,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cda29015d0f959922f7 HTTP/1.1" 200 None 2026-03-28 01:27:16,917 - lava_callback - DEBUG - Processing node 69c72cda29015d0f959922f7: baseline-arm64 2026-03-28 01:27:17,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:17,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:17,479 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cda29015d0f959922f7/log.txt.gz 2026-03-28 01:27:17,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:17,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:17,957 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cda29015d0f959922f7/lava-logs.yaml 2026-03-28 01:27:17,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:18,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:18,123 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cda29015d0f959922f7/lava_callback.json.gz 2026-03-28 01:27:18,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:18,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cda29015d0f959922f7 HTTP/1.1" 200 None 2026-03-28 01:27:18,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:18,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:27:18,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:18,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cda29015d0f959922f7 HTTP/1.1" 200 None 2026-03-28 01:27:18,214 - lava_callback - INFO - Completed processing callback for node 69c72cda29015d0f959922f7 2026-03-28 01:27:20,141 - lava_callback - INFO - Received callback for node 69c725ac29015d0f9598fa21 INFO: 172.21.0.1:45444 - "POST /node/69c725ac29015d0f9598fa21 HTTP/1.0" 202 Accepted 2026-03-28 01:27:20,151 - lava_callback - INFO - Starting async job submit for node 69c725ac29015d0f9598fa21 2026-03-28 01:27:20,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:20,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725ac29015d0f9598fa21 HTTP/1.1" 200 None 2026-03-28 01:27:20,196 - lava_callback - DEBUG - Processing node 69c725ac29015d0f9598fa21: ltp-ipc 2026-03-28 01:27:20,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:20,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:20,744 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c725ac29015d0f9598fa21/log.txt.gz 2026-03-28 01:27:20,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:21,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:21,239 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c725ac29015d0f9598fa21/lava-logs.yaml 2026-03-28 01:27:21,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:21,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:21,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c725ac29015d0f9598fa21/lava_callback.json.gz 2026-03-28 01:27:21,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:21,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725ac29015d0f9598fa21 HTTP/1.1" 200 None 2026-03-28 01:27:21,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:21,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:27:21,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:21,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725ac29015d0f9598fa21 HTTP/1.1" 200 None 2026-03-28 01:27:21,495 - lava_callback - INFO - Completed processing callback for node 69c725ac29015d0f9598fa21 INFO: 172.21.0.1:45448 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:27:35,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:35,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39458 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:28:10,679 - lava_callback - INFO - Received callback for node 69c725ae29015d0f9598fa24 INFO: 172.21.0.1:52048 - "POST /node/69c725ae29015d0f9598fa24 HTTP/1.0" 202 Accepted 2026-03-28 01:28:10,686 - lava_callback - INFO - Starting async job submit for node 69c725ae29015d0f9598fa24 2026-03-28 01:28:10,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:10,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725ae29015d0f9598fa24 HTTP/1.1" 200 None 2026-03-28 01:28:10,743 - lava_callback - DEBUG - Processing node 69c725ae29015d0f9598fa24: ltp-pty 2026-03-28 01:28:10,819 - lava_callback - INFO - Received callback for node 69c72ceb29015d0f95992344 INFO: 172.21.0.1:52054 - "POST /node/69c72ceb29015d0f95992344 HTTP/1.0" 202 Accepted 2026-03-28 01:28:10,836 - lava_callback - INFO - Starting async job submit for node 69c72ceb29015d0f95992344 2026-03-28 01:28:10,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:10,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ceb29015d0f95992344 HTTP/1.1" 200 None 2026-03-28 01:28:11,009 - lava_callback - DEBUG - Processing node 69c72ceb29015d0f95992344: baseline-nfs-arm64 2026-03-28 01:28:11,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:11,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:11,625 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c725ae29015d0f9598fa24/log.txt.gz 2026-03-28 01:28:11,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:11,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:11,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:11,886 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ceb29015d0f95992344/log.txt.gz 2026-03-28 01:28:12,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:12,152 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c725ae29015d0f9598fa24/lava-logs.yaml 2026-03-28 01:28:12,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:12,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:12,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:12,425 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ceb29015d0f95992344/lava-logs.yaml 2026-03-28 01:28:12,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:12,436 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c725ae29015d0f9598fa24/lava_callback.json.gz 2026-03-28 01:28:12,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:12,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725ae29015d0f9598fa24 HTTP/1.1" 200 None 2026-03-28 01:28:12,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:28:12,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725ae29015d0f9598fa24 HTTP/1.1" 200 None 2026-03-28 01:28:12,549 - lava_callback - INFO - Completed processing callback for node 69c725ae29015d0f9598fa24 2026-03-28 01:28:12,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:12,685 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ceb29015d0f95992344/lava_callback.json.gz 2026-03-28 01:28:12,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ceb29015d0f95992344 HTTP/1.1" 200 None 2026-03-28 01:28:12,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:28:12,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ceb29015d0f95992344 HTTP/1.1" 200 None 2026-03-28 01:28:12,787 - lava_callback - INFO - Completed processing callback for node 69c72ceb29015d0f95992344 INFO: 172.21.0.1:52058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43022 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:28:34,284 - lava_callback - INFO - Received callback for node 69c72ce429015d0f9599230f INFO: 172.21.0.1:39358 - "POST /node/69c72ce429015d0f9599230f HTTP/1.0" 202 Accepted 2026-03-28 01:28:34,291 - lava_callback - INFO - Starting async job submit for node 69c72ce429015d0f9599230f 2026-03-28 01:28:34,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:34,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce429015d0f9599230f HTTP/1.1" 200 None 2026-03-28 01:28:34,349 - lava_callback - DEBUG - Processing node 69c72ce429015d0f9599230f: baseline-arm64 2026-03-28 01:28:34,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:34,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:34,847 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce429015d0f9599230f/log.txt.gz 2026-03-28 01:28:35,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:35,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:35,279 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce429015d0f9599230f/lava-logs.yaml 2026-03-28 01:28:35,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:35,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:35,433 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce429015d0f9599230f/lava_callback.json.gz DEBUG: Setup failed for 69c72ce429015d0f9599230f. Transitting job node result: incomplete -> fail 2026-03-28 01:28:35,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:35,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce429015d0f9599230f HTTP/1.1" 200 None 2026-03-28 01:28:35,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:35,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:28:35,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:35,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce429015d0f9599230f HTTP/1.1" 200 None 2026-03-28 01:28:35,647 - lava_callback - INFO - Completed processing callback for node 69c72ce429015d0f9599230f 2026-03-28 01:28:35,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:35,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45374 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:30:59,456 - lava_callback - INFO - Received callback for node 69c725af29015d0f9598fa27 INFO: 172.21.0.1:45376 - "POST /node/69c725af29015d0f9598fa27 HTTP/1.0" 202 Accepted 2026-03-28 01:30:59,463 - lava_callback - INFO - Starting async job submit for node 69c725af29015d0f9598fa27 2026-03-28 01:30:59,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:30:59,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725af29015d0f9598fa27 HTTP/1.1" 200 None 2026-03-28 01:30:59,503 - lava_callback - DEBUG - Processing node 69c725af29015d0f9598fa27: ltp-sched 2026-03-28 01:30:59,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:00,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:00,011 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c725af29015d0f9598fa27/log.txt.gz 2026-03-28 01:31:00,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:00,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:00,483 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c725af29015d0f9598fa27/lava-logs.yaml 2026-03-28 01:31:00,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:00,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:00,825 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c725af29015d0f9598fa27/lava_callback.json.gz 2026-03-28 01:31:00,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:00,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725af29015d0f9598fa27 HTTP/1.1" 200 None 2026-03-28 01:31:00,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:00,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:31:00,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:00,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725af29015d0f9598fa27 HTTP/1.1" 200 None 2026-03-28 01:31:00,928 - lava_callback - INFO - Completed processing callback for node 69c725af29015d0f9598fa27 2026-03-28 01:31:05,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:05,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45114 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:31:39,691 - lava_callback - INFO - Received callback for node 69c725b129015d0f9598fa2a INFO: 172.21.0.1:45128 - "POST /node/69c725b129015d0f9598fa2a HTTP/1.0" 202 Accepted 2026-03-28 01:31:39,698 - lava_callback - INFO - Starting async job submit for node 69c725b129015d0f9598fa2a 2026-03-28 01:31:39,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:39,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b129015d0f9598fa2a HTTP/1.1" 200 None 2026-03-28 01:31:39,743 - lava_callback - DEBUG - Processing node 69c725b129015d0f9598fa2a: ltp-smoke 2026-03-28 01:31:40,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:40,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:40,920 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c725b129015d0f9598fa2a/log.txt.gz 2026-03-28 01:31:41,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:41,277 - lava_callback - INFO - Received callback for node 69c725b529015d0f9598fa30 INFO: 172.21.0.1:33056 - "POST /node/69c725b529015d0f9598fa30 HTTP/1.0" 202 Accepted 2026-03-28 01:31:41,288 - lava_callback - INFO - Starting async job submit for node 69c725b529015d0f9598fa30 2026-03-28 01:31:41,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:41,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b529015d0f9598fa30 HTTP/1.1" 200 None 2026-03-28 01:31:41,363 - lava_callback - DEBUG - Processing node 69c725b529015d0f9598fa30: ltp-syscalls-ipc 2026-03-28 01:31:41,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:41,786 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c725b129015d0f9598fa2a/lava-logs.yaml 2026-03-28 01:31:41,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:41,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:42,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:42,048 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c725b129015d0f9598fa2a/lava_callback.json.gz 2026-03-28 01:31:42,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:42,083 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c725b529015d0f9598fa30/log.txt.gz 2026-03-28 01:31:42,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:42,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b129015d0f9598fa2a HTTP/1.1" 200 None 2026-03-28 01:31:42,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:42,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:31:42,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:42,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725b129015d0f9598fa2a HTTP/1.1" 200 None 2026-03-28 01:31:42,335 - lava_callback - INFO - Completed processing callback for node 69c725b129015d0f9598fa2a 2026-03-28 01:31:42,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:42,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:42,846 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c725b529015d0f9598fa30/lava-logs.yaml 2026-03-28 01:31:42,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:43,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:43,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c725b529015d0f9598fa30/lava_callback.json.gz 2026-03-28 01:31:43,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:43,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b529015d0f9598fa30 HTTP/1.1" 200 None 2026-03-28 01:31:43,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:43,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:31:43,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:43,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725b529015d0f9598fa30 HTTP/1.1" 200 None 2026-03-28 01:31:43,423 - lava_callback - INFO - Completed processing callback for node 69c725b529015d0f9598fa30 INFO: 172.21.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50680 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:32:05,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:05,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57336 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:32:10,567 - lava_callback - INFO - Received callback for node 69c725b629015d0f9598fa33 INFO: 172.21.0.1:38510 - "POST /node/69c725b629015d0f9598fa33 HTTP/1.0" 202 Accepted 2026-03-28 01:32:10,577 - lava_callback - INFO - Starting async job submit for node 69c725b629015d0f9598fa33 2026-03-28 01:32:10,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:10,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b629015d0f9598fa33 HTTP/1.1" 200 None 2026-03-28 01:32:10,663 - lava_callback - DEBUG - Processing node 69c725b629015d0f9598fa33: ltp-timers 2026-03-28 01:32:11,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:32:11,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:32:11,611 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c725b629015d0f9598fa33/log.txt.gz 2026-03-28 01:32:11,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:32:12,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:32:12,133 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c725b629015d0f9598fa33/lava-logs.yaml 2026-03-28 01:32:12,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:32:12,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:32:12,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c725b629015d0f9598fa33/lava_callback.json.gz 2026-03-28 01:32:12,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:12,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b629015d0f9598fa33 HTTP/1.1" 200 None 2026-03-28 01:32:12,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:12,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:32:12,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:12,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725b629015d0f9598fa33 HTTP/1.1" 200 None 2026-03-28 01:32:12,571 - lava_callback - INFO - Completed processing callback for node 69c725b629015d0f9598fa33 INFO: 172.21.0.1:38514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50118 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:32:35,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:35,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37372 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:33:28,515 - lava_callback - INFO - Received callback for node 69c48cd3590270ad7b6d97ef INFO: 172.21.0.1:37380 - "POST /node/69c48cd3590270ad7b6d97ef HTTP/1.0" 202 Accepted 2026-03-28 01:33:28,520 - lava_callback - INFO - Starting async job submit for node 69c48cd3590270ad7b6d97ef 2026-03-28 01:33:28,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:28,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48cd3590270ad7b6d97ef HTTP/1.1" 200 None 2026-03-28 01:33:28,540 - lava_callback - DEBUG - Processing node 69c48cd3590270ad7b6d97ef: baseline-arm64-baylibre 2026-03-28 01:33:28,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48cd3590270ad7b6d97ef 2026-03-28 01:33:28,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:28,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48cd3590270ad7b6d97ef HTTP/1.1" 200 None 2026-03-28 01:33:28,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:28,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:33:28,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:28,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48cd3590270ad7b6d97ef HTTP/1.1" 200 None 2026-03-28 01:33:28,600 - lava_callback - ERROR - Error processing callback for node 69c48cd3590270ad7b6d97ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:33:32,018 - lava_callback - INFO - Received callback for node 69c72d0829015d0f95992392 INFO: 172.21.0.1:43456 - "POST /node/69c72d0829015d0f95992392 HTTP/1.0" 202 Accepted 2026-03-28 01:33:32,025 - lava_callback - INFO - Starting async job submit for node 69c72d0829015d0f95992392 2026-03-28 01:33:32,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:32,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0829015d0f95992392 HTTP/1.1" 200 None 2026-03-28 01:33:32,062 - lava_callback - DEBUG - Processing node 69c72d0829015d0f95992392: baseline-arm64-broonie 2026-03-28 01:33:32,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:33:32,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:33:32,460 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0829015d0f95992392/log.txt.gz 2026-03-28 01:33:32,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:33:32,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:33:32,791 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0829015d0f95992392/lava-logs.yaml 2026-03-28 01:33:32,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:33:32,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:33:32,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0829015d0f95992392/lava_callback.json.gz 2026-03-28 01:33:33,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:33,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0829015d0f95992392 HTTP/1.1" 200 None 2026-03-28 01:33:33,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:33,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:33:33,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:33,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0829015d0f95992392 HTTP/1.1" 200 None 2026-03-28 01:33:33,086 - lava_callback - INFO - Completed processing callback for node 69c72d0829015d0f95992392 2026-03-28 01:33:35,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:35,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47110 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:35:08,621 - lava_callback - INFO - Received callback for node 69c48d49590270ad7b6d980e INFO: 172.21.0.1:47112 - "POST /node/69c48d49590270ad7b6d980e HTTP/1.0" 202 Accepted 2026-03-28 01:35:08,629 - lava_callback - INFO - Starting async job submit for node 69c48d49590270ad7b6d980e 2026-03-28 01:35:08,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:08,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48d49590270ad7b6d980e HTTP/1.1" 200 None 2026-03-28 01:35:08,644 - lava_callback - DEBUG - Processing node 69c48d49590270ad7b6d980e: baseline-arm64-baylibre 2026-03-28 01:35:08,645 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48d49590270ad7b6d980e 2026-03-28 01:35:08,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:08,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48d49590270ad7b6d980e HTTP/1.1" 200 None 2026-03-28 01:35:08,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:08,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:35:08,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:08,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48d49590270ad7b6d980e HTTP/1.1" 200 None 2026-03-28 01:35:08,708 - lava_callback - ERROR - Error processing callback for node 69c48d49590270ad7b6d980e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43698 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:35:19,780 - lava_callback - INFO - Received callback for node 69c725b829015d0f9598fa36 INFO: 172.21.0.1:53886 - "POST /node/69c725b829015d0f9598fa36 HTTP/1.0" 202 Accepted 2026-03-28 01:35:19,786 - lava_callback - INFO - Starting async job submit for node 69c725b829015d0f9598fa36 2026-03-28 01:35:19,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:19,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b829015d0f9598fa36 HTTP/1.1" 200 None 2026-03-28 01:35:19,832 - lava_callback - DEBUG - Processing node 69c725b829015d0f9598fa36: ltp-watchqueue 2026-03-28 01:35:20,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:35:20,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:35:20,372 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c725b829015d0f9598fa36/log.txt.gz 2026-03-28 01:35:20,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:35:20,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:35:20,797 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c725b829015d0f9598fa36/lava-logs.yaml 2026-03-28 01:35:20,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:35:20,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:35:20,982 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c725b829015d0f9598fa36/lava_callback.json.gz 2026-03-28 01:35:21,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:21,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b829015d0f9598fa36 HTTP/1.1" 200 None 2026-03-28 01:35:21,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:21,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:35:21,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:21,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725b829015d0f9598fa36 HTTP/1.1" 200 None 2026-03-28 01:35:21,076 - lava_callback - INFO - Completed processing callback for node 69c725b829015d0f9598fa36 INFO: 172.21.0.1:53890 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:35:35,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:35,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56732 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:36:33,457 - lava_callback - INFO - Received callback for node 69c72c8029015d0f959922a6 INFO: 172.21.0.1:43654 - "POST /node/69c72c8029015d0f959922a6 HTTP/1.0" 202 Accepted 2026-03-28 01:36:33,464 - lava_callback - INFO - Starting async job submit for node 69c72c8029015d0f959922a6 2026-03-28 01:36:33,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:36:33,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72c8029015d0f959922a6 HTTP/1.1" 200 None 2026-03-28 01:36:33,560 - lava_callback - DEBUG - Processing node 69c72c8029015d0f959922a6: ltp-smoke 2026-03-28 01:36:34,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:36:34,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:36:34,257 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c72c8029015d0f959922a6/log.txt.gz 2026-03-28 01:36:34,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:36:34,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:36:34,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c72c8029015d0f959922a6/lava-logs.yaml 2026-03-28 01:36:34,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:36:34,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:36:34,987 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c72c8029015d0f959922a6/lava_callback.json.gz 2026-03-28 01:36:35,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:36:35,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72c8029015d0f959922a6 HTTP/1.1" 200 None 2026-03-28 01:36:35,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:36:35,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4229015d0f9598f742 HTTP/1.1" 200 None 2026-03-28 01:36:35,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:36:35,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72c8029015d0f959922a6 HTTP/1.1" 200 None 2026-03-28 01:36:35,390 - lava_callback - INFO - Completed processing callback for node 69c72c8029015d0f959922a6 2026-03-28 01:36:35,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:36:35,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58446 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:37:39,515 - lava_callback - INFO - Received callback for node 69c72cd029015d0f959922de INFO: 172.21.0.1:58456 - "POST /node/69c72cd029015d0f959922de HTTP/1.0" 202 Accepted 2026-03-28 01:37:39,522 - lava_callback - INFO - Starting async job submit for node 69c72cd029015d0f959922de 2026-03-28 01:37:39,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:39,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd029015d0f959922de HTTP/1.1" 200 None 2026-03-28 01:37:39,592 - lava_callback - DEBUG - Processing node 69c72cd029015d0f959922de: baseline-arm 2026-03-28 01:37:40,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:37:40,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:37:40,382 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c72cd029015d0f959922de/log.txt.gz 2026-03-28 01:37:40,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:37:41,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:37:41,002 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c72cd029015d0f959922de/lava-logs.yaml 2026-03-28 01:37:41,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:37:41,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:37:41,192 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c72cd029015d0f959922de/lava_callback.json.gz DEBUG: Setup failed for 69c72cd029015d0f959922de. Transitting job node result: incomplete -> fail 2026-03-28 01:37:41,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:41,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd029015d0f959922de HTTP/1.1" 200 None 2026-03-28 01:37:41,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:41,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:37:41,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:41,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd029015d0f959922de HTTP/1.1" 200 None 2026-03-28 01:37:41,468 - lava_callback - INFO - Completed processing callback for node 69c72cd029015d0f959922de INFO: 172.21.0.1:42972 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:37:48,739 - lava_callback - INFO - Received callback for node 69c48de9590270ad7b6d9824 INFO: 172.21.0.1:42978 - "POST /node/69c48de9590270ad7b6d9824 HTTP/1.0" 202 Accepted 2026-03-28 01:37:48,744 - lava_callback - INFO - Starting async job submit for node 69c48de9590270ad7b6d9824 2026-03-28 01:37:48,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:48,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48de9590270ad7b6d9824 HTTP/1.1" 200 None 2026-03-28 01:37:48,779 - lava_callback - DEBUG - Processing node 69c48de9590270ad7b6d9824: baseline-arm64-baylibre 2026-03-28 01:37:48,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48de9590270ad7b6d9824 2026-03-28 01:37:48,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:48,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48de9590270ad7b6d9824 HTTP/1.1" 200 None 2026-03-28 01:37:48,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:48,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:37:48,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:48,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48de9590270ad7b6d9824 HTTP/1.1" 200 None 2026-03-28 01:37:48,837 - lava_callback - ERROR - Error processing callback for node 69c48de9590270ad7b6d9824: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53818 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:38:05,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:38:05,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43586 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:39:43,342 - lava_callback - INFO - Received callback for node 69c72cd829015d0f959922f4 INFO: 172.21.0.1:58184 - "POST /node/69c72cd829015d0f959922f4 HTTP/1.0" 202 Accepted 2026-03-28 01:39:43,349 - lava_callback - INFO - Starting async job submit for node 69c72cd829015d0f959922f4 2026-03-28 01:39:43,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:39:43,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd829015d0f959922f4 HTTP/1.1" 200 None 2026-03-28 01:39:43,386 - lava_callback - DEBUG - Processing node 69c72cd829015d0f959922f4: baseline-arm64 2026-03-28 01:39:43,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:39:43,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:39:43,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd829015d0f959922f4/log.txt.gz 2026-03-28 01:39:44,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:39:44,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:39:44,368 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd829015d0f959922f4/lava-logs.yaml 2026-03-28 01:39:44,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:39:44,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:39:44,514 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd829015d0f959922f4/lava_callback.json.gz 2026-03-28 01:39:44,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:39:44,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd829015d0f959922f4 HTTP/1.1" 200 None 2026-03-28 01:39:44,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:39:44,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:39:44,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:39:44,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd829015d0f959922f4 HTTP/1.1" 200 None 2026-03-28 01:39:44,605 - lava_callback - INFO - Completed processing callback for node 69c72cd829015d0f959922f4 INFO: 172.21.0.1:58198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39106 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:40:05,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:05,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47006 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:40:19,639 - lava_callback - INFO - Received callback for node 69c72d0729015d0f9599238f INFO: 172.21.0.1:47010 - "POST /node/69c72d0729015d0f9599238f HTTP/1.0" 202 Accepted 2026-03-28 01:40:19,645 - lava_callback - INFO - Starting async job submit for node 69c72d0729015d0f9599238f 2026-03-28 01:40:19,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:19,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0729015d0f9599238f HTTP/1.1" 200 None 2026-03-28 01:40:19,683 - lava_callback - DEBUG - Processing node 69c72d0729015d0f9599238f: baseline-arm64-broonie 2026-03-28 01:40:19,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:40:20,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:40:20,076 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0729015d0f9599238f/log.txt.gz 2026-03-28 01:40:20,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:40:20,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:40:20,500 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0729015d0f9599238f/lava-logs.yaml 2026-03-28 01:40:20,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:40:20,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:40:20,654 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0729015d0f9599238f/lava_callback.json.gz 2026-03-28 01:40:20,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:20,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0729015d0f9599238f HTTP/1.1" 200 None 2026-03-28 01:40:20,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:20,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:40:20,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:20,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0729015d0f9599238f HTTP/1.1" 200 None 2026-03-28 01:40:20,747 - lava_callback - INFO - Completed processing callback for node 69c72d0729015d0f9599238f INFO: 172.21.0.1:35116 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:40:35,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:35,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59242 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:40:45,203 - lava_callback - INFO - Received callback for node 69c48e95590270ad7b6d9851 INFO: 172.21.0.1:43708 - "POST /node/69c48e95590270ad7b6d9851 HTTP/1.0" 202 Accepted 2026-03-28 01:40:45,208 - lava_callback - INFO - Starting async job submit for node 69c48e95590270ad7b6d9851 2026-03-28 01:40:45,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:45,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48e95590270ad7b6d9851 HTTP/1.1" 200 None 2026-03-28 01:40:45,225 - lava_callback - DEBUG - Processing node 69c48e95590270ad7b6d9851: baseline-arm64-clabbe 2026-03-28 01:40:45,226 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48e95590270ad7b6d9851 2026-03-28 01:40:45,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:45,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48e95590270ad7b6d9851 HTTP/1.1" 200 None 2026-03-28 01:40:45,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:45,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:40:45,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:45,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48e95590270ad7b6d9851 HTTP/1.1" 200 None 2026-03-28 01:40:45,312 - lava_callback - ERROR - Error processing callback for node 69c48e95590270ad7b6d9851: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60330 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:45226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39806 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:44:05,358 - lava_callback - INFO - Received callback for node 69c48f5f590270ad7b6d9876 INFO: 172.21.0.1:34278 - "POST /node/69c48f5f590270ad7b6d9876 HTTP/1.0" 202 Accepted 2026-03-28 01:44:05,364 - lava_callback - INFO - Starting async job submit for node 69c48f5f590270ad7b6d9876 2026-03-28 01:44:05,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:44:05,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48f5f590270ad7b6d9876 HTTP/1.1" 200 None 2026-03-28 01:44:05,392 - lava_callback - DEBUG - Processing node 69c48f5f590270ad7b6d9876: baseline-arm64-clabbe 2026-03-28 01:44:05,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48f5f590270ad7b6d9876 2026-03-28 01:44:05,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:44:05,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48f5f590270ad7b6d9876 HTTP/1.1" 200 None 2026-03-28 01:44:05,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:44:05,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:44:05,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:44:05,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48f5f590270ad7b6d9876 HTTP/1.1" 200 None 2026-03-28 01:44:05,452 - lava_callback - ERROR - Error processing callback for node 69c48f5f590270ad7b6d9876: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57438 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:45:19,854 - lava_callback - INFO - Received callback for node 69c72d0c29015d0f959923b8 INFO: 172.21.0.1:37038 - "POST /node/69c72d0c29015d0f959923b8 HTTP/1.0" 202 Accepted 2026-03-28 01:45:19,860 - lava_callback - INFO - Starting async job submit for node 69c72d0c29015d0f959923b8 2026-03-28 01:45:19,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:45:19,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0c29015d0f959923b8 HTTP/1.1" 200 None 2026-03-28 01:45:19,907 - lava_callback - DEBUG - Processing node 69c72d0c29015d0f959923b8: baseline-arm64-broonie 2026-03-28 01:45:20,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:45:20,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:45:20,341 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0c29015d0f959923b8/log.txt.gz 2026-03-28 01:45:20,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:45:20,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:45:20,703 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0c29015d0f959923b8/lava-logs.yaml 2026-03-28 01:45:20,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:45:20,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:45:20,855 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0c29015d0f959923b8/lava_callback.json.gz 2026-03-28 01:45:20,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:45:20,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0c29015d0f959923b8 HTTP/1.1" 200 None 2026-03-28 01:45:20,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:45:20,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:45:20,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:45:20,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0c29015d0f959923b8 HTTP/1.1" 200 None 2026-03-28 01:45:20,961 - lava_callback - INFO - Completed processing callback for node 69c72d0c29015d0f959923b8 INFO: 172.21.0.1:37040 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:45:35,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:45:35,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49596 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:47:05,524 - lava_callback - INFO - Received callback for node 69c49003590270ad7b6d989f INFO: 172.21.0.1:53588 - "POST /node/69c49003590270ad7b6d989f HTTP/1.0" 202 Accepted 2026-03-28 01:47:05,530 - lava_callback - INFO - Starting async job submit for node 69c49003590270ad7b6d989f 2026-03-28 01:47:05,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:47:05,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49003590270ad7b6d989f HTTP/1.1" 200 None 2026-03-28 01:47:05,546 - lava_callback - DEBUG - Processing node 69c49003590270ad7b6d989f: baseline-arm64-clabbe 2026-03-28 01:47:05,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49003590270ad7b6d989f 2026-03-28 01:47:05,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:47:05,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49003590270ad7b6d989f HTTP/1.1" 200 None 2026-03-28 01:47:05,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:47:05,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:47:05,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:47:05,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49003590270ad7b6d989f HTTP/1.1" 200 None 2026-03-28 01:47:05,647 - lava_callback - ERROR - Error processing callback for node 69c49003590270ad7b6d989f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60612 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:48:39,810 - lava_callback - INFO - Received callback for node 69c7337229015d0f9599264c INFO: 172.21.0.1:35900 - "POST /node/69c7337229015d0f9599264c HTTP/1.0" 202 Accepted 2026-03-28 01:48:39,817 - lava_callback - INFO - Starting async job submit for node 69c7337229015d0f9599264c 2026-03-28 01:48:39,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:48:39,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7337229015d0f9599264c HTTP/1.1" 200 None 2026-03-28 01:48:39,839 - lava_callback - DEBUG - Processing node 69c7337229015d0f9599264c: baseline-arm 2026-03-28 01:48:39,844 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:48:39,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:48:40,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:48:40,007 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7337229015d0f9599264c/lava-logs.yaml 2026-03-28 01:48:40,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:48:40,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:48:40,300 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7337229015d0f9599264c/lava_callback.json.gz 2026-03-28 01:48:40,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:48:40,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7337229015d0f9599264c HTTP/1.1" 200 None 2026-03-28 01:48:40,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:48:40,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:48:40,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:48:40,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7337229015d0f9599264c HTTP/1.1" 200 None 2026-03-28 01:48:40,362 - lava_callback - INFO - Completed processing callback for node 69c7337229015d0f9599264c INFO: 172.21.0.1:35916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45178 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:49:05,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:49:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33068 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:49:45,639 - lava_callback - INFO - Received callback for node 69c490a5590270ad7b6d98c5 INFO: 172.21.0.1:36802 - "POST /node/69c490a5590270ad7b6d98c5 HTTP/1.0" 202 Accepted 2026-03-28 01:49:45,645 - lava_callback - INFO - Starting async job submit for node 69c490a5590270ad7b6d98c5 2026-03-28 01:49:45,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:49:45,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c490a5590270ad7b6d98c5 HTTP/1.1" 200 None 2026-03-28 01:49:45,663 - lava_callback - DEBUG - Processing node 69c490a5590270ad7b6d98c5: baseline-arm64-clabbe 2026-03-28 01:49:45,663 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c490a5590270ad7b6d98c5 2026-03-28 01:49:45,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:49:45,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c490a5590270ad7b6d98c5 HTTP/1.1" 200 None 2026-03-28 01:49:45,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:49:45,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:49:45,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:49:45,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c490a5590270ad7b6d98c5 HTTP/1.1" 200 None 2026-03-28 01:49:45,717 - lava_callback - ERROR - Error processing callback for node 69c490a5590270ad7b6d98c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57662 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:50:06,783 - lava_callback - INFO - Received callback for node 69c733c129015d0f9599266d INFO: 172.21.0.1:57672 - "POST /node/69c733c129015d0f9599266d HTTP/1.0" 202 Accepted 2026-03-28 01:50:06,789 - lava_callback - INFO - Starting async job submit for node 69c733c129015d0f9599266d 2026-03-28 01:50:06,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:50:06,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c733c129015d0f9599266d HTTP/1.1" 200 None 2026-03-28 01:50:06,814 - lava_callback - DEBUG - Processing node 69c733c129015d0f9599266d: baseline-nfs-arm64 2026-03-28 01:50:06,815 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:50:06,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:50:06,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:50:06,974 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c733c129015d0f9599266d/lava-logs.yaml 2026-03-28 01:50:06,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:50:07,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:50:07,109 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c733c129015d0f9599266d/lava_callback.json.gz 2026-03-28 01:50:07,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:50:07,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c733c129015d0f9599266d HTTP/1.1" 200 None 2026-03-28 01:50:07,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:50:07,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:50:07,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:50:07,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c733c129015d0f9599266d HTTP/1.1" 200 None 2026-03-28 01:50:07,176 - lava_callback - INFO - Completed processing callback for node 69c733c129015d0f9599266d INFO: 172.21.0.1:56814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38076 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:50:35,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:50:36,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35924 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:51:05,656 - lava_callback - INFO - Received callback for node 69c490f7590270ad7b6d98d5 INFO: 172.21.0.1:53810 - "POST /node/69c490f7590270ad7b6d98d5 HTTP/1.0" 202 Accepted 2026-03-28 01:51:05,662 - lava_callback - INFO - Starting async job submit for node 69c490f7590270ad7b6d98d5 2026-03-28 01:51:05,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:51:05,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c490f7590270ad7b6d98d5 HTTP/1.1" 200 None 2026-03-28 01:51:05,686 - lava_callback - DEBUG - Processing node 69c490f7590270ad7b6d98d5: baseline-arm64-clabbe 2026-03-28 01:51:05,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c490f7590270ad7b6d98d5 2026-03-28 01:51:05,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:51:05,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c490f7590270ad7b6d98d5 HTTP/1.1" 200 None 2026-03-28 01:51:05,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:51:05,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:51:05,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:51:05,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c490f7590270ad7b6d98d5 HTTP/1.1" 200 None 2026-03-28 01:51:05,740 - lava_callback - ERROR - Error processing callback for node 69c490f7590270ad7b6d98d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42996 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:52:25,701 - lava_callback - INFO - Received callback for node 69c49149590270ad7b6d98e6 INFO: 172.21.0.1:51928 - "POST /node/69c49149590270ad7b6d98e6 HTTP/1.0" 202 Accepted 2026-03-28 01:52:25,706 - lava_callback - INFO - Starting async job submit for node 69c49149590270ad7b6d98e6 2026-03-28 01:52:25,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:52:25,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49149590270ad7b6d98e6 HTTP/1.1" 200 None 2026-03-28 01:52:25,729 - lava_callback - DEBUG - Processing node 69c49149590270ad7b6d98e6: baseline-arm64-clabbe 2026-03-28 01:52:25,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49149590270ad7b6d98e6 2026-03-28 01:52:25,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:52:25,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49149590270ad7b6d98e6 HTTP/1.1" 200 None 2026-03-28 01:52:25,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:52:25,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:52:25,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:52:25,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49149590270ad7b6d98e6 HTTP/1.1" 200 None 2026-03-28 01:52:25,781 - lava_callback - ERROR - Error processing callback for node 69c49149590270ad7b6d98e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51826 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:54:33,462 - lava_callback - INFO - Received callback for node 69c734b029015d0f95992693 INFO: 172.21.0.1:55918 - "POST /node/69c734b029015d0f95992693 HTTP/1.0" 202 Accepted 2026-03-28 01:54:33,473 - lava_callback - INFO - Starting async job submit for node 69c734b029015d0f95992693 2026-03-28 01:54:33,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:54:33,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c734b029015d0f95992693 HTTP/1.1" 200 None 2026-03-28 01:54:33,521 - lava_callback - DEBUG - Processing node 69c734b029015d0f95992693: baseline-arm64 2026-03-28 01:54:33,658 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:54:33,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:54:34,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:54:34,134 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c734b029015d0f95992693/lava-logs.yaml 2026-03-28 01:54:34,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:54:34,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:54:34,449 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c734b029015d0f95992693/lava_callback.json.gz 2026-03-28 01:54:34,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:54:34,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c734b029015d0f95992693 HTTP/1.1" 200 None 2026-03-28 01:54:34,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:54:34,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:54:34,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:54:34,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c734b029015d0f95992693 HTTP/1.1" 200 None 2026-03-28 01:54:34,609 - lava_callback - INFO - Completed processing callback for node 69c734b029015d0f95992693 2026-03-28 01:54:36,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:54:36,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60518 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:57:23,019 - lava_callback - INFO - Received callback for node 69c7350429015d0f959926a0 INFO: 172.21.0.1:58182 - "POST /node/69c7350429015d0f959926a0 HTTP/1.0" 202 Accepted 2026-03-28 01:57:23,026 - lava_callback - INFO - Starting async job submit for node 69c7350429015d0f959926a0 2026-03-28 01:57:23,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:57:23,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7350429015d0f959926a0 HTTP/1.1" 200 None 2026-03-28 01:57:23,072 - lava_callback - DEBUG - Processing node 69c7350429015d0f959926a0: baseline-arm64 2026-03-28 01:57:23,281 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:57:23,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:57:23,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:57:23,631 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7350429015d0f959926a0/lava-logs.yaml 2026-03-28 01:57:23,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:57:23,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:57:23,965 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7350429015d0f959926a0/lava_callback.json.gz 2026-03-28 01:57:24,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:57:24,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7350429015d0f959926a0 HTTP/1.1" 200 None 2026-03-28 01:57:24,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:57:24,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:57:24,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:57:24,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7350429015d0f959926a0 HTTP/1.1" 200 None 2026-03-28 01:57:24,153 - lava_callback - INFO - Completed processing callback for node 69c7350429015d0f959926a0 INFO: 172.21.0.1:58184 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:57:36,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:57:36,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:58:35,395 - lava_callback - INFO - Received callback for node 69c7355a29015d0f959926aa INFO: 172.21.0.1:44504 - "POST /node/69c7355a29015d0f959926aa HTTP/1.0" 202 Accepted 2026-03-28 01:58:35,401 - lava_callback - INFO - Starting async job submit for node 69c7355a29015d0f959926aa 2026-03-28 01:58:35,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:58:35,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7355a29015d0f959926aa HTTP/1.1" 200 None 2026-03-28 01:58:35,438 - lava_callback - DEBUG - Processing node 69c7355a29015d0f959926aa: baseline-arm64 2026-03-28 01:58:35,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:58:36,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:58:36,094 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7355a29015d0f959926aa/log.txt.gz 2026-03-28 01:58:36,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:44514 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:58:36,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 INFO: 172.21.0.1:44528 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:58:36,516 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7355a29015d0f959926aa/lava-logs.yaml 2026-03-28 01:58:36,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:58:36,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:58:36,701 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7355a29015d0f959926aa/lava_callback.json.gz 2026-03-28 01:58:36,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:58:36,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7355a29015d0f959926aa HTTP/1.1" 200 None 2026-03-28 01:58:36,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:58:36,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:58:36,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:58:36,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7355a29015d0f959926aa HTTP/1.1" 200 None 2026-03-28 01:58:36,862 - lava_callback - INFO - Completed processing callback for node 69c7355a29015d0f959926aa INFO: 172.21.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43572 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:59:06,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:06,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33306 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:59:46,278 - lava_callback - INFO - Received callback for node 69c72cea29015d0f9599231b INFO: 172.21.0.1:52460 - "POST /node/69c72cea29015d0f9599231b HTTP/1.0" 202 Accepted 2026-03-28 01:59:46,287 - lava_callback - INFO - Starting async job submit for node 69c72cea29015d0f9599231b 2026-03-28 01:59:46,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:46,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cea29015d0f9599231b HTTP/1.1" 200 None 2026-03-28 01:59:46,346 - lava_callback - DEBUG - Processing node 69c72cea29015d0f9599231b: baseline-nfs-arm64 INFO: 172.21.0.1:52470 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:59:47,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:47,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:47,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cea29015d0f9599231b/log.txt.gz 2026-03-28 01:59:48,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:48,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:48,567 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cea29015d0f9599231b/lava-logs.yaml 2026-03-28 01:59:48,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:48,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:48,933 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cea29015d0f9599231b/lava_callback.json.gz DEBUG: Setup failed for 69c72cea29015d0f9599231b. Transitting job node result: incomplete -> fail 2026-03-28 01:59:49,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:49,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cea29015d0f9599231b HTTP/1.1" 200 None 2026-03-28 01:59:49,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:49,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:59:49,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:49,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cea29015d0f9599231b HTTP/1.1" 200 None 2026-03-28 01:59:49,175 - lava_callback - INFO - Completed processing callback for node 69c72cea29015d0f9599231b 2026-03-28 01:59:51,034 - lava_callback - INFO - Received callback for node 69c735e629015d0f959926cd INFO: 172.21.0.1:35502 - "POST /node/69c735e629015d0f959926cd HTTP/1.0" 202 Accepted 2026-03-28 01:59:51,040 - lava_callback - INFO - Starting async job submit for node 69c735e629015d0f959926cd 2026-03-28 01:59:51,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:51,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c735e629015d0f959926cd HTTP/1.1" 200 None 2026-03-28 01:59:51,079 - lava_callback - DEBUG - Processing node 69c735e629015d0f959926cd: baseline-arm64-foundriesio 2026-03-28 01:59:51,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:51,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:51,570 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c735e629015d0f959926cd/log.txt.gz 2026-03-28 01:59:51,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:51,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:51,869 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c735e629015d0f959926cd/lava-logs.yaml 2026-03-28 01:59:51,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:52,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:52,017 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c735e629015d0f959926cd/lava_callback.json.gz 2026-03-28 01:59:52,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:52,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c735e629015d0f959926cd HTTP/1.1" 200 None 2026-03-28 01:59:52,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:52,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:59:52,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:52,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c735e629015d0f959926cd HTTP/1.1" 200 None 2026-03-28 01:59:52,134 - lava_callback - INFO - Completed processing callback for node 69c735e629015d0f959926cd INFO: 172.21.0.1:35514 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:00:06,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:00:06,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56498 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:03:47,129 - lava_callback - INFO - Received callback for node 69c736b929015d0f9599270b INFO: 172.21.0.1:56502 - "POST /node/69c736b929015d0f9599270b HTTP/1.0" 202 Accepted 2026-03-28 02:03:47,136 - lava_callback - INFO - Starting async job submit for node 69c736b929015d0f9599270b 2026-03-28 02:03:47,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:03:47,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c736b929015d0f9599270b HTTP/1.1" 200 None 2026-03-28 02:03:47,174 - lava_callback - DEBUG - Processing node 69c736b929015d0f9599270b: baseline-arm64 2026-03-28 02:03:47,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:03:47,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:03:47,795 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c736b929015d0f9599270b/log.txt.gz 2026-03-28 02:03:48,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:03:48,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:03:48,421 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c736b929015d0f9599270b/lava-logs.yaml 2026-03-28 02:03:48,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:03:48,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:03:48,605 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c736b929015d0f9599270b/lava_callback.json.gz 2026-03-28 02:03:48,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:03:48,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c736b929015d0f9599270b HTTP/1.1" 200 None 2026-03-28 02:03:48,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:03:48,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:03:48,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:03:48,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c736b929015d0f9599270b HTTP/1.1" 200 None 2026-03-28 02:03:48,776 - lava_callback - INFO - Completed processing callback for node 69c736b929015d0f9599270b INFO: 172.21.0.1:60338 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:04:06,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:04:06,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35824 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:12:33,714 - lava_callback - INFO - Received callback for node 69c7375629015d0f9599273a INFO: 172.21.0.1:45990 - "POST /node/69c7375629015d0f9599273a HTTP/1.0" 202 Accepted 2026-03-28 02:12:33,722 - lava_callback - INFO - Starting async job submit for node 69c7375629015d0f9599273a 2026-03-28 02:12:33,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:12:33,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7375629015d0f9599273a HTTP/1.1" 200 None 2026-03-28 02:12:33,774 - lava_callback - DEBUG - Processing node 69c7375629015d0f9599273a: baseline-arm64 2026-03-28 02:12:34,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:12:34,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:12:34,368 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7375629015d0f9599273a/log.txt.gz 2026-03-28 02:12:34,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:12:34,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:12:34,836 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7375629015d0f9599273a/lava-logs.yaml 2026-03-28 02:12:34,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:12:35,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:12:35,024 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7375629015d0f9599273a/lava_callback.json.gz DEBUG: Setup failed for 69c7375629015d0f9599273a. Transitting job node result: incomplete -> fail 2026-03-28 02:12:35,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:12:35,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7375629015d0f9599273a HTTP/1.1" 200 None 2026-03-28 02:12:35,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:12:35,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:12:35,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:12:35,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7375629015d0f9599273a HTTP/1.1" 200 None 2026-03-28 02:12:35,235 - lava_callback - INFO - Completed processing callback for node 69c7375629015d0f9599273a 2026-03-28 02:12:36,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:12:36,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59628 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:18:02,468 - lava_callback - INFO - Received callback for node 69c73a4229015d0f959927b3 INFO: 172.21.0.1:56108 - "POST /node/69c73a4229015d0f959927b3 HTTP/1.0" 202 Accepted 2026-03-28 02:18:02,474 - lava_callback - INFO - Starting async job submit for node 69c73a4229015d0f959927b3 2026-03-28 02:18:02,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:18:02,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73a4229015d0f959927b3 HTTP/1.1" 200 None 2026-03-28 02:18:02,500 - lava_callback - DEBUG - Processing node 69c73a4229015d0f959927b3: baseline-nfs-arm64 2026-03-28 02:18:02,501 - lava_callback - WARNING - Failed to upload log 2026-03-28 02:18:02,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:18:02,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:18:02,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73a4229015d0f959927b3/lava-logs.yaml 2026-03-28 02:18:02,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:18:02,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:18:02,946 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73a4229015d0f959927b3/lava_callback.json.gz 2026-03-28 02:18:02,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:18:02,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73a4229015d0f959927b3 HTTP/1.1" 200 None 2026-03-28 02:18:02,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:18:02,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:18:02,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:18:03,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73a4229015d0f959927b3 HTTP/1.1" 200 None 2026-03-28 02:18:03,017 - lava_callback - INFO - Completed processing callback for node 69c73a4229015d0f959927b3 2026-03-28 02:18:06,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:18:06,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41862 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:23:54,625 - lava_callback - INFO - Received callback for node 69c73b8c29015d0f959927e3 INFO: 172.21.0.1:34752 - "POST /node/69c73b8c29015d0f959927e3 HTTP/1.0" 202 Accepted 2026-03-28 02:23:54,632 - lava_callback - INFO - Starting async job submit for node 69c73b8c29015d0f959927e3 2026-03-28 02:23:54,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:23:54,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73b8c29015d0f959927e3 HTTP/1.1" 200 None 2026-03-28 02:23:54,677 - lava_callback - DEBUG - Processing node 69c73b8c29015d0f959927e3: baseline-arm64 2026-03-28 02:23:54,815 - lava_callback - WARNING - Failed to upload log 2026-03-28 02:23:54,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:23:55,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:23:55,160 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73b8c29015d0f959927e3/lava-logs.yaml 2026-03-28 02:23:55,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:23:55,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:23:55,339 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73b8c29015d0f959927e3/lava_callback.json.gz 2026-03-28 02:23:55,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:23:55,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73b8c29015d0f959927e3 HTTP/1.1" 200 None 2026-03-28 02:23:55,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:23:55,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:23:55,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:23:55,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73b8c29015d0f959927e3 HTTP/1.1" 200 None 2026-03-28 02:23:55,489 - lava_callback - INFO - Completed processing callback for node 69c73b8c29015d0f959927e3 INFO: 172.21.0.1:34756 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:24:06,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:24:06,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37054 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:25:31,449 - lava_callback - INFO - Received callback for node 69c49906590270ad7b6d99db INFO: 172.21.0.1:40610 - "POST /node/69c49906590270ad7b6d99db HTTP/1.0" 202 Accepted 2026-03-28 02:25:31,455 - lava_callback - INFO - Starting async job submit for node 69c49906590270ad7b6d99db 2026-03-28 02:25:31,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:25:31,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49906590270ad7b6d99db HTTP/1.1" 200 None 2026-03-28 02:25:31,470 - lava_callback - DEBUG - Processing node 69c49906590270ad7b6d99db: baseline-arm64-baylibre 2026-03-28 02:25:31,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49906590270ad7b6d99db 2026-03-28 02:25:31,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:25:31,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49906590270ad7b6d99db HTTP/1.1" 200 None 2026-03-28 02:25:31,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:25:31,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:25:31,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:25:31,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49906590270ad7b6d99db HTTP/1.1" 200 None 2026-03-28 02:25:31,526 - lava_callback - ERROR - Error processing callback for node 69c49906590270ad7b6d99db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34802 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:26:07,336 - lava_callback - INFO - Received callback for node 69c73bff29015d0f959927f2 INFO: 172.21.0.1:34812 - "POST /node/69c73bff29015d0f959927f2 HTTP/1.0" 202 Accepted 2026-03-28 02:26:07,343 - lava_callback - INFO - Starting async job submit for node 69c73bff29015d0f959927f2 2026-03-28 02:26:07,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:07,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73bff29015d0f959927f2 HTTP/1.1" 200 None 2026-03-28 02:26:07,389 - lava_callback - DEBUG - Processing node 69c73bff29015d0f959927f2: baseline-arm64 2026-03-28 02:26:07,551 - lava_callback - WARNING - Failed to upload log 2026-03-28 02:26:07,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:26:07,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:26:07,811 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73bff29015d0f959927f2/lava-logs.yaml 2026-03-28 02:26:07,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:26:07,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:26:07,973 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73bff29015d0f959927f2/lava_callback.json.gz 2026-03-28 02:26:08,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:08,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73bff29015d0f959927f2 HTTP/1.1" 200 None 2026-03-28 02:26:08,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:08,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:26:08,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:08,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73bff29015d0f959927f2 HTTP/1.1" 200 None 2026-03-28 02:26:08,131 - lava_callback - INFO - Completed processing callback for node 69c73bff29015d0f959927f2 INFO: 172.21.0.1:44516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44522 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:26:21,400 - lava_callback - INFO - Received callback for node 69c7386e29015d0f95992769 INFO: 172.21.0.1:54332 - "POST /node/69c7386e29015d0f95992769 HTTP/1.0" 202 Accepted 2026-03-28 02:26:21,407 - lava_callback - INFO - Starting async job submit for node 69c7386e29015d0f95992769 2026-03-28 02:26:21,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:21,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7386e29015d0f95992769 HTTP/1.1" 200 None 2026-03-28 02:26:21,487 - lava_callback - DEBUG - Processing node 69c7386e29015d0f95992769: baseline-arm 2026-03-28 02:26:22,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:26:22,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:26:22,222 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7386e29015d0f95992769/log.txt.gz 2026-03-28 02:26:22,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:26:22,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:26:22,838 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7386e29015d0f95992769/lava-logs.yaml 2026-03-28 02:26:22,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:26:23,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:26:23,002 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7386e29015d0f95992769/lava_callback.json.gz DEBUG: Setup failed for 69c7386e29015d0f95992769. Transitting job node result: incomplete -> fail 2026-03-28 02:26:23,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:23,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7386e29015d0f95992769 HTTP/1.1" 200 None 2026-03-28 02:26:23,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:23,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 02:26:23,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:23,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7386e29015d0f95992769 HTTP/1.1" 200 None 2026-03-28 02:26:23,271 - lava_callback - INFO - Completed processing callback for node 69c7386e29015d0f95992769 INFO: 172.21.0.1:54336 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:26:36,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:36,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:26:51,349 - lava_callback - INFO - Received callback for node 69c49969590270ad7b6d99e5 INFO: 172.21.0.1:37452 - "POST /node/69c49969590270ad7b6d99e5 HTTP/1.0" 202 Accepted 2026-03-28 02:26:51,355 - lava_callback - INFO - Starting async job submit for node 69c49969590270ad7b6d99e5 2026-03-28 02:26:51,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:51,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49969590270ad7b6d99e5 HTTP/1.1" 200 None 2026-03-28 02:26:51,371 - lava_callback - DEBUG - Processing node 69c49969590270ad7b6d99e5: baseline-arm64-baylibre 2026-03-28 02:26:51,371 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49969590270ad7b6d99e5 2026-03-28 02:26:51,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:51,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49969590270ad7b6d99e5 HTTP/1.1" 200 None 2026-03-28 02:26:51,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:51,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:26:51,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:51,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49969590270ad7b6d99e5 HTTP/1.1" 200 None 2026-03-28 02:26:51,427 - lava_callback - ERROR - Error processing callback for node 69c49969590270ad7b6d99e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49148 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:27:32,939 - lava_callback - INFO - Received callback for node 69c7386c29015d0f95992766 INFO: 172.21.0.1:39088 - "POST /node/69c7386c29015d0f95992766 HTTP/1.0" 202 Accepted 2026-03-28 02:27:32,946 - lava_callback - INFO - Starting async job submit for node 69c7386c29015d0f95992766 2026-03-28 02:27:33,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:33,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7386c29015d0f95992766 HTTP/1.1" 200 None 2026-03-28 02:27:33,027 - lava_callback - DEBUG - Processing node 69c7386c29015d0f95992766: ltp-smoke 2026-03-28 02:27:33,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:33,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:33,833 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7386c29015d0f95992766/log.txt.gz 2026-03-28 02:27:34,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:34,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:34,324 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7386c29015d0f95992766/lava-logs.yaml 2026-03-28 02:27:34,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:34,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:34,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7386c29015d0f95992766/lava_callback.json.gz 2026-03-28 02:27:34,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:34,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7386c29015d0f95992766 HTTP/1.1" 200 None 2026-03-28 02:27:34,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:34,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4229015d0f9598f742 HTTP/1.1" 200 None 2026-03-28 02:27:34,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:34,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7386c29015d0f95992766 HTTP/1.1" 200 None 2026-03-28 02:27:34,814 - lava_callback - INFO - Completed processing callback for node 69c7386c29015d0f95992766 2026-03-28 02:27:36,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:36,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39102 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:27:37,319 - lava_callback - INFO - Received callback for node 69c73c5c29015d0f95992804 INFO: 172.21.0.1:39104 - "POST /node/69c73c5c29015d0f95992804 HTTP/1.0" 202 Accepted 2026-03-28 02:27:37,326 - lava_callback - INFO - Starting async job submit for node 69c73c5c29015d0f95992804 2026-03-28 02:27:37,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:37,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73c5c29015d0f95992804 HTTP/1.1" 200 None 2026-03-28 02:27:37,363 - lava_callback - DEBUG - Processing node 69c73c5c29015d0f95992804: baseline-arm64 2026-03-28 02:27:37,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:37,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:37,771 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73c5c29015d0f95992804/log.txt.gz 2026-03-28 02:27:37,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:38,146 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:38,148 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73c5c29015d0f95992804/lava-logs.yaml 2026-03-28 02:27:38,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:38,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:38,463 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73c5c29015d0f95992804/lava_callback.json.gz 2026-03-28 02:27:38,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:38,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73c5c29015d0f95992804 HTTP/1.1" 200 None 2026-03-28 02:27:38,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:38,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:27:38,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:38,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73c5c29015d0f95992804 HTTP/1.1" 200 None 2026-03-28 02:27:38,545 - lava_callback - INFO - Completed processing callback for node 69c73c5c29015d0f95992804 INFO: 172.21.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56138 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:28:06,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:28:06,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45854 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:30:26,600 - lava_callback - INFO - Received callback for node 69c73cef29015d0f9599282b INFO: 172.21.0.1:45862 - "POST /node/69c73cef29015d0f9599282b HTTP/1.0" 202 Accepted 2026-03-28 02:30:26,606 - lava_callback - INFO - Starting async job submit for node 69c73cef29015d0f9599282b 2026-03-28 02:30:26,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:26,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73cef29015d0f9599282b HTTP/1.1" 200 None 2026-03-28 02:30:26,643 - lava_callback - DEBUG - Processing node 69c73cef29015d0f9599282b: baseline-arm64 2026-03-28 02:30:27,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:30:27,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:30:27,287 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73cef29015d0f9599282b/log.txt.gz 2026-03-28 02:30:27,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:30:27,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:30:27,789 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73cef29015d0f9599282b/lava-logs.yaml 2026-03-28 02:30:27,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:30:27,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:30:27,968 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73cef29015d0f9599282b/lava_callback.json.gz 2026-03-28 02:30:28,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:28,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73cef29015d0f9599282b HTTP/1.1" 200 None 2026-03-28 02:30:28,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:28,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:30:28,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:28,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73cef29015d0f9599282b HTTP/1.1" 200 None 2026-03-28 02:30:28,185 - lava_callback - INFO - Completed processing callback for node 69c73cef29015d0f9599282b 2026-03-28 02:30:31,553 - lava_callback - INFO - Received callback for node 69c49a34590270ad7b6d99f3 INFO: 172.21.0.1:43506 - "POST /node/69c49a34590270ad7b6d99f3 HTTP/1.0" 202 Accepted 2026-03-28 02:30:31,558 - lava_callback - INFO - Starting async job submit for node 69c49a34590270ad7b6d99f3 2026-03-28 02:30:31,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:31,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49a34590270ad7b6d99f3 HTTP/1.1" 200 None 2026-03-28 02:30:31,584 - lava_callback - DEBUG - Processing node 69c49a34590270ad7b6d99f3: baseline-arm64-baylibre 2026-03-28 02:30:31,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49a34590270ad7b6d99f3 2026-03-28 02:30:31,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:31,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49a34590270ad7b6d99f3 HTTP/1.1" 200 None 2026-03-28 02:30:31,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:31,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:30:31,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:31,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49a34590270ad7b6d99f3 HTTP/1.1" 200 None 2026-03-28 02:30:31,636 - lava_callback - ERROR - Error processing callback for node 69c49a34590270ad7b6d99f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 02:30:36,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:36,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39660 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:32:02,583 - lava_callback - INFO - Received callback for node 69c73d9729015d0f95992848 INFO: 172.21.0.1:49828 - "POST /node/69c73d9729015d0f95992848 HTTP/1.0" 202 Accepted 2026-03-28 02:32:02,589 - lava_callback - INFO - Starting async job submit for node 69c73d9729015d0f95992848 2026-03-28 02:32:02,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:02,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73d9729015d0f95992848 HTTP/1.1" 200 None 2026-03-28 02:32:02,606 - lava_callback - DEBUG - Processing node 69c73d9729015d0f95992848: baseline-nfs-arm64 2026-03-28 02:32:02,608 - lava_callback - WARNING - Failed to upload log 2026-03-28 02:32:02,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:32:02,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:32:02,762 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73d9729015d0f95992848/lava-logs.yaml 2026-03-28 02:32:02,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:32:02,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:32:02,922 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73d9729015d0f95992848/lava_callback.json.gz 2026-03-28 02:32:02,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:02,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73d9729015d0f95992848 HTTP/1.1" 200 None 2026-03-28 02:32:02,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:02,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:32:02,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:02,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73d9729015d0f95992848 HTTP/1.1" 200 None 2026-03-28 02:32:02,985 - lava_callback - INFO - Completed processing callback for node 69c73d9729015d0f95992848 2026-03-28 02:32:06,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:06,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49842 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:32:11,662 - lava_callback - INFO - Received callback for node 69c49a99590270ad7b6d99fe INFO: 172.21.0.1:41204 - "POST /node/69c49a99590270ad7b6d99fe HTTP/1.0" 202 Accepted 2026-03-28 02:32:11,668 - lava_callback - INFO - Starting async job submit for node 69c49a99590270ad7b6d99fe 2026-03-28 02:32:11,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:11,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49a99590270ad7b6d99fe HTTP/1.1" 200 None 2026-03-28 02:32:11,691 - lava_callback - DEBUG - Processing node 69c49a99590270ad7b6d99fe: baseline-arm64-baylibre 2026-03-28 02:32:11,691 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49a99590270ad7b6d99fe 2026-03-28 02:32:11,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:11,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49a99590270ad7b6d99fe HTTP/1.1" 200 None 2026-03-28 02:32:11,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:11,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:32:11,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:11,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49a99590270ad7b6d99fe HTTP/1.1" 200 None 2026-03-28 02:32:11,752 - lava_callback - ERROR - Error processing callback for node 69c49a99590270ad7b6d99fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41772 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:33:47,569 - lava_callback - INFO - Received callback for node 69c49af6590270ad7b6d9a0a INFO: 172.21.0.1:41774 - "POST /node/69c49af6590270ad7b6d9a0a HTTP/1.0" 202 Accepted 2026-03-28 02:33:47,575 - lava_callback - INFO - Starting async job submit for node 69c49af6590270ad7b6d9a0a 2026-03-28 02:33:47,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:33:47,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49af6590270ad7b6d9a0a HTTP/1.1" 200 None 2026-03-28 02:33:47,591 - lava_callback - DEBUG - Processing node 69c49af6590270ad7b6d9a0a: baseline-arm64-clabbe 2026-03-28 02:33:47,592 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49af6590270ad7b6d9a0a 2026-03-28 02:33:47,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:33:47,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49af6590270ad7b6d9a0a HTTP/1.1" 200 None 2026-03-28 02:33:47,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:33:47,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:33:47,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:33:47,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49af6590270ad7b6d9a0a HTTP/1.1" 200 None 2026-03-28 02:33:47,652 - lava_callback - ERROR - Error processing callback for node 69c49af6590270ad7b6d9a0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52800 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:34:11,717 - lava_callback - INFO - Received callback for node 69c735b129015d0f959926b7 INFO: 172.21.0.1:52806 - "POST /node/69c735b129015d0f959926b7 HTTP/1.0" 202 Accepted 2026-03-28 02:34:11,765 - lava_callback - INFO - Starting async job submit for node 69c735b129015d0f959926b7 2026-03-28 02:34:11,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:34:11,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c735b129015d0f959926b7 HTTP/1.1" 200 None 2026-03-28 02:34:11,812 - lava_callback - DEBUG - Processing node 69c735b129015d0f959926b7: baseline-arm-pengutronix 2026-03-28 02:34:11,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:34:12,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:34:12,093 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c735b129015d0f959926b7/log.txt.gz 2026-03-28 02:34:12,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:34:12,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:34:12,485 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c735b129015d0f959926b7/lava-logs.yaml 2026-03-28 02:34:12,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:34:12,625 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:34:12,627 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c735b129015d0f959926b7/lava_callback.json.gz DEBUG: Setup failed for 69c735b129015d0f959926b7. Transitting job node result: incomplete -> fail 2026-03-28 02:34:12,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:34:12,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c735b129015d0f959926b7 HTTP/1.1" 200 None 2026-03-28 02:34:12,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:34:12,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 02:34:12,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:34:12,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c735b129015d0f959926b7 HTTP/1.1" 200 None 2026-03-28 02:34:12,815 - lava_callback - INFO - Completed processing callback for node 69c735b129015d0f959926b7 INFO: 172.21.0.1:52812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43276 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:34:36,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:34:36,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40328 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:35:30,814 - lava_callback - INFO - Received callback for node 69c73e3329015d0f95992863 INFO: 172.21.0.1:42910 - "POST /node/69c73e3329015d0f95992863 HTTP/1.0" 202 Accepted 2026-03-28 02:35:30,821 - lava_callback - INFO - Starting async job submit for node 69c73e3329015d0f95992863 2026-03-28 02:35:30,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:35:30,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73e3329015d0f95992863 HTTP/1.1" 200 None 2026-03-28 02:35:30,885 - lava_callback - DEBUG - Processing node 69c73e3329015d0f95992863: baseline-arm64 2026-03-28 02:35:31,030 - lava_callback - WARNING - Failed to upload log 2026-03-28 02:35:31,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:35:31,309 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:35:31,311 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73e3329015d0f95992863/lava-logs.yaml 2026-03-28 02:35:31,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:35:31,454 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:35:31,456 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73e3329015d0f95992863/lava_callback.json.gz 2026-03-28 02:35:31,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:35:31,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73e3329015d0f95992863 HTTP/1.1" 200 None 2026-03-28 02:35:31,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:35:31,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:35:31,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:35:31,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73e3329015d0f95992863 HTTP/1.1" 200 None 2026-03-28 02:35:31,616 - lava_callback - INFO - Completed processing callback for node 69c73e3329015d0f95992863 2026-03-28 02:35:36,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:35:36,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54992 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:36:07,635 - lava_callback - INFO - Received callback for node 69c49b94590270ad7b6d9a31 INFO: 172.21.0.1:55008 - "POST /node/69c49b94590270ad7b6d9a31 HTTP/1.0" 202 Accepted 2026-03-28 02:36:07,641 - lava_callback - INFO - Starting async job submit for node 69c49b94590270ad7b6d9a31 2026-03-28 02:36:07,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:07,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49b94590270ad7b6d9a31 HTTP/1.1" 200 None 2026-03-28 02:36:07,657 - lava_callback - DEBUG - Processing node 69c49b94590270ad7b6d9a31: baseline-arm64-clabbe 2026-03-28 02:36:07,657 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49b94590270ad7b6d9a31 2026-03-28 02:36:07,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:07,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49b94590270ad7b6d9a31 HTTP/1.1" 200 None 2026-03-28 02:36:07,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:07,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:36:07,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:07,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49b94590270ad7b6d9a31 HTTP/1.1" 200 None 2026-03-28 02:36:07,719 - lava_callback - ERROR - Error processing callback for node 69c49b94590270ad7b6d9a31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43590 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:36:53,429 - lava_callback - INFO - Received callback for node 69c73e8629015d0f95992871 INFO: 172.21.0.1:52124 - "POST /node/69c73e8629015d0f95992871 HTTP/1.0" 202 Accepted 2026-03-28 02:36:53,436 - lava_callback - INFO - Starting async job submit for node 69c73e8629015d0f95992871 2026-03-28 02:36:53,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:53,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73e8629015d0f95992871 HTTP/1.1" 200 None 2026-03-28 02:36:53,481 - lava_callback - DEBUG - Processing node 69c73e8629015d0f95992871: baseline-arm64 2026-03-28 02:36:53,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:36:53,822 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:36:53,824 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73e8629015d0f95992871/log.txt.gz 2026-03-28 02:36:53,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:36:54,087 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:36:54,089 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73e8629015d0f95992871/lava-logs.yaml 2026-03-28 02:36:54,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:36:54,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:36:54,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73e8629015d0f95992871/lava_callback.json.gz 2026-03-28 02:36:54,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:54,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73e8629015d0f95992871 HTTP/1.1" 200 None 2026-03-28 02:36:54,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:54,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:36:54,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:54,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73e8629015d0f95992871 HTTP/1.1" 200 None 2026-03-28 02:36:54,392 - lava_callback - INFO - Completed processing callback for node 69c73e8629015d0f95992871 INFO: 172.21.0.1:52134 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:37:06,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:37:06,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58458 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:37:47,705 - lava_callback - INFO - Received callback for node 69c49be5590270ad7b6d9a3e INFO: 172.21.0.1:58474 - "POST /node/69c49be5590270ad7b6d9a3e HTTP/1.0" 202 Accepted 2026-03-28 02:37:47,711 - lava_callback - INFO - Starting async job submit for node 69c49be5590270ad7b6d9a3e 2026-03-28 02:37:47,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:37:47,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49be5590270ad7b6d9a3e HTTP/1.1" 200 None 2026-03-28 02:37:47,727 - lava_callback - DEBUG - Processing node 69c49be5590270ad7b6d9a3e: baseline-arm64-clabbe 2026-03-28 02:37:47,727 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49be5590270ad7b6d9a3e 2026-03-28 02:37:47,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:37:47,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49be5590270ad7b6d9a3e HTTP/1.1" 200 None 2026-03-28 02:37:47,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:37:47,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:37:47,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:37:47,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49be5590270ad7b6d9a3e HTTP/1.1" 200 None 2026-03-28 02:37:47,793 - lava_callback - ERROR - Error processing callback for node 69c49be5590270ad7b6d9a3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49522 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:38:02,236 - lava_callback - INFO - Received callback for node 69c73d4429015d0f9599283a INFO: 172.21.0.1:49216 - "POST /node/69c73d4429015d0f9599283a HTTP/1.0" 202 Accepted 2026-03-28 02:38:02,244 - lava_callback - INFO - Starting async job submit for node 69c73d4429015d0f9599283a 2026-03-28 02:38:02,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:38:02,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73d4429015d0f9599283a HTTP/1.1" 200 None 2026-03-28 02:38:02,306 - lava_callback - DEBUG - Processing node 69c73d4429015d0f9599283a: baseline-arm64 2026-03-28 02:38:02,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:38:02,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:38:02,858 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73d4429015d0f9599283a/log.txt.gz 2026-03-28 02:38:03,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:38:03,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:38:03,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73d4429015d0f9599283a/lava-logs.yaml 2026-03-28 02:38:03,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:38:03,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:38:03,492 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73d4429015d0f9599283a/lava_callback.json.gz DEBUG: Setup failed for 69c73d4429015d0f9599283a. Transitting job node result: incomplete -> fail 2026-03-28 02:38:03,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:38:03,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73d4429015d0f9599283a HTTP/1.1" 200 None 2026-03-28 02:38:03,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:38:03,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:38:03,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:38:03,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73d4429015d0f9599283a HTTP/1.1" 200 None 2026-03-28 02:38:03,698 - lava_callback - INFO - Completed processing callback for node 69c73d4429015d0f9599283a 2026-03-28 02:38:06,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:38:06,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60898 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:39:07,769 - lava_callback - INFO - Received callback for node 69c49c37590270ad7b6d9a59 INFO: 172.21.0.1:60906 - "POST /node/69c49c37590270ad7b6d9a59 HTTP/1.0" 202 Accepted 2026-03-28 02:39:07,775 - lava_callback - INFO - Starting async job submit for node 69c49c37590270ad7b6d9a59 2026-03-28 02:39:07,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:07,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49c37590270ad7b6d9a59 HTTP/1.1" 200 None 2026-03-28 02:39:07,806 - lava_callback - DEBUG - Processing node 69c49c37590270ad7b6d9a59: baseline-arm64-clabbe 2026-03-28 02:39:07,807 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49c37590270ad7b6d9a59 2026-03-28 02:39:07,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:07,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49c37590270ad7b6d9a59 HTTP/1.1" 200 None 2026-03-28 02:39:07,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:07,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:39:07,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:07,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49c37590270ad7b6d9a59 HTTP/1.1" 200 None 2026-03-28 02:39:07,890 - lava_callback - ERROR - Error processing callback for node 69c49c37590270ad7b6d9a59: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59466 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:39:53,480 - lava_callback - INFO - Received callback for node 69c73eff29015d0f9599288f INFO: 172.21.0.1:59004 - "POST /node/69c73eff29015d0f9599288f HTTP/1.0" 202 Accepted 2026-03-28 02:39:53,487 - lava_callback - INFO - Starting async job submit for node 69c73eff29015d0f9599288f 2026-03-28 02:39:53,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:53,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73eff29015d0f9599288f HTTP/1.1" 200 None 2026-03-28 02:39:53,525 - lava_callback - DEBUG - Processing node 69c73eff29015d0f9599288f: baseline-arm64 2026-03-28 02:39:53,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:39:54,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:39:54,174 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73eff29015d0f9599288f/log.txt.gz 2026-03-28 02:39:54,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:39:54,637 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:39:54,639 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73eff29015d0f9599288f/lava-logs.yaml 2026-03-28 02:39:54,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:39:54,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:39:54,813 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73eff29015d0f9599288f/lava_callback.json.gz 2026-03-28 02:39:54,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:54,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73eff29015d0f9599288f HTTP/1.1" 200 None 2026-03-28 02:39:54,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:54,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:39:54,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:54,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73eff29015d0f9599288f HTTP/1.1" 200 None 2026-03-28 02:39:54,999 - lava_callback - INFO - Completed processing callback for node 69c73eff29015d0f9599288f INFO: 172.21.0.1:59018 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:40:06,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:40:06,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33900 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:41:27,821 - lava_callback - INFO - Received callback for node 69c49cd4590270ad7b6d9a87 INFO: 172.21.0.1:33914 - "POST /node/69c49cd4590270ad7b6d9a87 HTTP/1.0" 202 Accepted 2026-03-28 02:41:27,827 - lava_callback - INFO - Starting async job submit for node 69c49cd4590270ad7b6d9a87 2026-03-28 02:41:27,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:41:27,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49cd4590270ad7b6d9a87 HTTP/1.1" 200 None 2026-03-28 02:41:27,843 - lava_callback - DEBUG - Processing node 69c49cd4590270ad7b6d9a87: baseline-arm64-clabbe 2026-03-28 02:41:27,844 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49cd4590270ad7b6d9a87 2026-03-28 02:41:27,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:41:27,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49cd4590270ad7b6d9a87 HTTP/1.1" 200 None 2026-03-28 02:41:27,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:41:27,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:41:27,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:41:27,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49cd4590270ad7b6d9a87 HTTP/1.1" 200 None 2026-03-28 02:41:27,904 - lava_callback - ERROR - Error processing callback for node 69c49cd4590270ad7b6d9a87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38240 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:42:32,559 - lava_callback - INFO - Received callback for node 69c49d11590270ad7b6d9a9a INFO: 172.21.0.1:46306 - "POST /node/69c49d11590270ad7b6d9a9a HTTP/1.0" 202 Accepted 2026-03-28 02:42:32,564 - lava_callback - INFO - Starting async job submit for node 69c49d11590270ad7b6d9a9a 2026-03-28 02:42:32,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:42:32,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d11590270ad7b6d9a9a HTTP/1.1" 200 None 2026-03-28 02:42:32,580 - lava_callback - DEBUG - Processing node 69c49d11590270ad7b6d9a9a: baseline-arm-baylibre 2026-03-28 02:42:32,580 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49d11590270ad7b6d9a9a 2026-03-28 02:42:32,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:42:32,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d11590270ad7b6d9a9a HTTP/1.1" 200 None 2026-03-28 02:42:32,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:42:32,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:42:32,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:42:32,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49d11590270ad7b6d9a9a HTTP/1.1" 200 None 2026-03-28 02:42:32,639 - lava_callback - ERROR - Error processing callback for node 69c49d11590270ad7b6d9a9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45390 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:43:27,908 - lava_callback - INFO - Received callback for node 69c49d4d590270ad7b6d9aad INFO: 172.21.0.1:45400 - "POST /node/69c49d4d590270ad7b6d9aad HTTP/1.0" 202 Accepted 2026-03-28 02:43:27,913 - lava_callback - INFO - Starting async job submit for node 69c49d4d590270ad7b6d9aad 2026-03-28 02:43:27,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:43:27,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d4d590270ad7b6d9aad HTTP/1.1" 200 None 2026-03-28 02:43:27,930 - lava_callback - DEBUG - Processing node 69c49d4d590270ad7b6d9aad: baseline-arm64-clabbe 2026-03-28 02:43:27,930 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49d4d590270ad7b6d9aad 2026-03-28 02:43:27,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:43:27,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d4d590270ad7b6d9aad HTTP/1.1" 200 None 2026-03-28 02:43:27,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:43:27,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:43:27,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:43:27,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49d4d590270ad7b6d9aad HTTP/1.1" 200 None 2026-03-28 02:43:27,995 - lava_callback - ERROR - Error processing callback for node 69c49d4d590270ad7b6d9aad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34928 - "GET /node/67ff8032c0534c95c39deb1e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:41398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53326 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:44:32,356 - lava_callback - INFO - Received callback for node 69c49d8a590270ad7b6d9aba INFO: 172.21.0.1:52698 - "POST /node/69c49d8a590270ad7b6d9aba HTTP/1.0" 202 Accepted 2026-03-28 02:44:32,361 - lava_callback - INFO - Starting async job submit for node 69c49d8a590270ad7b6d9aba 2026-03-28 02:44:32,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:44:32,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d8a590270ad7b6d9aba HTTP/1.1" 200 None 2026-03-28 02:44:32,394 - lava_callback - DEBUG - Processing node 69c49d8a590270ad7b6d9aba: baseline-arm-baylibre 2026-03-28 02:44:32,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49d8a590270ad7b6d9aba 2026-03-28 02:44:32,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:44:32,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d8a590270ad7b6d9aba HTTP/1.1" 200 None 2026-03-28 02:44:32,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:44:32,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:44:32,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:44:32,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49d8a590270ad7b6d9aba HTTP/1.1" 200 None 2026-03-28 02:44:32,459 - lava_callback - ERROR - Error processing callback for node 69c49d8a590270ad7b6d9aba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41064 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:45:12,375 - lava_callback - INFO - Received callback for node 69c49db1590270ad7b6d9acb INFO: 172.21.0.1:53936 - "POST /node/69c49db1590270ad7b6d9acb HTTP/1.0" 202 Accepted 2026-03-28 02:45:12,380 - lava_callback - INFO - Starting async job submit for node 69c49db1590270ad7b6d9acb 2026-03-28 02:45:12,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:12,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49db1590270ad7b6d9acb HTTP/1.1" 200 None 2026-03-28 02:45:12,396 - lava_callback - DEBUG - Processing node 69c49db1590270ad7b6d9acb: baseline-arm-baylibre 2026-03-28 02:45:12,396 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49db1590270ad7b6d9acb 2026-03-28 02:45:12,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:12,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49db1590270ad7b6d9acb HTTP/1.1" 200 None 2026-03-28 02:45:12,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:12,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:45:12,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:12,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49db1590270ad7b6d9acb HTTP/1.1" 200 None 2026-03-28 02:45:12,461 - lava_callback - ERROR - Error processing callback for node 69c49db1590270ad7b6d9acb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54048 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:45:52,407 - lava_callback - INFO - Received callback for node 69c49dd7590270ad7b6d9ad4 INFO: 172.21.0.1:57330 - "POST /node/69c49dd7590270ad7b6d9ad4 HTTP/1.0" 202 Accepted 2026-03-28 02:45:52,413 - lava_callback - INFO - Starting async job submit for node 69c49dd7590270ad7b6d9ad4 2026-03-28 02:45:52,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:52,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49dd7590270ad7b6d9ad4 HTTP/1.1" 200 None 2026-03-28 02:45:52,429 - lava_callback - DEBUG - Processing node 69c49dd7590270ad7b6d9ad4: baseline-arm-baylibre 2026-03-28 02:45:52,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49dd7590270ad7b6d9ad4 2026-03-28 02:45:52,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:52,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49dd7590270ad7b6d9ad4 HTTP/1.1" 200 None 2026-03-28 02:45:52,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:52,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:45:52,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:52,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49dd7590270ad7b6d9ad4 HTTP/1.1" 200 None 2026-03-28 02:45:52,494 - lava_callback - ERROR - Error processing callback for node 69c49dd7590270ad7b6d9ad4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:46:32,430 - lava_callback - INFO - Received callback for node 69c49dff590270ad7b6d9ae1 INFO: 172.21.0.1:57388 - "POST /node/69c49dff590270ad7b6d9ae1 HTTP/1.0" 202 Accepted 2026-03-28 02:46:32,436 - lava_callback - INFO - Starting async job submit for node 69c49dff590270ad7b6d9ae1 2026-03-28 02:46:32,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:32,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49dff590270ad7b6d9ae1 HTTP/1.1" 200 None 2026-03-28 02:46:32,453 - lava_callback - DEBUG - Processing node 69c49dff590270ad7b6d9ae1: baseline-arm-baylibre 2026-03-28 02:46:32,453 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49dff590270ad7b6d9ae1 2026-03-28 02:46:32,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:32,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49dff590270ad7b6d9ae1 HTTP/1.1" 200 None 2026-03-28 02:46:32,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:32,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:46:32,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:32,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49dff590270ad7b6d9ae1 HTTP/1.1" 200 None 2026-03-28 02:46:32,526 - lava_callback - ERROR - Error processing callback for node 69c49dff590270ad7b6d9ae1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57398 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:46:37,307 - lava_callback - INFO - Received callback for node 69c73fbd29015d0f959928c8 INFO: 172.21.0.1:57406 - "POST /node/69c73fbd29015d0f959928c8 HTTP/1.0" 202 Accepted 2026-03-28 02:46:37,315 - lava_callback - INFO - Starting async job submit for node 69c73fbd29015d0f959928c8 2026-03-28 02:46:37,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:37,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73fbd29015d0f959928c8 HTTP/1.1" 200 None 2026-03-28 02:46:37,372 - lava_callback - DEBUG - Processing node 69c73fbd29015d0f959928c8: baseline-arm64 2026-03-28 02:46:37,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:46:37,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:46:37,911 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73fbd29015d0f959928c8/log.txt.gz 2026-03-28 02:46:38,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:46:38,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:46:38,328 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73fbd29015d0f959928c8/lava-logs.yaml 2026-03-28 02:46:38,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:46:38,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:46:38,500 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73fbd29015d0f959928c8/lava_callback.json.gz DEBUG: Setup failed for 69c73fbd29015d0f959928c8. Transitting job node result: incomplete -> fail 2026-03-28 02:46:38,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:38,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73fbd29015d0f959928c8 HTTP/1.1" 200 None 2026-03-28 02:46:38,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:38,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:46:38,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:38,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73fbd29015d0f959928c8 HTTP/1.1" 200 None 2026-03-28 02:46:38,724 - lava_callback - INFO - Completed processing callback for node 69c73fbd29015d0f959928c8 INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34934 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:47:06,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:47:06,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33178 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:47:12,560 - lava_callback - INFO - Received callback for node 69c49e26590270ad7b6d9af0 INFO: 172.21.0.1:36764 - "POST /node/69c49e26590270ad7b6d9af0 HTTP/1.0" 202 Accepted 2026-03-28 02:47:12,566 - lava_callback - INFO - Starting async job submit for node 69c49e26590270ad7b6d9af0 2026-03-28 02:47:12,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:47:12,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49e26590270ad7b6d9af0 HTTP/1.1" 200 None 2026-03-28 02:47:12,582 - lava_callback - DEBUG - Processing node 69c49e26590270ad7b6d9af0: baseline-arm-baylibre 2026-03-28 02:47:12,582 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49e26590270ad7b6d9af0 2026-03-28 02:47:12,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:47:12,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49e26590270ad7b6d9af0 HTTP/1.1" 200 None 2026-03-28 02:47:12,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:47:12,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:47:12,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:47:12,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49e26590270ad7b6d9af0 HTTP/1.1" 200 None 2026-03-28 02:47:12,662 - lava_callback - ERROR - Error processing callback for node 69c49e26590270ad7b6d9af0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:49:08,237 - lava_callback - INFO - Received callback for node 69c49e9a590270ad7b6d9b13 INFO: 172.21.0.1:33332 - "POST /node/69c49e9a590270ad7b6d9b13 HTTP/1.0" 202 Accepted 2026-03-28 02:49:08,243 - lava_callback - INFO - Starting async job submit for node 69c49e9a590270ad7b6d9b13 2026-03-28 02:49:08,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:08,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49e9a590270ad7b6d9b13 HTTP/1.1" 200 None 2026-03-28 02:49:08,263 - lava_callback - DEBUG - Processing node 69c49e9a590270ad7b6d9b13: baseline-arm-clabbe 2026-03-28 02:49:08,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49e9a590270ad7b6d9b13 2026-03-28 02:49:08,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:08,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49e9a590270ad7b6d9b13 HTTP/1.1" 200 None 2026-03-28 02:49:08,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:08,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:49:08,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:08,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49e9a590270ad7b6d9b13 HTTP/1.1" 200 None 2026-03-28 02:49:08,333 - lava_callback - ERROR - Error processing callback for node 69c49e9a590270ad7b6d9b13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56872 - "GET /node/6992691098a8bdb31c392310%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:36808 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:49:48,227 - lava_callback - INFO - Received callback for node 69c49ec2590270ad7b6d9b1e INFO: 172.21.0.1:36816 - "POST /node/69c49ec2590270ad7b6d9b1e HTTP/1.0" 202 Accepted 2026-03-28 02:49:48,233 - lava_callback - INFO - Starting async job submit for node 69c49ec2590270ad7b6d9b1e 2026-03-28 02:49:48,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:48,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49ec2590270ad7b6d9b1e HTTP/1.1" 200 None 2026-03-28 02:49:48,249 - lava_callback - DEBUG - Processing node 69c49ec2590270ad7b6d9b1e: baseline-arm-clabbe 2026-03-28 02:49:48,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49ec2590270ad7b6d9b1e 2026-03-28 02:49:48,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:48,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49ec2590270ad7b6d9b1e HTTP/1.1" 200 None 2026-03-28 02:49:48,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:48,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:49:48,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:48,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49ec2590270ad7b6d9b1e HTTP/1.1" 200 None 2026-03-28 02:49:48,310 - lava_callback - ERROR - Error processing callback for node 69c49ec2590270ad7b6d9b1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50088 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:50:28,278 - lava_callback - INFO - Received callback for node 69c49eea590270ad7b6d9b28 INFO: 172.21.0.1:50104 - "POST /node/69c49eea590270ad7b6d9b28 HTTP/1.0" 202 Accepted 2026-03-28 02:50:28,283 - lava_callback - INFO - Starting async job submit for node 69c49eea590270ad7b6d9b28 2026-03-28 02:50:28,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:50:28,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49eea590270ad7b6d9b28 HTTP/1.1" 200 None 2026-03-28 02:50:28,299 - lava_callback - DEBUG - Processing node 69c49eea590270ad7b6d9b28: baseline-arm-clabbe 2026-03-28 02:50:28,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49eea590270ad7b6d9b28 2026-03-28 02:50:28,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:50:28,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49eea590270ad7b6d9b28 HTTP/1.1" 200 None 2026-03-28 02:50:28,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:50:28,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:50:28,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:50:28,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49eea590270ad7b6d9b28 HTTP/1.1" 200 None 2026-03-28 02:50:28,361 - lava_callback - ERROR - Error processing callback for node 69c49eea590270ad7b6d9b28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56142 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:51:08,291 - lava_callback - INFO - Received callback for node 69c49f12590270ad7b6d9b31 INFO: 172.21.0.1:56158 - "POST /node/69c49f12590270ad7b6d9b31 HTTP/1.0" 202 Accepted 2026-03-28 02:51:08,297 - lava_callback - INFO - Starting async job submit for node 69c49f12590270ad7b6d9b31 2026-03-28 02:51:08,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:08,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49f12590270ad7b6d9b31 HTTP/1.1" 200 None 2026-03-28 02:51:08,313 - lava_callback - DEBUG - Processing node 69c49f12590270ad7b6d9b31: baseline-arm-clabbe 2026-03-28 02:51:08,314 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49f12590270ad7b6d9b31 2026-03-28 02:51:08,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:08,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49f12590270ad7b6d9b31 HTTP/1.1" 200 None 2026-03-28 02:51:08,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:08,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:51:08,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:08,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49f12590270ad7b6d9b31 HTTP/1.1" 200 None 2026-03-28 02:51:08,375 - lava_callback - ERROR - Error processing callback for node 69c49f12590270ad7b6d9b31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48326 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:51:48,351 - lava_callback - INFO - Received callback for node 69c49f3f590270ad7b6d9b3c INFO: 172.21.0.1:48328 - "POST /node/69c49f3f590270ad7b6d9b3c HTTP/1.0" 202 Accepted 2026-03-28 02:51:48,357 - lava_callback - INFO - Starting async job submit for node 69c49f3f590270ad7b6d9b3c 2026-03-28 02:51:48,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:48,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49f3f590270ad7b6d9b3c HTTP/1.1" 200 None 2026-03-28 02:51:48,392 - lava_callback - DEBUG - Processing node 69c49f3f590270ad7b6d9b3c: baseline-arm-clabbe 2026-03-28 02:51:48,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49f3f590270ad7b6d9b3c 2026-03-28 02:51:48,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:48,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49f3f590270ad7b6d9b3c HTTP/1.1" 200 None 2026-03-28 02:51:48,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:48,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:51:48,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:48,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49f3f590270ad7b6d9b3c HTTP/1.1" 200 None 2026-03-28 02:51:48,444 - lava_callback - ERROR - Error processing callback for node 69c49f3f590270ad7b6d9b3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45884 - "GET /node/699268a098a8bdb31c392252%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:45894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42824 - "GET /node/69bbb7b4ec3f435d20d51c72%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60584 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:55:25,292 - lava_callback - INFO - Received callback for node 69c73ed429015d0f9599287c INFO: 172.21.0.1:54484 - "POST /node/69c73ed429015d0f9599287c HTTP/1.0" 202 Accepted 2026-03-28 02:55:25,308 - lava_callback - INFO - Starting async job submit for node 69c73ed429015d0f9599287c 2026-03-28 02:55:25,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:55:25,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ed429015d0f9599287c HTTP/1.1" 200 None 2026-03-28 02:55:25,399 - lava_callback - DEBUG - Processing node 69c73ed429015d0f9599287c: baseline-arm 2026-03-28 02:55:25,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:55:26,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:55:26,146 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c73ed429015d0f9599287c/log.txt.gz 2026-03-28 02:55:26,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:54496 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:55:26,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:55:26,756 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c73ed429015d0f9599287c/lava-logs.yaml 2026-03-28 02:55:26,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:55:27,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:55:27,113 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c73ed429015d0f9599287c/lava_callback.json.gz DEBUG: Setup failed for 69c73ed429015d0f9599287c. Transitting job node result: incomplete -> fail 2026-03-28 02:55:27,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:55:27,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ed429015d0f9599287c HTTP/1.1" 200 None 2026-03-28 02:55:27,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:55:27,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 02:55:27,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:55:27,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73ed429015d0f9599287c HTTP/1.1" 200 None 2026-03-28 02:55:27,418 - lava_callback - INFO - Completed processing callback for node 69c73ed429015d0f9599287c 2026-03-28 02:55:36,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:55:36,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33446 - "GET /node/68fad18ca011bb5928ee75a5%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-28 02:59:52,573 - lava_callback - INFO - Received callback for node 69c73f9d29015d0f959928b6 INFO: 172.21.0.1:54194 - "POST /node/69c73f9d29015d0f959928b6 HTTP/1.0" 202 Accepted 2026-03-28 02:59:52,621 - lava_callback - INFO - Starting async job submit for node 69c73f9d29015d0f959928b6 2026-03-28 02:59:52,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:59:52,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73f9d29015d0f959928b6 HTTP/1.1" 200 None 2026-03-28 02:59:52,672 - lava_callback - DEBUG - Processing node 69c73f9d29015d0f959928b6: baseline-arm-pengutronix 2026-03-28 02:59:52,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:59:53,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:59:53,074 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c73f9d29015d0f959928b6/log.txt.gz 2026-03-28 02:59:53,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:59:53,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:59:53,338 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c73f9d29015d0f959928b6/lava-logs.yaml 2026-03-28 02:59:53,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:59:53,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:59:53,628 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c73f9d29015d0f959928b6/lava_callback.json.gz DEBUG: Setup failed for 69c73f9d29015d0f959928b6. Transitting job node result: incomplete -> fail 2026-03-28 02:59:53,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:59:53,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73f9d29015d0f959928b6 HTTP/1.1" 200 None 2026-03-28 02:59:53,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:59:53,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 02:59:53,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:59:53,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73f9d29015d0f959928b6 HTTP/1.1" 200 None 2026-03-28 02:59:53,785 - lava_callback - INFO - Completed processing callback for node 69c73f9d29015d0f959928b6 INFO: 172.21.0.1:54198 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:00:06,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:00:06,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET /node/693e742a3c8f1be6d3cef710%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47898 - "GET /node/698e0aef952a14bf500e2400%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59858 - "GET /node/69876b22e31b58f834f197a8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:59870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52292 - "GET /node/69bbb7b8ec3f435d20d51c7f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:52302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37506 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:07:20,322 - lava_callback - INFO - Received callback for node 69c73ca929015d0f95992821 INFO: 172.21.0.1:46852 - "POST /node/69c73ca929015d0f95992821 HTTP/1.0" 202 Accepted 2026-03-28 03:07:20,331 - lava_callback - INFO - Starting async job submit for node 69c73ca929015d0f95992821 2026-03-28 03:07:20,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:07:20,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ca929015d0f95992821 HTTP/1.1" 200 None 2026-03-28 03:07:20,391 - lava_callback - DEBUG - Processing node 69c73ca929015d0f95992821: baseline-nfs-arm64 2026-03-28 03:07:21,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:07:21,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:07:21,707 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73ca929015d0f95992821/log.txt.gz 2026-03-28 03:07:22,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:07:22,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:07:22,623 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73ca929015d0f95992821/lava-logs.yaml 2026-03-28 03:07:22,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:07:22,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:07:22,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73ca929015d0f95992821/lava_callback.json.gz DEBUG: Setup failed for 69c73ca929015d0f95992821. Transitting job node result: incomplete -> fail 2026-03-28 03:07:22,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:07:22,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ca929015d0f95992821 HTTP/1.1" 200 None 2026-03-28 03:07:22,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:07:23,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:07:23,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:07:23,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73ca929015d0f95992821 HTTP/1.1" 200 None 2026-03-28 03:07:23,144 - lava_callback - INFO - Completed processing callback for node 69c73ca929015d0f95992821 INFO: 172.21.0.1:46856 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:07:36,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:07:36,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43144 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:08:34,422 - lava_callback - INFO - Received callback for node 69c73ef929015d0f9599288b INFO: 172.21.0.1:47246 - "POST /node/69c73ef929015d0f9599288b HTTP/1.0" 202 Accepted 2026-03-28 03:08:34,429 - lava_callback - INFO - Starting async job submit for node 69c73ef929015d0f9599288b 2026-03-28 03:08:34,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:08:34,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ef929015d0f9599288b HTTP/1.1" 200 None 2026-03-28 03:08:34,508 - lava_callback - DEBUG - Processing node 69c73ef929015d0f9599288b: ltp-smoke 2026-03-28 03:08:35,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:08:35,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:08:35,348 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c73ef929015d0f9599288b/log.txt.gz 2026-03-28 03:08:35,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:08:35,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:08:35,895 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c73ef929015d0f9599288b/lava-logs.yaml 2026-03-28 03:08:35,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:08:36,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:08:36,060 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c73ef929015d0f9599288b/lava_callback.json.gz 2026-03-28 03:08:36,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:08:36,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ef929015d0f9599288b HTTP/1.1" 200 None 2026-03-28 03:08:36,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:08:36,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4229015d0f9598f742 HTTP/1.1" 200 None 2026-03-28 03:08:36,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:08:36,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73ef929015d0f9599288b HTTP/1.1" 200 None 2026-03-28 03:08:36,384 - lava_callback - INFO - Completed processing callback for node 69c73ef929015d0f9599288b 2026-03-28 03:08:36,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:47248 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:08:36,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59382 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:11:03,193 - lava_callback - INFO - Received callback for node 69c72d0e29015d0f959923bc INFO: 172.21.0.1:41980 - "POST /node/69c72d0e29015d0f959923bc HTTP/1.0" 202 Accepted 2026-03-28 03:11:03,199 - lava_callback - INFO - Starting async job submit for node 69c72d0e29015d0f959923bc 2026-03-28 03:11:03,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:11:03,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0e29015d0f959923bc HTTP/1.1" 200 None 2026-03-28 03:11:03,261 - lava_callback - DEBUG - Processing node 69c72d0e29015d0f959923bc: baseline-arm64-broonie 2026-03-28 03:11:03,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:11:03,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:11:03,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0e29015d0f959923bc/log.txt.gz 2026-03-28 03:11:03,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:11:04,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:11:04,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0e29015d0f959923bc/lava-logs.yaml 2026-03-28 03:11:04,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:11:04,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:11:04,206 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0e29015d0f959923bc/lava_callback.json.gz 2026-03-28 03:11:04,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:11:04,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0e29015d0f959923bc HTTP/1.1" 200 None 2026-03-28 03:11:04,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:11:04,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:11:04,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:11:04,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0e29015d0f959923bc HTTP/1.1" 200 None 2026-03-28 03:11:04,322 - lava_callback - INFO - Completed processing callback for node 69c72d0e29015d0f959923bc INFO: 172.21.0.1:41996 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:11:06,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:11:06,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54564 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:19:33,540 - lava_callback - INFO - Received callback for node 69c7443729015d0f959929d3 INFO: 172.21.0.1:51554 - "POST /node/69c7443729015d0f959929d3 HTTP/1.0" 202 Accepted 2026-03-28 03:19:33,589 - lava_callback - INFO - Starting async job submit for node 69c7443729015d0f959929d3 2026-03-28 03:19:33,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:19:33,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7443729015d0f959929d3 HTTP/1.1" 200 None 2026-03-28 03:19:33,650 - lava_callback - DEBUG - Processing node 69c7443729015d0f959929d3: baseline-arm-pengutronix 2026-03-28 03:19:33,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:19:33,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:19:33,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7443729015d0f959929d3/log.txt.gz 2026-03-28 03:19:33,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:19:34,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:19:34,182 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7443729015d0f959929d3/lava-logs.yaml 2026-03-28 03:19:34,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:19:34,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:19:34,326 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7443729015d0f959929d3/lava_callback.json.gz DEBUG: Setup failed for 69c7443729015d0f959929d3. Transitting job node result: incomplete -> fail 2026-03-28 03:19:34,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:19:34,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7443729015d0f959929d3 HTTP/1.1" 200 None 2026-03-28 03:19:34,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:19:34,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 03:19:34,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:19:34,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7443729015d0f959929d3 HTTP/1.1" 200 None 2026-03-28 03:19:34,503 - lava_callback - INFO - Completed processing callback for node 69c7443729015d0f959929d3 INFO: 172.21.0.1:51558 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:19:36,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:19:36,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33186 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:21:14,442 - lava_callback - INFO - Received callback for node 69c4a623590270ad7b6d9bc6 INFO: 172.21.0.1:40270 - "POST /node/69c4a623590270ad7b6d9bc6 HTTP/1.0" 202 Accepted 2026-03-28 03:21:14,449 - lava_callback - INFO - Starting async job submit for node 69c4a623590270ad7b6d9bc6 2026-03-28 03:21:14,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:21:14,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a623590270ad7b6d9bc6 HTTP/1.1" 200 None 2026-03-28 03:21:14,478 - lava_callback - DEBUG - Processing node 69c4a623590270ad7b6d9bc6: baseline-arm64-baylibre 2026-03-28 03:21:14,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a623590270ad7b6d9bc6 2026-03-28 03:21:14,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:21:14,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a623590270ad7b6d9bc6 HTTP/1.1" 200 None 2026-03-28 03:21:14,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:21:14,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:21:14,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:21:14,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a623590270ad7b6d9bc6 HTTP/1.1" 200 None 2026-03-28 03:21:14,556 - lava_callback - ERROR - Error processing callback for node 69c4a623590270ad7b6d9bc6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38222 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:24:29,781 - lava_callback - INFO - Received callback for node 69c4a6db590270ad7b6d9bdd INFO: 172.21.0.1:44890 - "POST /node/69c4a6db590270ad7b6d9bdd HTTP/1.0" 202 Accepted 2026-03-28 03:24:29,786 - lava_callback - INFO - Starting async job submit for node 69c4a6db590270ad7b6d9bdd 2026-03-28 03:24:29,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:24:29,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a6db590270ad7b6d9bdd HTTP/1.1" 200 None 2026-03-28 03:24:29,825 - lava_callback - DEBUG - Processing node 69c4a6db590270ad7b6d9bdd: baseline-arm64-clabbe 2026-03-28 03:24:29,826 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a6db590270ad7b6d9bdd 2026-03-28 03:24:29,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:24:29,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a6db590270ad7b6d9bdd HTTP/1.1" 200 None 2026-03-28 03:24:29,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:24:29,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:24:29,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:24:29,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a6db590270ad7b6d9bdd HTTP/1.1" 200 None 2026-03-28 03:24:29,885 - lava_callback - ERROR - Error processing callback for node 69c4a6db590270ad7b6d9bdd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57854 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:25:48,648 - lava_callback - INFO - Received callback for node 69c72cd629015d0f959922ee INFO: 172.21.0.1:57856 - "POST /node/69c72cd629015d0f959922ee HTTP/1.0" 202 Accepted 2026-03-28 03:25:48,654 - lava_callback - INFO - Starting async job submit for node 69c72cd629015d0f959922ee 2026-03-28 03:25:48,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:25:48,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd629015d0f959922ee HTTP/1.1" 200 None 2026-03-28 03:25:48,688 - lava_callback - DEBUG - Processing node 69c72cd629015d0f959922ee: baseline-arm64 2026-03-28 03:25:49,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:25:49,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:25:49,211 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd629015d0f959922ee/log.txt.gz 2026-03-28 03:25:49,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:25:49,778 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:25:49,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd629015d0f959922ee/lava-logs.yaml 2026-03-28 03:25:49,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:25:49,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:25:49,957 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd629015d0f959922ee/lava_callback.json.gz 2026-03-28 03:25:49,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:25:49,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd629015d0f959922ee HTTP/1.1" 200 None 2026-03-28 03:25:49,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:25:50,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:25:50,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:25:50,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd629015d0f959922ee HTTP/1.1" 200 None 2026-03-28 03:25:50,032 - lava_callback - INFO - Completed processing callback for node 69c72cd629015d0f959922ee INFO: 172.21.0.1:33716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34142 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:26:06,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:26:06,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42320 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:26:54,575 - lava_callback - INFO - Received callback for node 69c72ce729015d0f95992315 INFO: 172.21.0.1:50066 - "POST /node/69c72ce729015d0f95992315 HTTP/1.0" 202 Accepted 2026-03-28 03:26:54,581 - lava_callback - INFO - Starting async job submit for node 69c72ce729015d0f95992315 2026-03-28 03:26:54,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:26:54,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce729015d0f95992315 HTTP/1.1" 200 None 2026-03-28 03:26:54,617 - lava_callback - DEBUG - Processing node 69c72ce729015d0f95992315: baseline-nfs-arm64 2026-03-28 03:26:55,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:26:55,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:26:55,209 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce729015d0f95992315/log.txt.gz 2026-03-28 03:26:55,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:26:55,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:26:55,730 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce729015d0f95992315/lava-logs.yaml 2026-03-28 03:26:55,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:26:55,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:26:55,911 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce729015d0f95992315/lava_callback.json.gz 2026-03-28 03:26:55,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:26:55,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce729015d0f95992315 HTTP/1.1" 200 None 2026-03-28 03:26:55,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:26:55,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:26:55,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:26:56,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce729015d0f95992315 HTTP/1.1" 200 None 2026-03-28 03:26:56,006 - lava_callback - INFO - Completed processing callback for node 69c72ce729015d0f95992315 INFO: 172.21.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59196 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:27:06,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:06,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43282 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:27:18,569 - lava_callback - INFO - Received callback for node 69c745f429015d0f959929f3 INFO: 172.21.0.1:43286 - "POST /node/69c745f429015d0f959929f3 HTTP/1.0" 202 Accepted 2026-03-28 03:27:18,577 - lava_callback - INFO - Starting async job submit for node 69c745f429015d0f959929f3 2026-03-28 03:27:18,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:18,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c745f429015d0f959929f3 HTTP/1.1" 200 None 2026-03-28 03:27:18,637 - lava_callback - DEBUG - Processing node 69c745f429015d0f959929f3: baseline-nfs-arm64 2026-03-28 03:27:19,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:27:19,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:27:19,949 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c745f429015d0f959929f3/log.txt.gz 2026-03-28 03:27:20,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:27:20,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:27:20,829 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c745f429015d0f959929f3/lava-logs.yaml 2026-03-28 03:27:20,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:27:21,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:27:21,055 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c745f429015d0f959929f3/lava_callback.json.gz DEBUG: Setup failed for 69c745f429015d0f959929f3. Transitting job node result: incomplete -> fail 2026-03-28 03:27:21,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:21,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c745f429015d0f959929f3 HTTP/1.1" 200 None 2026-03-28 03:27:21,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:21,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:27:21,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:21,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c745f429015d0f959929f3 HTTP/1.1" 200 None 2026-03-28 03:27:21,301 - lava_callback - INFO - Completed processing callback for node 69c745f429015d0f959929f3 INFO: 172.21.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49088 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:27:36,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:36,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58852 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:28:29,900 - lava_callback - INFO - Received callback for node 69c4a7cb590270ad7b6d9bf1 INFO: 172.21.0.1:43742 - "POST /node/69c4a7cb590270ad7b6d9bf1 HTTP/1.0" 202 Accepted 2026-03-28 03:28:29,906 - lava_callback - INFO - Starting async job submit for node 69c4a7cb590270ad7b6d9bf1 2026-03-28 03:28:29,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:28:29,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a7cb590270ad7b6d9bf1 HTTP/1.1" 200 None 2026-03-28 03:28:29,922 - lava_callback - DEBUG - Processing node 69c4a7cb590270ad7b6d9bf1: baseline-arm64-clabbe 2026-03-28 03:28:29,922 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a7cb590270ad7b6d9bf1 2026-03-28 03:28:29,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:28:29,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a7cb590270ad7b6d9bf1 HTTP/1.1" 200 None 2026-03-28 03:28:29,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:28:29,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:28:29,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:28:29,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a7cb590270ad7b6d9bf1 HTTP/1.1" 200 None 2026-03-28 03:28:29,974 - lava_callback - ERROR - Error processing callback for node 69c4a7cb590270ad7b6d9bf1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46306 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:30:30,017 - lava_callback - INFO - Received callback for node 69c4a843590270ad7b6d9bff INFO: 172.21.0.1:37500 - "POST /node/69c4a843590270ad7b6d9bff HTTP/1.0" 202 Accepted 2026-03-28 03:30:30,023 - lava_callback - INFO - Starting async job submit for node 69c4a843590270ad7b6d9bff 2026-03-28 03:30:30,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:30:30,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a843590270ad7b6d9bff HTTP/1.1" 200 None 2026-03-28 03:30:30,039 - lava_callback - DEBUG - Processing node 69c4a843590270ad7b6d9bff: baseline-arm64-clabbe 2026-03-28 03:30:30,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a843590270ad7b6d9bff 2026-03-28 03:30:30,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:30:30,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a843590270ad7b6d9bff HTTP/1.1" 200 None 2026-03-28 03:30:30,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:30:30,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:30:30,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:30:30,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a843590270ad7b6d9bff HTTP/1.1" 200 None 2026-03-28 03:30:30,104 - lava_callback - ERROR - Error processing callback for node 69c4a843590270ad7b6d9bff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34884 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:31:50,039 - lava_callback - INFO - Received callback for node 69c4a895590270ad7b6d9c0b INFO: 172.21.0.1:35138 - "POST /node/69c4a895590270ad7b6d9c0b HTTP/1.0" 202 Accepted 2026-03-28 03:31:50,045 - lava_callback - INFO - Starting async job submit for node 69c4a895590270ad7b6d9c0b 2026-03-28 03:31:50,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:31:50,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a895590270ad7b6d9c0b HTTP/1.1" 200 None 2026-03-28 03:31:50,080 - lava_callback - DEBUG - Processing node 69c4a895590270ad7b6d9c0b: baseline-arm64-clabbe 2026-03-28 03:31:50,080 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a895590270ad7b6d9c0b 2026-03-28 03:31:50,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:31:50,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a895590270ad7b6d9c0b HTTP/1.1" 200 None 2026-03-28 03:31:50,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:31:50,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:31:50,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:31:50,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a895590270ad7b6d9c0b HTTP/1.1" 200 None 2026-03-28 03:31:50,158 - lava_callback - ERROR - Error processing callback for node 69c4a895590270ad7b6d9c0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47582 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:34:30,362 - lava_callback - INFO - Received callback for node 69c4a934590270ad7b6d9c1a INFO: 172.21.0.1:48792 - "POST /node/69c4a934590270ad7b6d9c1a HTTP/1.0" 202 Accepted 2026-03-28 03:34:30,367 - lava_callback - INFO - Starting async job submit for node 69c4a934590270ad7b6d9c1a 2026-03-28 03:34:30,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:34:30,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a934590270ad7b6d9c1a HTTP/1.1" 200 None 2026-03-28 03:34:30,383 - lava_callback - DEBUG - Processing node 69c4a934590270ad7b6d9c1a: baseline-arm64-clabbe 2026-03-28 03:34:30,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a934590270ad7b6d9c1a 2026-03-28 03:34:30,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:34:30,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a934590270ad7b6d9c1a HTTP/1.1" 200 None 2026-03-28 03:34:30,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:34:30,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:34:30,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:34:30,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a934590270ad7b6d9c1a HTTP/1.1" 200 None 2026-03-28 03:34:30,442 - lava_callback - ERROR - Error processing callback for node 69c4a934590270ad7b6d9c1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58646 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:37:30,300 - lava_callback - INFO - Received callback for node 69c4a9e9590270ad7b6d9c29 INFO: 172.21.0.1:55402 - "POST /node/69c4a9e9590270ad7b6d9c29 HTTP/1.0" 202 Accepted 2026-03-28 03:37:30,305 - lava_callback - INFO - Starting async job submit for node 69c4a9e9590270ad7b6d9c29 2026-03-28 03:37:30,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:37:30,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a9e9590270ad7b6d9c29 HTTP/1.1" 200 None 2026-03-28 03:37:30,321 - lava_callback - DEBUG - Processing node 69c4a9e9590270ad7b6d9c29: baseline-arm64-clabbe 2026-03-28 03:37:30,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a9e9590270ad7b6d9c29 2026-03-28 03:37:30,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:37:30,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a9e9590270ad7b6d9c29 HTTP/1.1" 200 None 2026-03-28 03:37:30,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:37:30,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:37:30,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:37:30,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a9e9590270ad7b6d9c29 HTTP/1.1" 200 None 2026-03-28 03:37:30,558 - lava_callback - ERROR - Error processing callback for node 69c4a9e9590270ad7b6d9c29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55814 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:48:23,975 - lava_callback - INFO - Received callback for node 69c74ad529015d0f95992a84 INFO: 172.21.0.1:43396 - "POST /node/69c74ad529015d0f95992a84 HTTP/1.0" 202 Accepted 2026-03-28 03:48:23,983 - lava_callback - INFO - Starting async job submit for node 69c74ad529015d0f95992a84 2026-03-28 03:48:24,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:48:24,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c74ad529015d0f95992a84 HTTP/1.1" 200 None 2026-03-28 03:48:24,047 - lava_callback - DEBUG - Processing node 69c74ad529015d0f95992a84: baseline-nfs-arm64 2026-03-28 03:48:25,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:48:25,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:48:25,518 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c74ad529015d0f95992a84/log.txt.gz 2026-03-28 03:48:26,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:48:26,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:48:26,447 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c74ad529015d0f95992a84/lava-logs.yaml 2026-03-28 03:48:26,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:43408 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:48:26,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:48:26,650 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c74ad529015d0f95992a84/lava_callback.json.gz DEBUG: Setup failed for 69c74ad529015d0f95992a84. Transitting job node result: incomplete -> fail 2026-03-28 03:48:26,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:48:26,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c74ad529015d0f95992a84 HTTP/1.1" 200 None 2026-03-28 03:48:26,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:48:26,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:48:26,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:48:26,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c74ad529015d0f95992a84 HTTP/1.1" 200 None 2026-03-28 03:48:26,907 - lava_callback - INFO - Completed processing callback for node 69c74ad529015d0f95992a84 INFO: 172.21.0.1:53946 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:48:36,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:48:36,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46674 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:55:32,122 - lava_callback - INFO - Received callback for node 69c72cdf29015d0f95992303 INFO: 172.21.0.1:43814 - "POST /node/69c72cdf29015d0f95992303 HTTP/1.0" 202 Accepted 2026-03-28 03:55:32,128 - lava_callback - INFO - Starting async job submit for node 69c72cdf29015d0f95992303 2026-03-28 03:55:32,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:55:32,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdf29015d0f95992303 HTTP/1.1" 200 None 2026-03-28 03:55:32,168 - lava_callback - DEBUG - Processing node 69c72cdf29015d0f95992303: baseline-arm64 2026-03-28 03:55:32,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:55:32,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:55:32,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdf29015d0f95992303/log.txt.gz 2026-03-28 03:55:33,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:55:33,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:55:33,365 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdf29015d0f95992303/lava-logs.yaml 2026-03-28 03:55:33,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:55:33,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:55:33,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdf29015d0f95992303/lava_callback.json.gz 2026-03-28 03:55:33,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:55:33,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdf29015d0f95992303 HTTP/1.1" 200 None 2026-03-28 03:55:33,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:55:33,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:55:33,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:55:33,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cdf29015d0f95992303 HTTP/1.1" 200 None 2026-03-28 03:55:33,691 - lava_callback - INFO - Completed processing callback for node 69c72cdf29015d0f95992303 INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:55:36,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:55:36,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40090 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:41:18,803 - lava_callback - INFO - Received callback for node 69c4b8dc590270ad7b6d9c7f INFO: 172.21.0.1:40098 - "POST /node/69c4b8dc590270ad7b6d9c7f HTTP/1.0" 202 Accepted 2026-03-28 04:41:18,809 - lava_callback - INFO - Starting async job submit for node 69c4b8dc590270ad7b6d9c7f 2026-03-28 04:41:18,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:41:18,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b8dc590270ad7b6d9c7f HTTP/1.1" 200 None 2026-03-28 04:41:18,825 - lava_callback - DEBUG - Processing node 69c4b8dc590270ad7b6d9c7f: baseline-arm-baylibre 2026-03-28 04:41:18,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b8dc590270ad7b6d9c7f 2026-03-28 04:41:18,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:41:18,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b8dc590270ad7b6d9c7f HTTP/1.1" 200 None 2026-03-28 04:41:18,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:41:18,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:41:18,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:41:18,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b8dc590270ad7b6d9c7f HTTP/1.1" 200 None 2026-03-28 04:41:18,891 - lava_callback - ERROR - Error processing callback for node 69c4b8dc590270ad7b6d9c7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52848 - "GET /node/6992680698a8bdb31c3920ea%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-28 04:42:18,808 - lava_callback - INFO - Received callback for node 69c4b917590270ad7b6d9c8e INFO: 172.21.0.1:52862 - "POST /node/69c4b917590270ad7b6d9c8e HTTP/1.0" 202 Accepted 2026-03-28 04:42:18,814 - lava_callback - INFO - Starting async job submit for node 69c4b917590270ad7b6d9c8e 2026-03-28 04:42:18,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:18,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b917590270ad7b6d9c8e HTTP/1.1" 200 None 2026-03-28 04:42:18,831 - lava_callback - DEBUG - Processing node 69c4b917590270ad7b6d9c8e: baseline-arm-baylibre 2026-03-28 04:42:18,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b917590270ad7b6d9c8e 2026-03-28 04:42:18,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:19,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b917590270ad7b6d9c8e HTTP/1.1" 200 None 2026-03-28 04:42:19,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:19,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:42:19,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:19,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b917590270ad7b6d9c8e HTTP/1.1" 200 None 2026-03-28 04:42:19,068 - lava_callback - ERROR - Error processing callback for node 69c4b917590270ad7b6d9c8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38026 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:42:39,041 - lava_callback - INFO - Received callback for node 69c4b93b590270ad7b6d9c98 INFO: 172.21.0.1:38030 - "POST /node/69c4b93b590270ad7b6d9c98 HTTP/1.0" 202 Accepted 2026-03-28 04:42:39,046 - lava_callback - INFO - Starting async job submit for node 69c4b93b590270ad7b6d9c98 2026-03-28 04:42:39,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:39,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b93b590270ad7b6d9c98 HTTP/1.1" 200 None 2026-03-28 04:42:39,063 - lava_callback - DEBUG - Processing node 69c4b93b590270ad7b6d9c98: baseline-arm-baylibre 2026-03-28 04:42:39,064 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b93b590270ad7b6d9c98 2026-03-28 04:42:39,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:39,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b93b590270ad7b6d9c98 HTTP/1.1" 200 None 2026-03-28 04:42:39,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:39,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:42:39,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:39,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b93b590270ad7b6d9c98 HTTP/1.1" 200 None 2026-03-28 04:42:39,119 - lava_callback - ERROR - Error processing callback for node 69c4b93b590270ad7b6d9c98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59794 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:43:18,867 - lava_callback - INFO - Received callback for node 69c4b960590270ad7b6d9ca3 INFO: 172.21.0.1:59804 - "POST /node/69c4b960590270ad7b6d9ca3 HTTP/1.0" 202 Accepted 2026-03-28 04:43:18,872 - lava_callback - INFO - Starting async job submit for node 69c4b960590270ad7b6d9ca3 2026-03-28 04:43:18,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:43:18,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b960590270ad7b6d9ca3 HTTP/1.1" 200 None 2026-03-28 04:43:18,896 - lava_callback - DEBUG - Processing node 69c4b960590270ad7b6d9ca3: baseline-arm-baylibre 2026-03-28 04:43:18,896 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b960590270ad7b6d9ca3 2026-03-28 04:43:18,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:43:18,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b960590270ad7b6d9ca3 HTTP/1.1" 200 None 2026-03-28 04:43:18,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:43:18,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:43:18,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:43:18,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b960590270ad7b6d9ca3 HTTP/1.1" 200 None 2026-03-28 04:43:18,980 - lava_callback - ERROR - Error processing callback for node 69c4b960590270ad7b6d9ca3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41746 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:44:18,973 - lava_callback - INFO - Received callback for node 69c4b990590270ad7b6d9cb3 INFO: 172.21.0.1:41760 - "POST /node/69c4b990590270ad7b6d9cb3 HTTP/1.0" 202 Accepted 2026-03-28 04:44:18,978 - lava_callback - INFO - Starting async job submit for node 69c4b990590270ad7b6d9cb3 2026-03-28 04:44:18,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:18,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b990590270ad7b6d9cb3 HTTP/1.1" 200 None 2026-03-28 04:44:18,995 - lava_callback - DEBUG - Processing node 69c4b990590270ad7b6d9cb3: baseline-arm-baylibre 2026-03-28 04:44:18,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b990590270ad7b6d9cb3 2026-03-28 04:44:18,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:19,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b990590270ad7b6d9cb3 HTTP/1.1" 200 None 2026-03-28 04:44:19,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:19,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:44:19,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:19,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b990590270ad7b6d9cb3 HTTP/1.1" 200 None 2026-03-28 04:44:19,075 - lava_callback - ERROR - Error processing callback for node 69c4b990590270ad7b6d9cb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54048 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:44:58,947 - lava_callback - INFO - Received callback for node 69c4b9b5590270ad7b6d9cbf INFO: 172.21.0.1:54058 - "POST /node/69c4b9b5590270ad7b6d9cbf HTTP/1.0" 202 Accepted 2026-03-28 04:44:58,952 - lava_callback - INFO - Starting async job submit for node 69c4b9b5590270ad7b6d9cbf 2026-03-28 04:44:58,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:58,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b9b5590270ad7b6d9cbf HTTP/1.1" 200 None 2026-03-28 04:44:58,969 - lava_callback - DEBUG - Processing node 69c4b9b5590270ad7b6d9cbf: baseline-arm-baylibre 2026-03-28 04:44:58,969 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b9b5590270ad7b6d9cbf 2026-03-28 04:44:58,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:59,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b9b5590270ad7b6d9cbf HTTP/1.1" 200 None 2026-03-28 04:44:59,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:59,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:44:59,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:59,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b9b5590270ad7b6d9cbf HTTP/1.1" 200 None 2026-03-28 04:44:59,052 - lava_callback - ERROR - Error processing callback for node 69c4b9b5590270ad7b6d9cbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:45:18,967 - lava_callback - INFO - Received callback for node 69c4b9da590270ad7b6d9cd0 INFO: 172.21.0.1:45814 - "POST /node/69c4b9da590270ad7b6d9cd0 HTTP/1.0" 202 Accepted 2026-03-28 04:45:18,973 - lava_callback - INFO - Starting async job submit for node 69c4b9da590270ad7b6d9cd0 2026-03-28 04:45:18,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:45:18,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b9da590270ad7b6d9cd0 HTTP/1.1" 200 None 2026-03-28 04:45:18,989 - lava_callback - DEBUG - Processing node 69c4b9da590270ad7b6d9cd0: baseline-arm-baylibre 2026-03-28 04:45:18,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b9da590270ad7b6d9cd0 2026-03-28 04:45:18,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:45:19,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b9da590270ad7b6d9cd0 HTTP/1.1" 200 None 2026-03-28 04:45:19,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:45:19,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:45:19,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:45:19,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b9da590270ad7b6d9cd0 HTTP/1.1" 200 None 2026-03-28 04:45:19,072 - lava_callback - ERROR - Error processing callback for node 69c4b9da590270ad7b6d9cd0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34032 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:48:13,277 - lava_callback - INFO - Received callback for node 69c4ba7d590270ad7b6d9d07 INFO: 172.21.0.1:55932 - "POST /node/69c4ba7d590270ad7b6d9d07 HTTP/1.0" 202 Accepted 2026-03-28 04:48:13,284 - lava_callback - INFO - Starting async job submit for node 69c4ba7d590270ad7b6d9d07 2026-03-28 04:48:13,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:13,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ba7d590270ad7b6d9d07 HTTP/1.1" 200 None 2026-03-28 04:48:13,300 - lava_callback - DEBUG - Processing node 69c4ba7d590270ad7b6d9d07: baseline-arm-clabbe 2026-03-28 04:48:13,301 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4ba7d590270ad7b6d9d07 2026-03-28 04:48:13,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:13,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ba7d590270ad7b6d9d07 HTTP/1.1" 200 None 2026-03-28 04:48:13,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:13,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:48:13,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:13,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ba7d590270ad7b6d9d07 HTTP/1.1" 200 None 2026-03-28 04:48:13,354 - lava_callback - ERROR - Error processing callback for node 69c4ba7d590270ad7b6d9d07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39322 - "GET /node/69b12a0da870a1dc3343bb74%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:48:53,347 - lava_callback - INFO - Received callback for node 69c4baa5590270ad7b6d9d1b INFO: 172.21.0.1:45498 - "POST /node/69c4baa5590270ad7b6d9d1b HTTP/1.0" 202 Accepted 2026-03-28 04:48:53,353 - lava_callback - INFO - Starting async job submit for node 69c4baa5590270ad7b6d9d1b 2026-03-28 04:48:53,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:53,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4baa5590270ad7b6d9d1b HTTP/1.1" 200 None 2026-03-28 04:48:53,370 - lava_callback - DEBUG - Processing node 69c4baa5590270ad7b6d9d1b: baseline-arm-clabbe 2026-03-28 04:48:53,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4baa5590270ad7b6d9d1b 2026-03-28 04:48:53,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:53,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4baa5590270ad7b6d9d1b HTTP/1.1" 200 None 2026-03-28 04:48:53,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:53,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:48:53,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:53,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4baa5590270ad7b6d9d1b HTTP/1.1" 200 None 2026-03-28 04:48:53,447 - lava_callback - ERROR - Error processing callback for node 69c4baa5590270ad7b6d9d1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35872 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:49:33,228 - lava_callback - INFO - Received callback for node 69c4bacd590270ad7b6d9d25 INFO: 172.21.0.1:50142 - "POST /node/69c4bacd590270ad7b6d9d25 HTTP/1.0" 202 Accepted 2026-03-28 04:49:33,233 - lava_callback - INFO - Starting async job submit for node 69c4bacd590270ad7b6d9d25 2026-03-28 04:49:33,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:49:33,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bacd590270ad7b6d9d25 HTTP/1.1" 200 None 2026-03-28 04:49:33,249 - lava_callback - DEBUG - Processing node 69c4bacd590270ad7b6d9d25: baseline-arm-clabbe 2026-03-28 04:49:33,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bacd590270ad7b6d9d25 2026-03-28 04:49:33,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:49:33,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bacd590270ad7b6d9d25 HTTP/1.1" 200 None 2026-03-28 04:49:33,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:49:33,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:49:33,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:49:33,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bacd590270ad7b6d9d25 HTTP/1.1" 200 None 2026-03-28 04:49:33,301 - lava_callback - ERROR - Error processing callback for node 69c4bacd590270ad7b6d9d25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37758 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 04:50:15,871 - lava_callback - INFO - Received callback for node 69c4baf5590270ad7b6d9d38 INFO: 172.21.0.1:37764 - "POST /node/69c4baf5590270ad7b6d9d38 HTTP/1.0" 202 Accepted 2026-03-28 04:50:15,877 - lava_callback - INFO - Starting async job submit for node 69c4baf5590270ad7b6d9d38 2026-03-28 04:50:15,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:15,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4baf5590270ad7b6d9d38 HTTP/1.1" 200 None 2026-03-28 04:50:15,893 - lava_callback - DEBUG - Processing node 69c4baf5590270ad7b6d9d38: baseline-arm-clabbe 2026-03-28 04:50:15,893 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4baf5590270ad7b6d9d38 2026-03-28 04:50:15,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:15,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4baf5590270ad7b6d9d38 HTTP/1.1" 200 None 2026-03-28 04:50:15,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:15,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:50:15,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:15,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4baf5590270ad7b6d9d38 HTTP/1.1" 200 None 2026-03-28 04:50:15,957 - lava_callback - ERROR - Error processing callback for node 69c4baf5590270ad7b6d9d38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58832 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:50:39,291 - lava_callback - INFO - Received callback for node 69c4bb17590270ad7b6d9d49 INFO: 172.21.0.1:58848 - "POST /node/69c4bb17590270ad7b6d9d49 HTTP/1.0" 202 Accepted 2026-03-28 04:50:39,296 - lava_callback - INFO - Starting async job submit for node 69c4bb17590270ad7b6d9d49 2026-03-28 04:50:39,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:39,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb17590270ad7b6d9d49 HTTP/1.1" 200 None 2026-03-28 04:50:39,320 - lava_callback - DEBUG - Processing node 69c4bb17590270ad7b6d9d49: baseline-arm-baylibre 2026-03-28 04:50:39,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bb17590270ad7b6d9d49 2026-03-28 04:50:39,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:39,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb17590270ad7b6d9d49 HTTP/1.1" 200 None 2026-03-28 04:50:39,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:39,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:50:39,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:39,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bb17590270ad7b6d9d49 HTTP/1.1" 200 None 2026-03-28 04:50:39,388 - lava_callback - ERROR - Error processing callback for node 69c4bb17590270ad7b6d9d49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56474 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:51:19,302 - lava_callback - INFO - Received callback for node 69c4bb3c590270ad7b6d9d5a INFO: 172.21.0.1:56486 - "POST /node/69c4bb3c590270ad7b6d9d5a HTTP/1.0" 202 Accepted 2026-03-28 04:51:19,308 - lava_callback - INFO - Starting async job submit for node 69c4bb3c590270ad7b6d9d5a 2026-03-28 04:51:19,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:19,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb3c590270ad7b6d9d5a HTTP/1.1" 200 None 2026-03-28 04:51:19,324 - lava_callback - DEBUG - Processing node 69c4bb3c590270ad7b6d9d5a: baseline-arm-baylibre 2026-03-28 04:51:19,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bb3c590270ad7b6d9d5a 2026-03-28 04:51:19,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:19,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb3c590270ad7b6d9d5a HTTP/1.1" 200 None 2026-03-28 04:51:19,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:19,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:51:19,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:19,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bb3c590270ad7b6d9d5a HTTP/1.1" 200 None 2026-03-28 04:51:19,377 - lava_callback - ERROR - Error processing callback for node 69c4bb3c590270ad7b6d9d5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60660 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:51:53,415 - lava_callback - INFO - Received callback for node 69c4bb63590270ad7b6d9d67 INFO: 172.21.0.1:52350 - "POST /node/69c4bb63590270ad7b6d9d67 HTTP/1.0" 202 Accepted 2026-03-28 04:51:53,421 - lava_callback - INFO - Starting async job submit for node 69c4bb63590270ad7b6d9d67 2026-03-28 04:51:53,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:53,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb63590270ad7b6d9d67 HTTP/1.1" 200 None 2026-03-28 04:51:53,437 - lava_callback - DEBUG - Processing node 69c4bb63590270ad7b6d9d67: baseline-arm-clabbe 2026-03-28 04:51:53,437 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bb63590270ad7b6d9d67 2026-03-28 04:51:53,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:53,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb63590270ad7b6d9d67 HTTP/1.1" 200 None 2026-03-28 04:51:53,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:53,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:51:53,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:53,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bb63590270ad7b6d9d67 HTTP/1.1" 200 None 2026-03-28 04:51:53,501 - lava_callback - ERROR - Error processing callback for node 69c4bb63590270ad7b6d9d67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58990 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:53:53,473 - lava_callback - INFO - Received callback for node 69c4bbd0590270ad7b6d9da6 INFO: 172.21.0.1:36528 - "POST /node/69c4bbd0590270ad7b6d9da6 HTTP/1.0" 202 Accepted 2026-03-28 04:53:53,479 - lava_callback - INFO - Starting async job submit for node 69c4bbd0590270ad7b6d9da6 2026-03-28 04:53:53,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:53:53,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bbd0590270ad7b6d9da6 HTTP/1.1" 200 None 2026-03-28 04:53:53,498 - lava_callback - DEBUG - Processing node 69c4bbd0590270ad7b6d9da6: baseline-arm-clabbe 2026-03-28 04:53:53,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bbd0590270ad7b6d9da6 2026-03-28 04:53:53,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:53:53,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bbd0590270ad7b6d9da6 HTTP/1.1" 200 None 2026-03-28 04:53:53,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:53:53,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:53:53,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:53:53,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bbd0590270ad7b6d9da6 HTTP/1.1" 200 None 2026-03-28 04:53:53,557 - lava_callback - ERROR - Error processing callback for node 69c4bbd0590270ad7b6d9da6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42822 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:54:33,473 - lava_callback - INFO - Received callback for node 69c4bbf6590270ad7b6d9dbd INFO: 172.21.0.1:53382 - "POST /node/69c4bbf6590270ad7b6d9dbd HTTP/1.0" 202 Accepted 2026-03-28 04:54:33,479 - lava_callback - INFO - Starting async job submit for node 69c4bbf6590270ad7b6d9dbd 2026-03-28 04:54:33,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:33,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bbf6590270ad7b6d9dbd HTTP/1.1" 200 None 2026-03-28 04:54:33,495 - lava_callback - DEBUG - Processing node 69c4bbf6590270ad7b6d9dbd: baseline-arm-clabbe 2026-03-28 04:54:33,495 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bbf6590270ad7b6d9dbd 2026-03-28 04:54:33,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:33,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bbf6590270ad7b6d9dbd HTTP/1.1" 200 None 2026-03-28 04:54:33,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:33,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:54:33,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:33,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bbf6590270ad7b6d9dbd HTTP/1.1" 200 None 2026-03-28 04:54:33,546 - lava_callback - ERROR - Error processing callback for node 69c4bbf6590270ad7b6d9dbd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57954 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:54:53,510 - lava_callback - INFO - Received callback for node 69c4bc1b590270ad7b6d9dcf INFO: 172.21.0.1:41414 - "POST /node/69c4bc1b590270ad7b6d9dcf HTTP/1.0" 202 Accepted 2026-03-28 04:54:53,515 - lava_callback - INFO - Starting async job submit for node 69c4bc1b590270ad7b6d9dcf 2026-03-28 04:54:53,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:53,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc1b590270ad7b6d9dcf HTTP/1.1" 200 None 2026-03-28 04:54:53,531 - lava_callback - DEBUG - Processing node 69c4bc1b590270ad7b6d9dcf: baseline-arm-clabbe 2026-03-28 04:54:53,531 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bc1b590270ad7b6d9dcf 2026-03-28 04:54:53,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:53,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc1b590270ad7b6d9dcf HTTP/1.1" 200 None 2026-03-28 04:54:53,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:53,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:54:53,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:53,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bc1b590270ad7b6d9dcf HTTP/1.1" 200 None 2026-03-28 04:54:53,596 - lava_callback - ERROR - Error processing callback for node 69c4bc1b590270ad7b6d9dcf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53880 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:55:33,500 - lava_callback - INFO - Received callback for node 69c4bc40590270ad7b6d9de4 INFO: 172.21.0.1:48558 - "POST /node/69c4bc40590270ad7b6d9de4 HTTP/1.0" 202 Accepted 2026-03-28 04:55:33,505 - lava_callback - INFO - Starting async job submit for node 69c4bc40590270ad7b6d9de4 2026-03-28 04:55:33,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:55:33,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc40590270ad7b6d9de4 HTTP/1.1" 200 None 2026-03-28 04:55:33,521 - lava_callback - DEBUG - Processing node 69c4bc40590270ad7b6d9de4: baseline-arm-clabbe 2026-03-28 04:55:33,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bc40590270ad7b6d9de4 2026-03-28 04:55:33,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:55:33,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc40590270ad7b6d9de4 HTTP/1.1" 200 None 2026-03-28 04:55:33,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:55:33,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:55:33,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:55:33,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bc40590270ad7b6d9de4 HTTP/1.1" 200 None 2026-03-28 04:55:33,571 - lava_callback - ERROR - Error processing callback for node 69c4bc40590270ad7b6d9de4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53848 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:56:13,582 - lava_callback - INFO - Received callback for node 69c4bc66590270ad7b6d9e01 INFO: 172.21.0.1:45602 - "POST /node/69c4bc66590270ad7b6d9e01 HTTP/1.0" 202 Accepted 2026-03-28 04:56:13,588 - lava_callback - INFO - Starting async job submit for node 69c4bc66590270ad7b6d9e01 2026-03-28 04:56:13,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:56:13,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc66590270ad7b6d9e01 HTTP/1.1" 200 None 2026-03-28 04:56:13,606 - lava_callback - DEBUG - Processing node 69c4bc66590270ad7b6d9e01: baseline-arm-clabbe 2026-03-28 04:56:13,606 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bc66590270ad7b6d9e01 2026-03-28 04:56:13,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:56:13,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc66590270ad7b6d9e01 HTTP/1.1" 200 None 2026-03-28 04:56:13,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:56:13,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:56:13,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:56:13,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bc66590270ad7b6d9e01 HTTP/1.1" 200 None 2026-03-28 04:56:13,687 - lava_callback - ERROR - Error processing callback for node 69c4bc66590270ad7b6d9e01: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45684 - "GET /node/6993f9bc3e9ba4930771d9a2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:44862 - "GET /node/69a0078262feda9f0694ec6f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52728 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:59:13,702 - lava_callback - INFO - Received callback for node 69c4bd1a590270ad7b6d9e27 INFO: 172.21.0.1:58382 - "POST /node/69c4bd1a590270ad7b6d9e27 HTTP/1.0" 202 Accepted 2026-03-28 04:59:13,708 - lava_callback - INFO - Starting async job submit for node 69c4bd1a590270ad7b6d9e27 2026-03-28 04:59:13,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:13,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd1a590270ad7b6d9e27 HTTP/1.1" 200 None 2026-03-28 04:59:13,724 - lava_callback - DEBUG - Processing node 69c4bd1a590270ad7b6d9e27: baseline-arm-clabbe 2026-03-28 04:59:13,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bd1a590270ad7b6d9e27 2026-03-28 04:59:13,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:13,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd1a590270ad7b6d9e27 HTTP/1.1" 200 None 2026-03-28 04:59:13,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:13,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:59:13,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:13,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bd1a590270ad7b6d9e27 HTTP/1.1" 200 None 2026-03-28 04:59:13,775 - lava_callback - ERROR - Error processing callback for node 69c4bd1a590270ad7b6d9e27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34158 - "GET /node/69865c69177ca0f097bd452b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:34164 - "GET /node/6989388885773790cd34d199%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:34174 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:59:53,675 - lava_callback - INFO - Received callback for node 69c4bd41590270ad7b6d9e30 INFO: 172.21.0.1:48526 - "POST /node/69c4bd41590270ad7b6d9e30 HTTP/1.0" 202 Accepted 2026-03-28 04:59:53,681 - lava_callback - INFO - Starting async job submit for node 69c4bd41590270ad7b6d9e30 2026-03-28 04:59:53,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:53,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd41590270ad7b6d9e30 HTTP/1.1" 200 None 2026-03-28 04:59:53,697 - lava_callback - DEBUG - Processing node 69c4bd41590270ad7b6d9e30: baseline-arm-clabbe 2026-03-28 04:59:53,697 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bd41590270ad7b6d9e30 2026-03-28 04:59:53,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:53,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd41590270ad7b6d9e30 HTTP/1.1" 200 None 2026-03-28 04:59:53,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:53,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:59:53,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:53,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bd41590270ad7b6d9e30 HTTP/1.1" 200 None 2026-03-28 04:59:53,780 - lava_callback - ERROR - Error processing callback for node 69c4bd41590270ad7b6d9e30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45588 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:00:33,742 - lava_callback - INFO - Received callback for node 69c4bd66590270ad7b6d9e45 INFO: 172.21.0.1:42226 - "POST /node/69c4bd66590270ad7b6d9e45 HTTP/1.0" 202 Accepted 2026-03-28 05:00:33,748 - lava_callback - INFO - Starting async job submit for node 69c4bd66590270ad7b6d9e45 2026-03-28 05:00:33,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:00:33,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd66590270ad7b6d9e45 HTTP/1.1" 200 None 2026-03-28 05:00:33,766 - lava_callback - DEBUG - Processing node 69c4bd66590270ad7b6d9e45: baseline-arm-clabbe 2026-03-28 05:00:33,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bd66590270ad7b6d9e45 2026-03-28 05:00:33,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:00:33,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd66590270ad7b6d9e45 HTTP/1.1" 200 None 2026-03-28 05:00:33,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:00:33,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:00:33,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:00:33,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bd66590270ad7b6d9e45 HTTP/1.1" 200 None 2026-03-28 05:00:33,851 - lava_callback - ERROR - Error processing callback for node 69c4bd66590270ad7b6d9e45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45290 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:07:54,344 - lava_callback - INFO - Received callback for node 69c4bf24590270ad7b6d9e90 INFO: 172.21.0.1:43432 - "POST /node/69c4bf24590270ad7b6d9e90 HTTP/1.0" 202 Accepted 2026-03-28 05:07:54,350 - lava_callback - INFO - Starting async job submit for node 69c4bf24590270ad7b6d9e90 2026-03-28 05:07:54,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:07:54,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bf24590270ad7b6d9e90 HTTP/1.1" 200 None 2026-03-28 05:07:54,367 - lava_callback - DEBUG - Processing node 69c4bf24590270ad7b6d9e90: baseline-arm-clabbe 2026-03-28 05:07:54,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bf24590270ad7b6d9e90 2026-03-28 05:07:54,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:07:54,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bf24590270ad7b6d9e90 HTTP/1.1" 200 None 2026-03-28 05:07:54,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:07:54,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:07:54,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:07:54,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bf24590270ad7b6d9e90 HTTP/1.1" 200 None 2026-03-28 05:07:54,417 - lava_callback - ERROR - Error processing callback for node 69c4bf24590270ad7b6d9e90: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54590 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:08:34,129 - lava_callback - INFO - Received callback for node 69c4bf48590270ad7b6d9ea2 INFO: 172.21.0.1:38518 - "POST /node/69c4bf48590270ad7b6d9ea2 HTTP/1.0" 202 Accepted 2026-03-28 05:08:34,135 - lava_callback - INFO - Starting async job submit for node 69c4bf48590270ad7b6d9ea2 2026-03-28 05:08:34,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:08:34,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bf48590270ad7b6d9ea2 HTTP/1.1" 200 None 2026-03-28 05:08:34,151 - lava_callback - DEBUG - Processing node 69c4bf48590270ad7b6d9ea2: baseline-arm-clabbe 2026-03-28 05:08:34,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bf48590270ad7b6d9ea2 2026-03-28 05:08:34,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:08:34,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bf48590270ad7b6d9ea2 HTTP/1.1" 200 None 2026-03-28 05:08:34,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:08:34,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:08:34,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:08:34,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bf48590270ad7b6d9ea2 HTTP/1.1" 200 None 2026-03-28 05:08:34,215 - lava_callback - ERROR - Error processing callback for node 69c4bf48590270ad7b6d9ea2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50938 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:12:20,495 - lava_callback - INFO - Received callback for node 69c4c02d590270ad7b6d9ec7 INFO: 172.21.0.1:55194 - "POST /node/69c4c02d590270ad7b6d9ec7 HTTP/1.0" 202 Accepted 2026-03-28 05:12:20,501 - lava_callback - INFO - Starting async job submit for node 69c4c02d590270ad7b6d9ec7 2026-03-28 05:12:20,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:12:20,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c02d590270ad7b6d9ec7 HTTP/1.1" 200 None 2026-03-28 05:12:20,516 - lava_callback - DEBUG - Processing node 69c4c02d590270ad7b6d9ec7: baseline-arm64-baylibre 2026-03-28 05:12:20,516 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c02d590270ad7b6d9ec7 2026-03-28 05:12:20,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:12:20,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c02d590270ad7b6d9ec7 HTTP/1.1" 200 None 2026-03-28 05:12:20,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:12:20,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:12:20,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:12:20,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c02d590270ad7b6d9ec7 HTTP/1.1" 200 None 2026-03-28 05:12:20,575 - lava_callback - ERROR - Error processing callback for node 69c4c02d590270ad7b6d9ec7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43894 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:13:40,610 - lava_callback - INFO - Received callback for node 69c4c07d590270ad7b6d9ed2 INFO: 172.21.0.1:47938 - "POST /node/69c4c07d590270ad7b6d9ed2 HTTP/1.0" 202 Accepted 2026-03-28 05:13:40,615 - lava_callback - INFO - Starting async job submit for node 69c4c07d590270ad7b6d9ed2 2026-03-28 05:13:40,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:13:40,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c07d590270ad7b6d9ed2 HTTP/1.1" 200 None 2026-03-28 05:13:40,631 - lava_callback - DEBUG - Processing node 69c4c07d590270ad7b6d9ed2: baseline-arm64-baylibre 2026-03-28 05:13:40,631 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c07d590270ad7b6d9ed2 2026-03-28 05:13:40,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:13:40,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c07d590270ad7b6d9ed2 HTTP/1.1" 200 None 2026-03-28 05:13:40,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:13:40,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:13:40,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:13:40,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c07d590270ad7b6d9ed2 HTTP/1.1" 200 None 2026-03-28 05:13:40,695 - lava_callback - ERROR - Error processing callback for node 69c4c07d590270ad7b6d9ed2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41046 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:15:00,674 - lava_callback - INFO - Received callback for node 69c4c0cd590270ad7b6d9edf INFO: 172.21.0.1:43636 - "POST /node/69c4c0cd590270ad7b6d9edf HTTP/1.0" 202 Accepted 2026-03-28 05:15:00,679 - lava_callback - INFO - Starting async job submit for node 69c4c0cd590270ad7b6d9edf 2026-03-28 05:15:00,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:15:00,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c0cd590270ad7b6d9edf HTTP/1.1" 200 None 2026-03-28 05:15:00,696 - lava_callback - DEBUG - Processing node 69c4c0cd590270ad7b6d9edf: baseline-arm64-baylibre 2026-03-28 05:15:00,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c0cd590270ad7b6d9edf 2026-03-28 05:15:00,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:15:00,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c0cd590270ad7b6d9edf HTTP/1.1" 200 None 2026-03-28 05:15:00,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:15:00,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:15:00,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:15:00,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c0cd590270ad7b6d9edf HTTP/1.1" 200 None 2026-03-28 05:15:00,766 - lava_callback - ERROR - Error processing callback for node 69c4c0cd590270ad7b6d9edf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34494 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:16:20,721 - lava_callback - INFO - Received callback for node 69c4c11f590270ad7b6d9eec INFO: 172.21.0.1:37080 - "POST /node/69c4c11f590270ad7b6d9eec HTTP/1.0" 202 Accepted 2026-03-28 05:16:20,727 - lava_callback - INFO - Starting async job submit for node 69c4c11f590270ad7b6d9eec 2026-03-28 05:16:20,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:16:20,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c11f590270ad7b6d9eec HTTP/1.1" 200 None 2026-03-28 05:16:20,743 - lava_callback - DEBUG - Processing node 69c4c11f590270ad7b6d9eec: baseline-arm64-baylibre 2026-03-28 05:16:20,743 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c11f590270ad7b6d9eec 2026-03-28 05:16:20,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:16:20,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c11f590270ad7b6d9eec HTTP/1.1" 200 None 2026-03-28 05:16:20,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:16:20,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:16:20,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:16:20,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c11f590270ad7b6d9eec HTTP/1.1" 200 None 2026-03-28 05:16:20,807 - lava_callback - ERROR - Error processing callback for node 69c4c11f590270ad7b6d9eec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48800 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:18:14,575 - lava_callback - INFO - Received callback for node 69c4c190590270ad7b6d9efc INFO: 172.21.0.1:56016 - "POST /node/69c4c190590270ad7b6d9efc HTTP/1.0" 202 Accepted 2026-03-28 05:18:14,581 - lava_callback - INFO - Starting async job submit for node 69c4c190590270ad7b6d9efc 2026-03-28 05:18:14,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:18:14,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c190590270ad7b6d9efc HTTP/1.1" 200 None 2026-03-28 05:18:14,596 - lava_callback - DEBUG - Processing node 69c4c190590270ad7b6d9efc: baseline-arm-clabbe 2026-03-28 05:18:14,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c190590270ad7b6d9efc 2026-03-28 05:18:14,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:18:14,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c190590270ad7b6d9efc HTTP/1.1" 200 None 2026-03-28 05:18:14,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:18:14,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:18:14,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:18:14,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c190590270ad7b6d9efc HTTP/1.1" 200 None 2026-03-28 05:18:14,646 - lava_callback - ERROR - Error processing callback for node 69c4c190590270ad7b6d9efc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33144 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:22:34,767 - lava_callback - INFO - Received callback for node 69c4c28f590270ad7b6d9f17 INFO: 172.21.0.1:51614 - "POST /node/69c4c28f590270ad7b6d9f17 HTTP/1.0" 202 Accepted 2026-03-28 05:22:34,772 - lava_callback - INFO - Starting async job submit for node 69c4c28f590270ad7b6d9f17 2026-03-28 05:22:34,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:22:34,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c28f590270ad7b6d9f17 HTTP/1.1" 200 None 2026-03-28 05:22:34,788 - lava_callback - DEBUG - Processing node 69c4c28f590270ad7b6d9f17: baseline-arm64-clabbe 2026-03-28 05:22:34,788 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c28f590270ad7b6d9f17 2026-03-28 05:22:34,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:22:34,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c28f590270ad7b6d9f17 HTTP/1.1" 200 None 2026-03-28 05:22:34,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:22:34,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:22:34,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:22:34,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c28f590270ad7b6d9f17 HTTP/1.1" 200 None 2026-03-28 05:22:34,853 - lava_callback - ERROR - Error processing callback for node 69c4c28f590270ad7b6d9f17: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:23:54,750 - lava_callback - INFO - Received callback for node 69c4c2de590270ad7b6d9f26 INFO: 172.21.0.1:51298 - "POST /node/69c4c2de590270ad7b6d9f26 HTTP/1.0" 202 Accepted 2026-03-28 05:23:54,756 - lava_callback - INFO - Starting async job submit for node 69c4c2de590270ad7b6d9f26 2026-03-28 05:23:54,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:23:54,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c2de590270ad7b6d9f26 HTTP/1.1" 200 None 2026-03-28 05:23:54,777 - lava_callback - DEBUG - Processing node 69c4c2de590270ad7b6d9f26: baseline-arm64-clabbe 2026-03-28 05:23:54,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c2de590270ad7b6d9f26 2026-03-28 05:23:54,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:23:54,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c2de590270ad7b6d9f26 HTTP/1.1" 200 None 2026-03-28 05:23:54,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:23:54,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:23:54,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:23:54,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c2de590270ad7b6d9f26 HTTP/1.1" 200 None 2026-03-28 05:23:54,848 - lava_callback - ERROR - Error processing callback for node 69c4c2de590270ad7b6d9f26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34500 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:25:34,843 - lava_callback - INFO - Received callback for node 69c4c343590270ad7b6d9f2f INFO: 172.21.0.1:56940 - "POST /node/69c4c343590270ad7b6d9f2f HTTP/1.0" 202 Accepted 2026-03-28 05:25:34,850 - lava_callback - INFO - Starting async job submit for node 69c4c343590270ad7b6d9f2f 2026-03-28 05:25:34,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:25:34,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c343590270ad7b6d9f2f HTTP/1.1" 200 None 2026-03-28 05:25:34,866 - lava_callback - DEBUG - Processing node 69c4c343590270ad7b6d9f2f: baseline-arm64-clabbe 2026-03-28 05:25:34,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c343590270ad7b6d9f2f 2026-03-28 05:25:34,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:25:34,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c343590270ad7b6d9f2f HTTP/1.1" 200 None 2026-03-28 05:25:34,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:25:34,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:25:34,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:25:34,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c343590270ad7b6d9f2f HTTP/1.1" 200 None 2026-03-28 05:25:34,917 - lava_callback - ERROR - Error processing callback for node 69c4c343590270ad7b6d9f2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49594 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:26:54,880 - lava_callback - INFO - Received callback for node 69c4c394590270ad7b6d9f3e INFO: 172.21.0.1:54428 - "POST /node/69c4c394590270ad7b6d9f3e HTTP/1.0" 202 Accepted 2026-03-28 05:26:54,885 - lava_callback - INFO - Starting async job submit for node 69c4c394590270ad7b6d9f3e 2026-03-28 05:26:54,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:26:54,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c394590270ad7b6d9f3e HTTP/1.1" 200 None 2026-03-28 05:26:54,901 - lava_callback - DEBUG - Processing node 69c4c394590270ad7b6d9f3e: baseline-arm64-clabbe 2026-03-28 05:26:54,901 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c394590270ad7b6d9f3e 2026-03-28 05:26:54,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:26:54,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c394590270ad7b6d9f3e HTTP/1.1" 200 None 2026-03-28 05:26:54,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:26:54,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:26:54,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:26:54,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c394590270ad7b6d9f3e HTTP/1.1" 200 None 2026-03-28 05:26:54,957 - lava_callback - ERROR - Error processing callback for node 69c4c394590270ad7b6d9f3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45190 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:28:14,957 - lava_callback - INFO - Received callback for node 69c4c3e5590270ad7b6d9f4a INFO: 172.21.0.1:44944 - "POST /node/69c4c3e5590270ad7b6d9f4a HTTP/1.0" 202 Accepted 2026-03-28 05:28:14,963 - lava_callback - INFO - Starting async job submit for node 69c4c3e5590270ad7b6d9f4a 2026-03-28 05:28:14,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:28:14,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c3e5590270ad7b6d9f4a HTTP/1.1" 200 None 2026-03-28 05:28:14,984 - lava_callback - DEBUG - Processing node 69c4c3e5590270ad7b6d9f4a: baseline-arm64-clabbe 2026-03-28 05:28:14,985 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c3e5590270ad7b6d9f4a 2026-03-28 05:28:14,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:28:14,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c3e5590270ad7b6d9f4a HTTP/1.1" 200 None 2026-03-28 05:28:15,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:28:15,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:28:15,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:28:15,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c3e5590270ad7b6d9f4a HTTP/1.1" 200 None 2026-03-28 05:28:15,066 - lava_callback - ERROR - Error processing callback for node 69c4c3e5590270ad7b6d9f4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56732 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:29:34,950 - lava_callback - INFO - Received callback for node 69c4c435590270ad7b6d9f53 INFO: 172.21.0.1:60812 - "POST /node/69c4c435590270ad7b6d9f53 HTTP/1.0" 202 Accepted 2026-03-28 05:29:34,955 - lava_callback - INFO - Starting async job submit for node 69c4c435590270ad7b6d9f53 2026-03-28 05:29:34,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:29:34,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c435590270ad7b6d9f53 HTTP/1.1" 200 None 2026-03-28 05:29:34,971 - lava_callback - DEBUG - Processing node 69c4c435590270ad7b6d9f53: baseline-arm64-clabbe 2026-03-28 05:29:34,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c435590270ad7b6d9f53 2026-03-28 05:29:34,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:29:34,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c435590270ad7b6d9f53 HTTP/1.1" 200 None 2026-03-28 05:29:34,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:29:34,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:29:35,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:29:35,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c435590270ad7b6d9f53 HTTP/1.1" 200 None 2026-03-28 05:29:35,028 - lava_callback - ERROR - Error processing callback for node 69c4c435590270ad7b6d9f53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44050 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:30:55,113 - lava_callback - INFO - Received callback for node 69c4c478590270ad7b6d9f5e INFO: 172.21.0.1:48794 - "POST /node/69c4c478590270ad7b6d9f5e HTTP/1.0" 202 Accepted 2026-03-28 05:30:55,119 - lava_callback - INFO - Starting async job submit for node 69c4c478590270ad7b6d9f5e 2026-03-28 05:30:55,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:30:55,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c478590270ad7b6d9f5e HTTP/1.1" 200 None 2026-03-28 05:30:55,140 - lava_callback - DEBUG - Processing node 69c4c478590270ad7b6d9f5e: baseline-arm-clabbe 2026-03-28 05:30:55,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c478590270ad7b6d9f5e 2026-03-28 05:30:55,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:30:55,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c478590270ad7b6d9f5e HTTP/1.1" 200 None 2026-03-28 05:30:55,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:30:55,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:30:55,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:30:55,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c478590270ad7b6d9f5e HTTP/1.1" 200 None 2026-03-28 05:30:55,195 - lava_callback - ERROR - Error processing callback for node 69c4c478590270ad7b6d9f5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:31:15,005 - lava_callback - INFO - Received callback for node 69c4c49f590270ad7b6d9f75 INFO: 172.21.0.1:39198 - "POST /node/69c4c49f590270ad7b6d9f75 HTTP/1.0" 202 Accepted 2026-03-28 05:31:15,011 - lava_callback - INFO - Starting async job submit for node 69c4c49f590270ad7b6d9f75 2026-03-28 05:31:15,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:15,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c49f590270ad7b6d9f75 HTTP/1.1" 200 None 2026-03-28 05:31:15,027 - lava_callback - DEBUG - Processing node 69c4c49f590270ad7b6d9f75: baseline-arm-clabbe 2026-03-28 05:31:15,028 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c49f590270ad7b6d9f75 2026-03-28 05:31:15,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:15,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c49f590270ad7b6d9f75 HTTP/1.1" 200 None 2026-03-28 05:31:15,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:15,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:31:15,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:15,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c49f590270ad7b6d9f75 HTTP/1.1" 200 None 2026-03-28 05:31:15,082 - lava_callback - ERROR - Error processing callback for node 69c4c49f590270ad7b6d9f75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49372 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:31:55,068 - lava_callback - INFO - Received callback for node 69c4c4c4590270ad7b6d9f84 INFO: 172.21.0.1:39086 - "POST /node/69c4c4c4590270ad7b6d9f84 HTTP/1.0" 202 Accepted 2026-03-28 05:31:55,074 - lava_callback - INFO - Starting async job submit for node 69c4c4c4590270ad7b6d9f84 2026-03-28 05:31:55,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:55,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c4c4590270ad7b6d9f84 HTTP/1.1" 200 None 2026-03-28 05:31:55,090 - lava_callback - DEBUG - Processing node 69c4c4c4590270ad7b6d9f84: baseline-arm-clabbe 2026-03-28 05:31:55,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c4c4590270ad7b6d9f84 2026-03-28 05:31:55,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:55,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c4c4590270ad7b6d9f84 HTTP/1.1" 200 None 2026-03-28 05:31:55,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:55,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:31:55,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:55,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c4c4590270ad7b6d9f84 HTTP/1.1" 200 None 2026-03-28 05:31:55,151 - lava_callback - ERROR - Error processing callback for node 69c4c4c4590270ad7b6d9f84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58770 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:32:35,083 - lava_callback - INFO - Received callback for node 69c4c4ea590270ad7b6d9f95 INFO: 172.21.0.1:52262 - "POST /node/69c4c4ea590270ad7b6d9f95 HTTP/1.0" 202 Accepted 2026-03-28 05:32:35,088 - lava_callback - INFO - Starting async job submit for node 69c4c4ea590270ad7b6d9f95 2026-03-28 05:32:35,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:32:35,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c4ea590270ad7b6d9f95 HTTP/1.1" 200 None 2026-03-28 05:32:35,105 - lava_callback - DEBUG - Processing node 69c4c4ea590270ad7b6d9f95: baseline-arm-clabbe 2026-03-28 05:32:35,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c4ea590270ad7b6d9f95 2026-03-28 05:32:35,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:32:35,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c4ea590270ad7b6d9f95 HTTP/1.1" 200 None 2026-03-28 05:32:35,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:32:35,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:32:35,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:32:35,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c4ea590270ad7b6d9f95 HTTP/1.1" 200 None 2026-03-28 05:32:35,162 - lava_callback - ERROR - Error processing callback for node 69c4c4ea590270ad7b6d9f95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41922 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:16:44,080 - lava_callback - INFO - Received callback for node 69c4cf49590270ad7b6d9ffc INFO: 172.21.0.1:41926 - "POST /node/69c4cf49590270ad7b6d9ffc HTTP/1.0" 202 Accepted 2026-03-28 06:16:44,086 - lava_callback - INFO - Starting async job submit for node 69c4cf49590270ad7b6d9ffc 2026-03-28 06:16:44,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:16:44,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cf49590270ad7b6d9ffc HTTP/1.1" 200 None 2026-03-28 06:16:44,124 - lava_callback - DEBUG - Processing node 69c4cf49590270ad7b6d9ffc: baseline-arm64-baylibre 2026-03-28 06:16:44,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4cf49590270ad7b6d9ffc 2026-03-28 06:16:44,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:16:44,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cf49590270ad7b6d9ffc HTTP/1.1" 200 None 2026-03-28 06:16:44,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:16:44,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:16:44,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:16:44,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4cf49590270ad7b6d9ffc HTTP/1.1" 200 None 2026-03-28 06:16:44,182 - lava_callback - ERROR - Error processing callback for node 69c4cf49590270ad7b6d9ffc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:18:24,283 - lava_callback - INFO - Received callback for node 69c4cf9d590270ad7b6da00c INFO: 172.21.0.1:40484 - "POST /node/69c4cf9d590270ad7b6da00c HTTP/1.0" 202 Accepted 2026-03-28 06:18:24,288 - lava_callback - INFO - Starting async job submit for node 69c4cf9d590270ad7b6da00c 2026-03-28 06:18:24,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:18:24,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cf9d590270ad7b6da00c HTTP/1.1" 200 None 2026-03-28 06:18:24,311 - lava_callback - DEBUG - Processing node 69c4cf9d590270ad7b6da00c: baseline-arm64-baylibre 2026-03-28 06:18:24,312 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4cf9d590270ad7b6da00c 2026-03-28 06:18:24,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:18:24,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cf9d590270ad7b6da00c HTTP/1.1" 200 None 2026-03-28 06:18:24,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:18:24,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:18:24,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:18:24,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4cf9d590270ad7b6da00c HTTP/1.1" 200 None 2026-03-28 06:18:24,379 - lava_callback - ERROR - Error processing callback for node 69c4cf9d590270ad7b6da00c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50232 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:19:44,182 - lava_callback - INFO - Received callback for node 69c4cff1590270ad7b6da018 INFO: 172.21.0.1:50236 - "POST /node/69c4cff1590270ad7b6da018 HTTP/1.0" 202 Accepted 2026-03-28 06:19:44,188 - lava_callback - INFO - Starting async job submit for node 69c4cff1590270ad7b6da018 2026-03-28 06:19:44,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:19:44,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cff1590270ad7b6da018 HTTP/1.1" 200 None 2026-03-28 06:19:44,212 - lava_callback - DEBUG - Processing node 69c4cff1590270ad7b6da018: baseline-arm64-baylibre 2026-03-28 06:19:44,212 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4cff1590270ad7b6da018 2026-03-28 06:19:44,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:19:44,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cff1590270ad7b6da018 HTTP/1.1" 200 None 2026-03-28 06:19:44,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:19:44,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:19:44,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:19:44,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4cff1590270ad7b6da018 HTTP/1.1" 200 None 2026-03-28 06:19:44,298 - lava_callback - ERROR - Error processing callback for node 69c4cff1590270ad7b6da018: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35480 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:21:04,306 - lava_callback - INFO - Received callback for node 69c4d043590270ad7b6da02a INFO: 172.21.0.1:39812 - "POST /node/69c4d043590270ad7b6da02a HTTP/1.0" 202 Accepted 2026-03-28 06:21:04,312 - lava_callback - INFO - Starting async job submit for node 69c4d043590270ad7b6da02a 2026-03-28 06:21:04,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:21:04,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d043590270ad7b6da02a HTTP/1.1" 200 None 2026-03-28 06:21:04,328 - lava_callback - DEBUG - Processing node 69c4d043590270ad7b6da02a: baseline-arm64-baylibre 2026-03-28 06:21:04,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d043590270ad7b6da02a 2026-03-28 06:21:04,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:21:04,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d043590270ad7b6da02a HTTP/1.1" 200 None 2026-03-28 06:21:04,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:21:04,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:21:04,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:21:04,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d043590270ad7b6da02a HTTP/1.1" 200 None 2026-03-28 06:21:04,388 - lava_callback - ERROR - Error processing callback for node 69c4d043590270ad7b6da02a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41304 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:22:24,451 - lava_callback - INFO - Received callback for node 69c4d093590270ad7b6da038 INFO: 172.21.0.1:49332 - "POST /node/69c4d093590270ad7b6da038 HTTP/1.0" 202 Accepted 2026-03-28 06:22:24,456 - lava_callback - INFO - Starting async job submit for node 69c4d093590270ad7b6da038 2026-03-28 06:22:24,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:22:24,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d093590270ad7b6da038 HTTP/1.1" 200 None 2026-03-28 06:22:24,490 - lava_callback - DEBUG - Processing node 69c4d093590270ad7b6da038: baseline-arm64-baylibre 2026-03-28 06:22:24,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d093590270ad7b6da038 2026-03-28 06:22:24,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:22:24,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d093590270ad7b6da038 HTTP/1.1" 200 None 2026-03-28 06:22:24,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:22:24,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:22:24,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:22:24,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d093590270ad7b6da038 HTTP/1.1" 200 None 2026-03-28 06:22:24,543 - lava_callback - ERROR - Error processing callback for node 69c4d093590270ad7b6da038: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47638 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:23:44,439 - lava_callback - INFO - Received callback for node 69c4d0e6590270ad7b6da041 INFO: 172.21.0.1:47648 - "POST /node/69c4d0e6590270ad7b6da041 HTTP/1.0" 202 Accepted 2026-03-28 06:23:44,445 - lava_callback - INFO - Starting async job submit for node 69c4d0e6590270ad7b6da041 2026-03-28 06:23:44,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:23:44,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d0e6590270ad7b6da041 HTTP/1.1" 200 None 2026-03-28 06:23:44,461 - lava_callback - DEBUG - Processing node 69c4d0e6590270ad7b6da041: baseline-arm64-baylibre 2026-03-28 06:23:44,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d0e6590270ad7b6da041 2026-03-28 06:23:44,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:23:44,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d0e6590270ad7b6da041 HTTP/1.1" 200 None 2026-03-28 06:23:44,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:23:44,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:23:44,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:23:44,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d0e6590270ad7b6da041 HTTP/1.1" 200 None 2026-03-28 06:23:44,553 - lava_callback - ERROR - Error processing callback for node 69c4d0e6590270ad7b6da041: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42528 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:24:21,898 - lava_callback - INFO - Received callback for node 69c72d0229015d0f95992382 INFO: 172.21.0.1:41078 - "POST /node/69c72d0229015d0f95992382 HTTP/1.0" 202 Accepted 2026-03-28 06:24:21,905 - lava_callback - INFO - Starting async job submit for node 69c72d0229015d0f95992382 2026-03-28 06:24:21,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:24:21,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0229015d0f95992382 HTTP/1.1" 200 None 2026-03-28 06:24:21,959 - lava_callback - DEBUG - Processing node 69c72d0229015d0f95992382: baseline-arm64-broonie 2026-03-28 06:24:22,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:24:22,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:24:22,330 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0229015d0f95992382/log.txt.gz 2026-03-28 06:24:22,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:24:22,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:24:22,693 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0229015d0f95992382/lava-logs.yaml 2026-03-28 06:24:22,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:24:22,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:24:22,856 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0229015d0f95992382/lava_callback.json.gz 2026-03-28 06:24:22,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:24:22,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0229015d0f95992382 HTTP/1.1" 200 None 2026-03-28 06:24:22,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:24:22,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 06:24:22,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:24:22,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0229015d0f95992382 HTTP/1.1" 200 None 2026-03-28 06:24:22,949 - lava_callback - INFO - Completed processing callback for node 69c72d0229015d0f95992382 INFO: 172.21.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48648 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:24:36,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:24:36,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40662 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:25:04,520 - lava_callback - INFO - Received callback for node 69c4d136590270ad7b6da04f INFO: 172.21.0.1:57682 - "POST /node/69c4d136590270ad7b6da04f HTTP/1.0" 202 Accepted 2026-03-28 06:25:04,525 - lava_callback - INFO - Starting async job submit for node 69c4d136590270ad7b6da04f 2026-03-28 06:25:04,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:25:04,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d136590270ad7b6da04f HTTP/1.1" 200 None 2026-03-28 06:25:04,541 - lava_callback - DEBUG - Processing node 69c4d136590270ad7b6da04f: baseline-arm64-baylibre 2026-03-28 06:25:04,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d136590270ad7b6da04f 2026-03-28 06:25:04,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:25:04,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d136590270ad7b6da04f HTTP/1.1" 200 None 2026-03-28 06:25:04,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:25:04,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:25:04,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:25:04,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d136590270ad7b6da04f HTTP/1.1" 200 None 2026-03-28 06:25:04,594 - lava_callback - ERROR - Error processing callback for node 69c4d136590270ad7b6da04f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:26:24,533 - lava_callback - INFO - Received callback for node 69c4d183590270ad7b6da05c INFO: 172.21.0.1:53540 - "POST /node/69c4d183590270ad7b6da05c HTTP/1.0" 202 Accepted 2026-03-28 06:26:24,539 - lava_callback - INFO - Starting async job submit for node 69c4d183590270ad7b6da05c 2026-03-28 06:26:24,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:26:24,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d183590270ad7b6da05c HTTP/1.1" 200 None 2026-03-28 06:26:24,555 - lava_callback - DEBUG - Processing node 69c4d183590270ad7b6da05c: baseline-arm64-baylibre 2026-03-28 06:26:24,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d183590270ad7b6da05c 2026-03-28 06:26:24,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:26:24,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d183590270ad7b6da05c HTTP/1.1" 200 None 2026-03-28 06:26:24,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:26:24,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:26:24,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:26:24,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d183590270ad7b6da05c HTTP/1.1" 200 None 2026-03-28 06:26:24,612 - lava_callback - ERROR - Error processing callback for node 69c4d183590270ad7b6da05c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:27:44,628 - lava_callback - INFO - Received callback for node 69c4d1d1590270ad7b6da065 INFO: 172.21.0.1:56390 - "POST /node/69c4d1d1590270ad7b6da065 HTTP/1.0" 202 Accepted 2026-03-28 06:27:44,634 - lava_callback - INFO - Starting async job submit for node 69c4d1d1590270ad7b6da065 2026-03-28 06:27:44,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:27:44,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d1d1590270ad7b6da065 HTTP/1.1" 200 None 2026-03-28 06:27:44,651 - lava_callback - DEBUG - Processing node 69c4d1d1590270ad7b6da065: baseline-arm64-baylibre 2026-03-28 06:27:44,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d1d1590270ad7b6da065 2026-03-28 06:27:44,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:27:44,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d1d1590270ad7b6da065 HTTP/1.1" 200 None 2026-03-28 06:27:44,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:27:44,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:27:44,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:27:44,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d1d1590270ad7b6da065 HTTP/1.1" 200 None 2026-03-28 06:27:44,716 - lava_callback - ERROR - Error processing callback for node 69c4d1d1590270ad7b6da065: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60248 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:29:04,712 - lava_callback - INFO - Received callback for node 69c4d220590270ad7b6da06f INFO: 172.21.0.1:48692 - "POST /node/69c4d220590270ad7b6da06f HTTP/1.0" 202 Accepted 2026-03-28 06:29:04,721 - lava_callback - INFO - Starting async job submit for node 69c4d220590270ad7b6da06f 2026-03-28 06:29:04,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:29:04,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d220590270ad7b6da06f HTTP/1.1" 200 None 2026-03-28 06:29:04,760 - lava_callback - DEBUG - Processing node 69c4d220590270ad7b6da06f: baseline-arm64-baylibre 2026-03-28 06:29:04,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d220590270ad7b6da06f 2026-03-28 06:29:04,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:29:04,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d220590270ad7b6da06f HTTP/1.1" 200 None 2026-03-28 06:29:04,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:29:04,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:29:04,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:29:04,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d220590270ad7b6da06f HTTP/1.1" 200 None 2026-03-28 06:29:04,849 - lava_callback - ERROR - Error processing callback for node 69c4d220590270ad7b6da06f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53308 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:30:24,742 - lava_callback - INFO - Received callback for node 69c4d26e590270ad7b6da07c INFO: 172.21.0.1:40508 - "POST /node/69c4d26e590270ad7b6da07c HTTP/1.0" 202 Accepted 2026-03-28 06:30:24,748 - lava_callback - INFO - Starting async job submit for node 69c4d26e590270ad7b6da07c 2026-03-28 06:30:24,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:30:24,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d26e590270ad7b6da07c HTTP/1.1" 200 None 2026-03-28 06:30:24,764 - lava_callback - DEBUG - Processing node 69c4d26e590270ad7b6da07c: baseline-arm64-baylibre 2026-03-28 06:30:24,764 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d26e590270ad7b6da07c 2026-03-28 06:30:24,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:30:24,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d26e590270ad7b6da07c HTTP/1.1" 200 None 2026-03-28 06:30:24,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:30:24,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:30:24,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:30:24,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d26e590270ad7b6da07c HTTP/1.1" 200 None 2026-03-28 06:30:24,862 - lava_callback - ERROR - Error processing callback for node 69c4d26e590270ad7b6da07c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44874 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:31:44,880 - lava_callback - INFO - Received callback for node 69c4d2bd590270ad7b6da086 INFO: 172.21.0.1:44876 - "POST /node/69c4d2bd590270ad7b6da086 HTTP/1.0" 202 Accepted 2026-03-28 06:31:44,885 - lava_callback - INFO - Starting async job submit for node 69c4d2bd590270ad7b6da086 2026-03-28 06:31:44,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:31:44,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d2bd590270ad7b6da086 HTTP/1.1" 200 None 2026-03-28 06:31:44,912 - lava_callback - DEBUG - Processing node 69c4d2bd590270ad7b6da086: baseline-arm64-baylibre 2026-03-28 06:31:44,912 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d2bd590270ad7b6da086 2026-03-28 06:31:44,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:31:44,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d2bd590270ad7b6da086 HTTP/1.1" 200 None 2026-03-28 06:31:44,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:31:44,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:31:44,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:31:44,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d2bd590270ad7b6da086 HTTP/1.1" 200 None 2026-03-28 06:31:44,977 - lava_callback - ERROR - Error processing callback for node 69c4d2bd590270ad7b6da086: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54612 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:32:57,612 - lava_callback - INFO - Received callback for node 69c4d312590270ad7b6da091 INFO: 172.21.0.1:54616 - "POST /node/69c4d312590270ad7b6da091 HTTP/1.0" 202 Accepted 2026-03-28 06:32:57,617 - lava_callback - INFO - Starting async job submit for node 69c4d312590270ad7b6da091 2026-03-28 06:32:57,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:32:57,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d312590270ad7b6da091 HTTP/1.1" 200 None 2026-03-28 06:32:57,641 - lava_callback - DEBUG - Processing node 69c4d312590270ad7b6da091: baseline-arm64-clabbe 2026-03-28 06:32:57,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d312590270ad7b6da091 2026-03-28 06:32:57,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:32:57,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d312590270ad7b6da091 HTTP/1.1" 200 None 2026-03-28 06:32:57,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:32:57,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:32:57,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:32:57,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d312590270ad7b6da091 HTTP/1.1" 200 None 2026-03-28 06:32:57,701 - lava_callback - ERROR - Error processing callback for node 69c4d312590270ad7b6da091: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42418 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:34:17,657 - lava_callback - INFO - Received callback for node 69c4d360590270ad7b6da09d INFO: 172.21.0.1:42428 - "POST /node/69c4d360590270ad7b6da09d HTTP/1.0" 202 Accepted 2026-03-28 06:34:17,662 - lava_callback - INFO - Starting async job submit for node 69c4d360590270ad7b6da09d 2026-03-28 06:34:17,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:34:17,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d360590270ad7b6da09d HTTP/1.1" 200 None 2026-03-28 06:34:17,688 - lava_callback - DEBUG - Processing node 69c4d360590270ad7b6da09d: baseline-arm64-clabbe 2026-03-28 06:34:17,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d360590270ad7b6da09d 2026-03-28 06:34:17,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:34:17,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d360590270ad7b6da09d HTTP/1.1" 200 None 2026-03-28 06:34:17,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:34:17,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:34:17,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:34:17,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d360590270ad7b6da09d HTTP/1.1" 200 None 2026-03-28 06:34:17,801 - lava_callback - ERROR - Error processing callback for node 69c4d360590270ad7b6da09d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36768 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:35:37,704 - lava_callback - INFO - Received callback for node 69c4d3b0590270ad7b6da0a3 INFO: 172.21.0.1:36774 - "POST /node/69c4d3b0590270ad7b6da0a3 HTTP/1.0" 202 Accepted 2026-03-28 06:35:37,710 - lava_callback - INFO - Starting async job submit for node 69c4d3b0590270ad7b6da0a3 2026-03-28 06:35:37,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:35:37,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d3b0590270ad7b6da0a3 HTTP/1.1" 200 None 2026-03-28 06:35:37,749 - lava_callback - DEBUG - Processing node 69c4d3b0590270ad7b6da0a3: baseline-arm64-clabbe 2026-03-28 06:35:37,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d3b0590270ad7b6da0a3 2026-03-28 06:35:37,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:35:37,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d3b0590270ad7b6da0a3 HTTP/1.1" 200 None 2026-03-28 06:35:37,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:35:37,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:35:37,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:35:37,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d3b0590270ad7b6da0a3 HTTP/1.1" 200 None 2026-03-28 06:35:37,849 - lava_callback - ERROR - Error processing callback for node 69c4d3b0590270ad7b6da0a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59006 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:36:57,793 - lava_callback - INFO - Received callback for node 69c4d3ff590270ad7b6da0ad INFO: 172.21.0.1:59020 - "POST /node/69c4d3ff590270ad7b6da0ad HTTP/1.0" 202 Accepted 2026-03-28 06:36:57,798 - lava_callback - INFO - Starting async job submit for node 69c4d3ff590270ad7b6da0ad 2026-03-28 06:36:57,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:36:57,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d3ff590270ad7b6da0ad HTTP/1.1" 200 None 2026-03-28 06:36:57,816 - lava_callback - DEBUG - Processing node 69c4d3ff590270ad7b6da0ad: baseline-arm64-clabbe 2026-03-28 06:36:57,816 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d3ff590270ad7b6da0ad 2026-03-28 06:36:57,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:36:57,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d3ff590270ad7b6da0ad HTTP/1.1" 200 None 2026-03-28 06:36:57,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:36:57,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:36:57,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:36:57,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d3ff590270ad7b6da0ad HTTP/1.1" 200 None 2026-03-28 06:36:57,874 - lava_callback - ERROR - Error processing callback for node 69c4d3ff590270ad7b6da0ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41522 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:38:17,842 - lava_callback - INFO - Received callback for node 69c4d44f590270ad7b6da0ba INFO: 172.21.0.1:41524 - "POST /node/69c4d44f590270ad7b6da0ba HTTP/1.0" 202 Accepted 2026-03-28 06:38:17,848 - lava_callback - INFO - Starting async job submit for node 69c4d44f590270ad7b6da0ba 2026-03-28 06:38:17,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:38:17,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d44f590270ad7b6da0ba HTTP/1.1" 200 None 2026-03-28 06:38:17,864 - lava_callback - DEBUG - Processing node 69c4d44f590270ad7b6da0ba: baseline-arm64-clabbe 2026-03-28 06:38:17,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d44f590270ad7b6da0ba 2026-03-28 06:38:17,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:38:17,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d44f590270ad7b6da0ba HTTP/1.1" 200 None 2026-03-28 06:38:17,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:38:17,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:38:17,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:38:17,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d44f590270ad7b6da0ba HTTP/1.1" 200 None 2026-03-28 06:38:17,936 - lava_callback - ERROR - Error processing callback for node 69c4d44f590270ad7b6da0ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58462 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:39:37,861 - lava_callback - INFO - Received callback for node 69c4d49e590270ad7b6da0c7 INFO: 172.21.0.1:58466 - "POST /node/69c4d49e590270ad7b6da0c7 HTTP/1.0" 202 Accepted 2026-03-28 06:39:37,866 - lava_callback - INFO - Starting async job submit for node 69c4d49e590270ad7b6da0c7 2026-03-28 06:39:37,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:39:37,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d49e590270ad7b6da0c7 HTTP/1.1" 200 None 2026-03-28 06:39:37,882 - lava_callback - DEBUG - Processing node 69c4d49e590270ad7b6da0c7: baseline-arm64-clabbe 2026-03-28 06:39:37,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d49e590270ad7b6da0c7 2026-03-28 06:39:37,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:39:37,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d49e590270ad7b6da0c7 HTTP/1.1" 200 None 2026-03-28 06:39:37,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:39:37,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:39:37,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:39:37,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d49e590270ad7b6da0c7 HTTP/1.1" 200 None 2026-03-28 06:39:37,937 - lava_callback - ERROR - Error processing callback for node 69c4d49e590270ad7b6da0c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40794 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:40:45,308 - lava_callback - INFO - Received callback for node 69c4d4da590270ad7b6da0d4 INFO: 172.21.0.1:40800 - "POST /node/69c4d4da590270ad7b6da0d4 HTTP/1.0" 202 Accepted 2026-03-28 06:40:45,313 - lava_callback - INFO - Starting async job submit for node 69c4d4da590270ad7b6da0d4 2026-03-28 06:40:45,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:40:45,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d4da590270ad7b6da0d4 HTTP/1.1" 200 None 2026-03-28 06:40:45,339 - lava_callback - DEBUG - Processing node 69c4d4da590270ad7b6da0d4: baseline-arm-baylibre 2026-03-28 06:40:45,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d4da590270ad7b6da0d4 2026-03-28 06:40:45,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:40:45,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d4da590270ad7b6da0d4 HTTP/1.1" 200 None 2026-03-28 06:40:45,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:40:45,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:40:45,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:40:45,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d4da590270ad7b6da0d4 HTTP/1.1" 200 None 2026-03-28 06:40:45,396 - lava_callback - ERROR - Error processing callback for node 69c4d4da590270ad7b6da0d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51628 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:41:05,291 - lava_callback - INFO - Received callback for node 69c4d4ff590270ad7b6da0e2 INFO: 172.21.0.1:52162 - "POST /node/69c4d4ff590270ad7b6da0e2 HTTP/1.0" 202 Accepted 2026-03-28 06:41:05,296 - lava_callback - INFO - Starting async job submit for node 69c4d4ff590270ad7b6da0e2 2026-03-28 06:41:05,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:05,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d4ff590270ad7b6da0e2 HTTP/1.1" 200 None 2026-03-28 06:41:05,335 - lava_callback - DEBUG - Processing node 69c4d4ff590270ad7b6da0e2: baseline-arm-baylibre 2026-03-28 06:41:05,335 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d4ff590270ad7b6da0e2 2026-03-28 06:41:05,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:05,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d4ff590270ad7b6da0e2 HTTP/1.1" 200 None 2026-03-28 06:41:05,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:05,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:41:05,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:05,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d4ff590270ad7b6da0e2 HTTP/1.1" 200 None 2026-03-28 06:41:05,389 - lava_callback - ERROR - Error processing callback for node 69c4d4ff590270ad7b6da0e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34430 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:41:45,320 - lava_callback - INFO - Received callback for node 69c4d524590270ad7b6da122 INFO: 172.21.0.1:34446 - "POST /node/69c4d524590270ad7b6da122 HTTP/1.0" 202 Accepted 2026-03-28 06:41:45,325 - lava_callback - INFO - Starting async job submit for node 69c4d524590270ad7b6da122 2026-03-28 06:41:45,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:45,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d524590270ad7b6da122 HTTP/1.1" 200 None 2026-03-28 06:41:45,340 - lava_callback - DEBUG - Processing node 69c4d524590270ad7b6da122: baseline-arm-baylibre 2026-03-28 06:41:45,341 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d524590270ad7b6da122 2026-03-28 06:41:45,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:45,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d524590270ad7b6da122 HTTP/1.1" 200 None 2026-03-28 06:41:45,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:45,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:41:45,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:45,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d524590270ad7b6da122 HTTP/1.1" 200 None 2026-03-28 06:41:45,393 - lava_callback - ERROR - Error processing callback for node 69c4d524590270ad7b6da122: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55896 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:46:25,620 - lava_callback - INFO - Received callback for node 69c4d63c590270ad7b6da186 INFO: 172.21.0.1:57854 - "POST /node/69c4d63c590270ad7b6da186 HTTP/1.0" 202 Accepted 2026-03-28 06:46:25,626 - lava_callback - INFO - Starting async job submit for node 69c4d63c590270ad7b6da186 2026-03-28 06:46:25,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:46:25,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d63c590270ad7b6da186 HTTP/1.1" 200 None 2026-03-28 06:46:25,642 - lava_callback - DEBUG - Processing node 69c4d63c590270ad7b6da186: baseline-arm-baylibre 2026-03-28 06:46:25,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d63c590270ad7b6da186 2026-03-28 06:46:25,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:46:25,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d63c590270ad7b6da186 HTTP/1.1" 200 None 2026-03-28 06:46:25,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:46:25,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:46:25,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:46:25,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d63c590270ad7b6da186 HTTP/1.1" 200 None 2026-03-28 06:46:25,708 - lava_callback - ERROR - Error processing callback for node 69c4d63c590270ad7b6da186: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37834 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:51:18,350 - lava_callback - INFO - Received callback for node 69c4d762590270ad7b6da1f7 INFO: 172.21.0.1:37842 - "POST /node/69c4d762590270ad7b6da1f7 HTTP/1.0" 202 Accepted 2026-03-28 06:51:18,356 - lava_callback - INFO - Starting async job submit for node 69c4d762590270ad7b6da1f7 2026-03-28 06:51:18,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:18,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d762590270ad7b6da1f7 HTTP/1.1" 200 None 2026-03-28 06:51:18,371 - lava_callback - DEBUG - Processing node 69c4d762590270ad7b6da1f7: baseline-arm-clabbe 2026-03-28 06:51:18,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d762590270ad7b6da1f7 2026-03-28 06:51:18,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:18,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d762590270ad7b6da1f7 HTTP/1.1" 200 None 2026-03-28 06:51:18,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:18,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:51:18,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:18,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d762590270ad7b6da1f7 HTTP/1.1" 200 None 2026-03-28 06:51:18,435 - lava_callback - ERROR - Error processing callback for node 69c4d762590270ad7b6da1f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39918 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:51:58,371 - lava_callback - INFO - Received callback for node 69c4d788590270ad7b6da206 INFO: 172.21.0.1:39934 - "POST /node/69c4d788590270ad7b6da206 HTTP/1.0" 202 Accepted 2026-03-28 06:51:58,376 - lava_callback - INFO - Starting async job submit for node 69c4d788590270ad7b6da206 2026-03-28 06:51:58,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:58,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d788590270ad7b6da206 HTTP/1.1" 200 None 2026-03-28 06:51:58,392 - lava_callback - DEBUG - Processing node 69c4d788590270ad7b6da206: baseline-arm-clabbe 2026-03-28 06:51:58,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d788590270ad7b6da206 2026-03-28 06:51:58,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:58,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d788590270ad7b6da206 HTTP/1.1" 200 None 2026-03-28 06:51:58,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:58,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:51:58,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:58,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d788590270ad7b6da206 HTTP/1.1" 200 None 2026-03-28 06:51:58,467 - lava_callback - ERROR - Error processing callback for node 69c4d788590270ad7b6da206: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36590 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:52:38,468 - lava_callback - INFO - Received callback for node 69c4d7ad590270ad7b6da216 INFO: 172.21.0.1:36592 - "POST /node/69c4d7ad590270ad7b6da216 HTTP/1.0" 202 Accepted 2026-03-28 06:52:38,473 - lava_callback - INFO - Starting async job submit for node 69c4d7ad590270ad7b6da216 2026-03-28 06:52:38,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:52:38,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7ad590270ad7b6da216 HTTP/1.1" 200 None 2026-03-28 06:52:38,489 - lava_callback - DEBUG - Processing node 69c4d7ad590270ad7b6da216: baseline-arm-clabbe 2026-03-28 06:52:38,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d7ad590270ad7b6da216 2026-03-28 06:52:38,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:52:38,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7ad590270ad7b6da216 HTTP/1.1" 200 None 2026-03-28 06:52:38,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:52:38,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:52:38,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:52:38,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d7ad590270ad7b6da216 HTTP/1.1" 200 None 2026-03-28 06:52:38,561 - lava_callback - ERROR - Error processing callback for node 69c4d7ad590270ad7b6da216: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47424 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:53:18,414 - lava_callback - INFO - Received callback for node 69c4d7d3590270ad7b6da22f INFO: 172.21.0.1:47430 - "POST /node/69c4d7d3590270ad7b6da22f HTTP/1.0" 202 Accepted 2026-03-28 06:53:18,420 - lava_callback - INFO - Starting async job submit for node 69c4d7d3590270ad7b6da22f 2026-03-28 06:53:18,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:18,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7d3590270ad7b6da22f HTTP/1.1" 200 None 2026-03-28 06:53:18,436 - lava_callback - DEBUG - Processing node 69c4d7d3590270ad7b6da22f: baseline-arm-clabbe 2026-03-28 06:53:18,436 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d7d3590270ad7b6da22f 2026-03-28 06:53:18,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:18,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7d3590270ad7b6da22f HTTP/1.1" 200 None 2026-03-28 06:53:18,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:18,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:53:18,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:18,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d7d3590270ad7b6da22f HTTP/1.1" 200 None 2026-03-28 06:53:18,488 - lava_callback - ERROR - Error processing callback for node 69c4d7d3590270ad7b6da22f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43566 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:53:58,473 - lava_callback - INFO - Received callback for node 69c4d7f8590270ad7b6da239 INFO: 172.21.0.1:43580 - "POST /node/69c4d7f8590270ad7b6da239 HTTP/1.0" 202 Accepted 2026-03-28 06:53:58,479 - lava_callback - INFO - Starting async job submit for node 69c4d7f8590270ad7b6da239 2026-03-28 06:53:58,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:58,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7f8590270ad7b6da239 HTTP/1.1" 200 None 2026-03-28 06:53:58,500 - lava_callback - DEBUG - Processing node 69c4d7f8590270ad7b6da239: baseline-arm-clabbe 2026-03-28 06:53:58,500 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d7f8590270ad7b6da239 2026-03-28 06:53:58,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:58,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7f8590270ad7b6da239 HTTP/1.1" 200 None 2026-03-28 06:53:58,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:58,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:53:58,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:58,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d7f8590270ad7b6da239 HTTP/1.1" 200 None 2026-03-28 06:53:58,560 - lava_callback - ERROR - Error processing callback for node 69c4d7f8590270ad7b6da239: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44790 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:54:38,522 - lava_callback - INFO - Received callback for node 69c4d81f590270ad7b6da249 INFO: 172.21.0.1:44800 - "POST /node/69c4d81f590270ad7b6da249 HTTP/1.0" 202 Accepted 2026-03-28 06:54:38,528 - lava_callback - INFO - Starting async job submit for node 69c4d81f590270ad7b6da249 2026-03-28 06:54:38,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:54:38,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d81f590270ad7b6da249 HTTP/1.1" 200 None 2026-03-28 06:54:38,544 - lava_callback - DEBUG - Processing node 69c4d81f590270ad7b6da249: baseline-arm-clabbe 2026-03-28 06:54:38,544 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d81f590270ad7b6da249 2026-03-28 06:54:38,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:54:38,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d81f590270ad7b6da249 HTTP/1.1" 200 None 2026-03-28 06:54:38,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:54:38,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:54:38,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:54:38,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d81f590270ad7b6da249 HTTP/1.1" 200 None 2026-03-28 06:54:38,596 - lava_callback - ERROR - Error processing callback for node 69c4d81f590270ad7b6da249: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:55:58,593 - lava_callback - INFO - Received callback for node 69c4d86a590270ad7b6da270 INFO: 172.21.0.1:37116 - "POST /node/69c4d86a590270ad7b6da270 HTTP/1.0" 202 Accepted 2026-03-28 06:55:58,599 - lava_callback - INFO - Starting async job submit for node 69c4d86a590270ad7b6da270 2026-03-28 06:55:58,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:55:58,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d86a590270ad7b6da270 HTTP/1.1" 200 None 2026-03-28 06:55:58,615 - lava_callback - DEBUG - Processing node 69c4d86a590270ad7b6da270: baseline-arm-clabbe 2026-03-28 06:55:58,616 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d86a590270ad7b6da270 2026-03-28 06:55:58,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:55:58,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d86a590270ad7b6da270 HTTP/1.1" 200 None 2026-03-28 06:55:58,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:55:58,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:55:58,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:55:58,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d86a590270ad7b6da270 HTTP/1.1" 200 None 2026-03-28 06:55:58,670 - lava_callback - ERROR - Error processing callback for node 69c4d86a590270ad7b6da270: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37554 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:56:18,551 - lava_callback - INFO - Received callback for node 69c4d890590270ad7b6da27b INFO: 172.21.0.1:37560 - "POST /node/69c4d890590270ad7b6da27b HTTP/1.0" 202 Accepted 2026-03-28 06:56:18,557 - lava_callback - INFO - Starting async job submit for node 69c4d890590270ad7b6da27b 2026-03-28 06:56:18,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:18,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d890590270ad7b6da27b HTTP/1.1" 200 None 2026-03-28 06:56:18,581 - lava_callback - DEBUG - Processing node 69c4d890590270ad7b6da27b: baseline-arm-clabbe 2026-03-28 06:56:18,582 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d890590270ad7b6da27b 2026-03-28 06:56:18,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:18,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d890590270ad7b6da27b HTTP/1.1" 200 None 2026-03-28 06:56:18,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:18,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:56:18,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:18,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d890590270ad7b6da27b HTTP/1.1" 200 None 2026-03-28 06:56:18,643 - lava_callback - ERROR - Error processing callback for node 69c4d890590270ad7b6da27b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:56:58,577 - lava_callback - INFO - Received callback for node 69c4d8b5590270ad7b6da28c INFO: 172.21.0.1:56702 - "POST /node/69c4d8b5590270ad7b6da28c HTTP/1.0" 202 Accepted 2026-03-28 06:56:58,583 - lava_callback - INFO - Starting async job submit for node 69c4d8b5590270ad7b6da28c 2026-03-28 06:56:58,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:58,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d8b5590270ad7b6da28c HTTP/1.1" 200 None 2026-03-28 06:56:58,599 - lava_callback - DEBUG - Processing node 69c4d8b5590270ad7b6da28c: baseline-arm-clabbe 2026-03-28 06:56:58,600 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d8b5590270ad7b6da28c 2026-03-28 06:56:58,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:58,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d8b5590270ad7b6da28c HTTP/1.1" 200 None 2026-03-28 06:56:58,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:58,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:56:58,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:58,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d8b5590270ad7b6da28c HTTP/1.1" 200 None 2026-03-28 06:56:58,660 - lava_callback - ERROR - Error processing callback for node 69c4d8b5590270ad7b6da28c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35820 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:57:38,601 - lava_callback - INFO - Received callback for node 69c4d8dc590270ad7b6da29e INFO: 172.21.0.1:35830 - "POST /node/69c4d8dc590270ad7b6da29e HTTP/1.0" 202 Accepted 2026-03-28 06:57:38,606 - lava_callback - INFO - Starting async job submit for node 69c4d8dc590270ad7b6da29e 2026-03-28 06:57:38,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:57:38,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d8dc590270ad7b6da29e HTTP/1.1" 200 None 2026-03-28 06:57:38,622 - lava_callback - DEBUG - Processing node 69c4d8dc590270ad7b6da29e: baseline-arm-clabbe 2026-03-28 06:57:38,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d8dc590270ad7b6da29e 2026-03-28 06:57:38,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:57:38,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d8dc590270ad7b6da29e HTTP/1.1" 200 None 2026-03-28 06:57:38,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:57:38,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:57:38,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:57:38,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d8dc590270ad7b6da29e HTTP/1.1" 200 None 2026-03-28 06:57:38,674 - lava_callback - ERROR - Error processing callback for node 69c4d8dc590270ad7b6da29e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50718 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:58:18,711 - lava_callback - INFO - Received callback for node 69c4d901590270ad7b6da2b5 INFO: 172.21.0.1:50730 - "POST /node/69c4d901590270ad7b6da2b5 HTTP/1.0" 202 Accepted 2026-03-28 06:58:18,716 - lava_callback - INFO - Starting async job submit for node 69c4d901590270ad7b6da2b5 2026-03-28 06:58:18,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:18,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d901590270ad7b6da2b5 HTTP/1.1" 200 None 2026-03-28 06:58:18,732 - lava_callback - DEBUG - Processing node 69c4d901590270ad7b6da2b5: baseline-arm-clabbe 2026-03-28 06:58:18,733 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d901590270ad7b6da2b5 2026-03-28 06:58:18,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:18,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d901590270ad7b6da2b5 HTTP/1.1" 200 None 2026-03-28 06:58:18,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:18,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:58:18,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:18,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d901590270ad7b6da2b5 HTTP/1.1" 200 None 2026-03-28 06:58:18,792 - lava_callback - ERROR - Error processing callback for node 69c4d901590270ad7b6da2b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59620 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:58:58,643 - lava_callback - INFO - Received callback for node 69c4d926590270ad7b6da2c2 INFO: 172.21.0.1:59636 - "POST /node/69c4d926590270ad7b6da2c2 HTTP/1.0" 202 Accepted 2026-03-28 06:58:58,648 - lava_callback - INFO - Starting async job submit for node 69c4d926590270ad7b6da2c2 2026-03-28 06:58:58,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:58,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d926590270ad7b6da2c2 HTTP/1.1" 200 None 2026-03-28 06:58:58,664 - lava_callback - DEBUG - Processing node 69c4d926590270ad7b6da2c2: baseline-arm-clabbe 2026-03-28 06:58:58,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d926590270ad7b6da2c2 2026-03-28 06:58:58,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:58,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d926590270ad7b6da2c2 HTTP/1.1" 200 None 2026-03-28 06:58:58,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:58,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:58:58,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:58,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d926590270ad7b6da2c2 HTTP/1.1" 200 None 2026-03-28 06:58:58,718 - lava_callback - ERROR - Error processing callback for node 69c4d926590270ad7b6da2c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44550 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:59:38,784 - lava_callback - INFO - Received callback for node 69c4d94c590270ad7b6da2d5 INFO: 172.21.0.1:44564 - "POST /node/69c4d94c590270ad7b6da2d5 HTTP/1.0" 202 Accepted 2026-03-28 06:59:38,790 - lava_callback - INFO - Starting async job submit for node 69c4d94c590270ad7b6da2d5 2026-03-28 06:59:38,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:38,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d94c590270ad7b6da2d5 HTTP/1.1" 200 None 2026-03-28 06:59:38,806 - lava_callback - DEBUG - Processing node 69c4d94c590270ad7b6da2d5: baseline-arm-clabbe 2026-03-28 06:59:38,806 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d94c590270ad7b6da2d5 2026-03-28 06:59:38,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:38,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d94c590270ad7b6da2d5 HTTP/1.1" 200 None 2026-03-28 06:59:38,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:38,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:59:38,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:38,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d94c590270ad7b6da2d5 HTTP/1.1" 200 None 2026-03-28 06:59:38,858 - lava_callback - ERROR - Error processing callback for node 69c4d94c590270ad7b6da2d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56780 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:59:42,942 - lava_callback - INFO - Received callback for node 69c72d0a29015d0f959923b5 INFO: 172.21.0.1:56782 - "POST /node/69c72d0a29015d0f959923b5 HTTP/1.0" 202 Accepted 2026-03-28 06:59:42,948 - lava_callback - INFO - Starting async job submit for node 69c72d0a29015d0f959923b5 2026-03-28 06:59:42,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:42,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0a29015d0f959923b5 HTTP/1.1" 200 None 2026-03-28 06:59:42,986 - lava_callback - DEBUG - Processing node 69c72d0a29015d0f959923b5: baseline-arm64-broonie 2026-03-28 06:59:43,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:59:43,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:59:43,386 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0a29015d0f959923b5/log.txt.gz 2026-03-28 06:59:43,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:59:43,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:59:43,792 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0a29015d0f959923b5/lava-logs.yaml 2026-03-28 06:59:43,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:59:43,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:59:43,929 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0a29015d0f959923b5/lava_callback.json.gz 2026-03-28 06:59:43,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:43,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0a29015d0f959923b5 HTTP/1.1" 200 None 2026-03-28 06:59:43,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:43,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 06:59:43,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:44,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0a29015d0f959923b5 HTTP/1.1" 200 None 2026-03-28 06:59:44,020 - lava_callback - INFO - Completed processing callback for node 69c72d0a29015d0f959923b5 INFO: 172.21.0.1:56784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39426 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:00:06,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:06,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:00:18,851 - lava_callback - INFO - Received callback for node 69c4d972590270ad7b6da2e6 INFO: 172.21.0.1:33608 - "POST /node/69c4d972590270ad7b6da2e6 HTTP/1.0" 202 Accepted 2026-03-28 07:00:18,857 - lava_callback - INFO - Starting async job submit for node 69c4d972590270ad7b6da2e6 2026-03-28 07:00:18,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:18,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d972590270ad7b6da2e6 HTTP/1.1" 200 None 2026-03-28 07:00:18,873 - lava_callback - DEBUG - Processing node 69c4d972590270ad7b6da2e6: baseline-arm-clabbe 2026-03-28 07:00:18,873 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d972590270ad7b6da2e6 2026-03-28 07:00:18,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:18,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d972590270ad7b6da2e6 HTTP/1.1" 200 None 2026-03-28 07:00:18,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:18,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 07:00:18,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:18,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d972590270ad7b6da2e6 HTTP/1.1" 200 None 2026-03-28 07:00:18,934 - lava_callback - ERROR - Error processing callback for node 69c4d972590270ad7b6da2e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41386 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:00:58,837 - lava_callback - INFO - Received callback for node 69c4d998590270ad7b6da303 INFO: 172.21.0.1:41396 - "POST /node/69c4d998590270ad7b6da303 HTTP/1.0" 202 Accepted 2026-03-28 07:00:58,843 - lava_callback - INFO - Starting async job submit for node 69c4d998590270ad7b6da303 2026-03-28 07:00:58,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:58,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d998590270ad7b6da303 HTTP/1.1" 200 None 2026-03-28 07:00:58,858 - lava_callback - DEBUG - Processing node 69c4d998590270ad7b6da303: baseline-arm-clabbe 2026-03-28 07:00:58,858 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d998590270ad7b6da303 2026-03-28 07:00:58,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:58,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d998590270ad7b6da303 HTTP/1.1" 200 None 2026-03-28 07:00:58,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:58,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 07:00:58,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:58,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d998590270ad7b6da303 HTTP/1.1" 200 None 2026-03-28 07:00:58,917 - lava_callback - ERROR - Error processing callback for node 69c4d998590270ad7b6da303: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38114 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:01:38,850 - lava_callback - INFO - Received callback for node 69c4d9bf590270ad7b6da30d INFO: 172.21.0.1:38124 - "POST /node/69c4d9bf590270ad7b6da30d HTTP/1.0" 202 Accepted 2026-03-28 07:01:38,856 - lava_callback - INFO - Starting async job submit for node 69c4d9bf590270ad7b6da30d 2026-03-28 07:01:38,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:01:38,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d9bf590270ad7b6da30d HTTP/1.1" 200 None 2026-03-28 07:01:38,892 - lava_callback - DEBUG - Processing node 69c4d9bf590270ad7b6da30d: baseline-arm-clabbe 2026-03-28 07:01:38,892 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d9bf590270ad7b6da30d 2026-03-28 07:01:38,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:01:38,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d9bf590270ad7b6da30d HTTP/1.1" 200 None 2026-03-28 07:01:38,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:01:38,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:01:38,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:01:38,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d9bf590270ad7b6da30d HTTP/1.1" 200 None 2026-03-28 07:01:38,981 - lava_callback - ERROR - Error processing callback for node 69c4d9bf590270ad7b6da30d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52372 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:02:18,876 - lava_callback - INFO - Received callback for node 69c4d9e5590270ad7b6da31c INFO: 172.21.0.1:52374 - "POST /node/69c4d9e5590270ad7b6da31c HTTP/1.0" 202 Accepted 2026-03-28 07:02:18,882 - lava_callback - INFO - Starting async job submit for node 69c4d9e5590270ad7b6da31c 2026-03-28 07:02:18,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:18,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d9e5590270ad7b6da31c HTTP/1.1" 200 None 2026-03-28 07:02:18,898 - lava_callback - DEBUG - Processing node 69c4d9e5590270ad7b6da31c: baseline-arm-clabbe 2026-03-28 07:02:18,898 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d9e5590270ad7b6da31c 2026-03-28 07:02:18,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:18,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d9e5590270ad7b6da31c HTTP/1.1" 200 None 2026-03-28 07:02:18,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:18,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:02:18,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:18,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d9e5590270ad7b6da31c HTTP/1.1" 200 None 2026-03-28 07:02:18,952 - lava_callback - ERROR - Error processing callback for node 69c4d9e5590270ad7b6da31c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46748 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:02:38,864 - lava_callback - INFO - Received callback for node 69c4da0a590270ad7b6da336 INFO: 172.21.0.1:46764 - "POST /node/69c4da0a590270ad7b6da336 HTTP/1.0" 202 Accepted 2026-03-28 07:02:38,870 - lava_callback - INFO - Starting async job submit for node 69c4da0a590270ad7b6da336 2026-03-28 07:02:38,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:38,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da0a590270ad7b6da336 HTTP/1.1" 200 None 2026-03-28 07:02:38,886 - lava_callback - DEBUG - Processing node 69c4da0a590270ad7b6da336: baseline-arm-clabbe 2026-03-28 07:02:38,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4da0a590270ad7b6da336 2026-03-28 07:02:38,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:38,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da0a590270ad7b6da336 HTTP/1.1" 200 None 2026-03-28 07:02:38,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:38,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:02:38,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:38,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4da0a590270ad7b6da336 HTTP/1.1" 200 None 2026-03-28 07:02:38,939 - lava_callback - ERROR - Error processing callback for node 69c4da0a590270ad7b6da336: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37196 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:03:18,986 - lava_callback - INFO - Received callback for node 69c4da2f590270ad7b6da34d INFO: 172.21.0.1:37198 - "POST /node/69c4da2f590270ad7b6da34d HTTP/1.0" 202 Accepted 2026-03-28 07:03:18,992 - lava_callback - INFO - Starting async job submit for node 69c4da2f590270ad7b6da34d 2026-03-28 07:03:18,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:19,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da2f590270ad7b6da34d HTTP/1.1" 200 None 2026-03-28 07:03:19,007 - lava_callback - DEBUG - Processing node 69c4da2f590270ad7b6da34d: baseline-arm-clabbe 2026-03-28 07:03:19,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4da2f590270ad7b6da34d 2026-03-28 07:03:19,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:19,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da2f590270ad7b6da34d HTTP/1.1" 200 None 2026-03-28 07:03:19,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:19,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:03:19,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:19,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4da2f590270ad7b6da34d HTTP/1.1" 200 None 2026-03-28 07:03:19,059 - lava_callback - ERROR - Error processing callback for node 69c4da2f590270ad7b6da34d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57068 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:03:58,918 - lava_callback - INFO - Received callback for node 69c4da55590270ad7b6da35c INFO: 172.21.0.1:57072 - "POST /node/69c4da55590270ad7b6da35c HTTP/1.0" 202 Accepted 2026-03-28 07:03:58,924 - lava_callback - INFO - Starting async job submit for node 69c4da55590270ad7b6da35c 2026-03-28 07:03:58,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:58,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da55590270ad7b6da35c HTTP/1.1" 200 None 2026-03-28 07:03:58,940 - lava_callback - DEBUG - Processing node 69c4da55590270ad7b6da35c: baseline-arm-clabbe 2026-03-28 07:03:58,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4da55590270ad7b6da35c 2026-03-28 07:03:58,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:58,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da55590270ad7b6da35c HTTP/1.1" 200 None 2026-03-28 07:03:58,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:58,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:03:58,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:58,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4da55590270ad7b6da35c HTTP/1.1" 200 None 2026-03-28 07:03:58,992 - lava_callback - ERROR - Error processing callback for node 69c4da55590270ad7b6da35c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53990 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:04:39,013 - lava_callback - INFO - Received callback for node 69c4da7a590270ad7b6da36d INFO: 172.21.0.1:54002 - "POST /node/69c4da7a590270ad7b6da36d HTTP/1.0" 202 Accepted 2026-03-28 07:04:39,018 - lava_callback - INFO - Starting async job submit for node 69c4da7a590270ad7b6da36d 2026-03-28 07:04:39,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:04:39,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da7a590270ad7b6da36d HTTP/1.1" 200 None 2026-03-28 07:04:39,034 - lava_callback - DEBUG - Processing node 69c4da7a590270ad7b6da36d: baseline-arm-clabbe 2026-03-28 07:04:39,034 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4da7a590270ad7b6da36d 2026-03-28 07:04:39,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:04:39,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da7a590270ad7b6da36d HTTP/1.1" 200 None 2026-03-28 07:04:39,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:04:39,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:04:39,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:04:39,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4da7a590270ad7b6da36d HTTP/1.1" 200 None 2026-03-28 07:04:39,121 - lava_callback - ERROR - Error processing callback for node 69c4da7a590270ad7b6da36d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54758 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:05:19,002 - lava_callback - INFO - Received callback for node 69c4daa1590270ad7b6da37e INFO: 172.21.0.1:54764 - "POST /node/69c4daa1590270ad7b6da37e HTTP/1.0" 202 Accepted 2026-03-28 07:05:19,008 - lava_callback - INFO - Starting async job submit for node 69c4daa1590270ad7b6da37e 2026-03-28 07:05:19,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:19,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4daa1590270ad7b6da37e HTTP/1.1" 200 None 2026-03-28 07:05:19,025 - lava_callback - DEBUG - Processing node 69c4daa1590270ad7b6da37e: baseline-arm-clabbe 2026-03-28 07:05:19,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4daa1590270ad7b6da37e 2026-03-28 07:05:19,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:19,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4daa1590270ad7b6da37e HTTP/1.1" 200 None 2026-03-28 07:05:19,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:19,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:05:19,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:19,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4daa1590270ad7b6da37e HTTP/1.1" 200 None 2026-03-28 07:05:19,083 - lava_callback - ERROR - Error processing callback for node 69c4daa1590270ad7b6da37e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 07:05:20,103 - lava_callback - INFO - Received callback for node 69c72cf129015d0f95992350 INFO: 172.21.0.1:50738 - "POST /node/69c72cf129015d0f95992350 HTTP/1.0" 202 Accepted 2026-03-28 07:05:20,110 - lava_callback - INFO - Starting async job submit for node 69c72cf129015d0f95992350 2026-03-28 07:05:20,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:20,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf129015d0f95992350 HTTP/1.1" 200 None 2026-03-28 07:05:20,164 - lava_callback - DEBUG - Processing node 69c72cf129015d0f95992350: baseline-nfs-arm64 2026-03-28 07:05:20,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:05:20,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:05:20,971 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf129015d0f95992350/log.txt.gz 2026-03-28 07:05:21,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:05:21,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:05:21,467 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf129015d0f95992350/lava-logs.yaml 2026-03-28 07:05:21,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:05:21,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:05:21,656 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf129015d0f95992350/lava_callback.json.gz 2026-03-28 07:05:21,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:21,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf129015d0f95992350 HTTP/1.1" 200 None 2026-03-28 07:05:21,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:21,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 07:05:21,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:21,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cf129015d0f95992350 HTTP/1.1" 200 None 2026-03-28 07:05:21,824 - lava_callback - INFO - Completed processing callback for node 69c72cf129015d0f95992350 INFO: 172.21.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42048 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:05:36,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:36,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47464 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:05:59,018 - lava_callback - INFO - Received callback for node 69c4dac6590270ad7b6da38f INFO: 172.21.0.1:47480 - "POST /node/69c4dac6590270ad7b6da38f HTTP/1.0" 202 Accepted 2026-03-28 07:05:59,023 - lava_callback - INFO - Starting async job submit for node 69c4dac6590270ad7b6da38f 2026-03-28 07:05:59,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:59,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dac6590270ad7b6da38f HTTP/1.1" 200 None 2026-03-28 07:05:59,051 - lava_callback - DEBUG - Processing node 69c4dac6590270ad7b6da38f: baseline-arm-clabbe 2026-03-28 07:05:59,052 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dac6590270ad7b6da38f 2026-03-28 07:05:59,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:59,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dac6590270ad7b6da38f HTTP/1.1" 200 None 2026-03-28 07:05:59,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:59,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:05:59,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:59,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dac6590270ad7b6da38f HTTP/1.1" 200 None 2026-03-28 07:05:59,109 - lava_callback - ERROR - Error processing callback for node 69c4dac6590270ad7b6da38f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39768 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:06:39,039 - lava_callback - INFO - Received callback for node 69c4daeb590270ad7b6da39c INFO: 172.21.0.1:39776 - "POST /node/69c4daeb590270ad7b6da39c HTTP/1.0" 202 Accepted 2026-03-28 07:06:39,045 - lava_callback - INFO - Starting async job submit for node 69c4daeb590270ad7b6da39c 2026-03-28 07:06:39,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:06:39,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4daeb590270ad7b6da39c HTTP/1.1" 200 None 2026-03-28 07:06:39,065 - lava_callback - DEBUG - Processing node 69c4daeb590270ad7b6da39c: baseline-arm-clabbe 2026-03-28 07:06:39,065 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4daeb590270ad7b6da39c 2026-03-28 07:06:39,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:06:39,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4daeb590270ad7b6da39c HTTP/1.1" 200 None 2026-03-28 07:06:39,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:06:39,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:06:39,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:06:39,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4daeb590270ad7b6da39c HTTP/1.1" 200 None 2026-03-28 07:06:39,129 - lava_callback - ERROR - Error processing callback for node 69c4daeb590270ad7b6da39c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34172 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:07:19,059 - lava_callback - INFO - Received callback for node 69c4db11590270ad7b6da3ab INFO: 172.21.0.1:34180 - "POST /node/69c4db11590270ad7b6da3ab HTTP/1.0" 202 Accepted 2026-03-28 07:07:19,064 - lava_callback - INFO - Starting async job submit for node 69c4db11590270ad7b6da3ab 2026-03-28 07:07:19,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:19,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db11590270ad7b6da3ab HTTP/1.1" 200 None 2026-03-28 07:07:19,087 - lava_callback - DEBUG - Processing node 69c4db11590270ad7b6da3ab: baseline-arm-clabbe 2026-03-28 07:07:19,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4db11590270ad7b6da3ab 2026-03-28 07:07:19,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:19,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db11590270ad7b6da3ab HTTP/1.1" 200 None 2026-03-28 07:07:19,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:19,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:07:19,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:19,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4db11590270ad7b6da3ab HTTP/1.1" 200 None 2026-03-28 07:07:19,144 - lava_callback - ERROR - Error processing callback for node 69c4db11590270ad7b6da3ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48054 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:07:39,069 - lava_callback - INFO - Received callback for node 69c4db37590270ad7b6da3bd INFO: 172.21.0.1:48070 - "POST /node/69c4db37590270ad7b6da3bd HTTP/1.0" 202 Accepted 2026-03-28 07:07:39,074 - lava_callback - INFO - Starting async job submit for node 69c4db37590270ad7b6da3bd 2026-03-28 07:07:39,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:39,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db37590270ad7b6da3bd HTTP/1.1" 200 None 2026-03-28 07:07:39,090 - lava_callback - DEBUG - Processing node 69c4db37590270ad7b6da3bd: baseline-arm-clabbe 2026-03-28 07:07:39,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4db37590270ad7b6da3bd 2026-03-28 07:07:39,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:39,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db37590270ad7b6da3bd HTTP/1.1" 200 None 2026-03-28 07:07:39,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:39,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:07:39,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:39,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4db37590270ad7b6da3bd HTTP/1.1" 200 None 2026-03-28 07:07:39,148 - lava_callback - ERROR - Error processing callback for node 69c4db37590270ad7b6da3bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53134 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:08:19,098 - lava_callback - INFO - Received callback for node 69c4db5c590270ad7b6da3ce INFO: 172.21.0.1:53146 - "POST /node/69c4db5c590270ad7b6da3ce HTTP/1.0" 202 Accepted 2026-03-28 07:08:19,104 - lava_callback - INFO - Starting async job submit for node 69c4db5c590270ad7b6da3ce 2026-03-28 07:08:19,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:19,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db5c590270ad7b6da3ce HTTP/1.1" 200 None 2026-03-28 07:08:19,120 - lava_callback - DEBUG - Processing node 69c4db5c590270ad7b6da3ce: baseline-arm-clabbe 2026-03-28 07:08:19,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4db5c590270ad7b6da3ce 2026-03-28 07:08:19,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:19,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db5c590270ad7b6da3ce HTTP/1.1" 200 None 2026-03-28 07:08:19,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:19,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:08:19,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:19,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4db5c590270ad7b6da3ce HTTP/1.1" 200 None 2026-03-28 07:08:19,204 - lava_callback - ERROR - Error processing callback for node 69c4db5c590270ad7b6da3ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60478 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:08:59,158 - lava_callback - INFO - Received callback for node 69c4db81590270ad7b6da3db INFO: 172.21.0.1:60480 - "POST /node/69c4db81590270ad7b6da3db HTTP/1.0" 202 Accepted 2026-03-28 07:08:59,164 - lava_callback - INFO - Starting async job submit for node 69c4db81590270ad7b6da3db 2026-03-28 07:08:59,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:59,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db81590270ad7b6da3db HTTP/1.1" 200 None 2026-03-28 07:08:59,179 - lava_callback - DEBUG - Processing node 69c4db81590270ad7b6da3db: baseline-arm-clabbe 2026-03-28 07:08:59,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4db81590270ad7b6da3db 2026-03-28 07:08:59,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:59,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db81590270ad7b6da3db HTTP/1.1" 200 None 2026-03-28 07:08:59,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:59,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:08:59,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:59,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4db81590270ad7b6da3db HTTP/1.1" 200 None 2026-03-28 07:08:59,241 - lava_callback - ERROR - Error processing callback for node 69c4db81590270ad7b6da3db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49378 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:09:39,267 - lava_callback - INFO - Received callback for node 69c4dba6590270ad7b6da3ec INFO: 172.21.0.1:49382 - "POST /node/69c4dba6590270ad7b6da3ec HTTP/1.0" 202 Accepted 2026-03-28 07:09:39,273 - lava_callback - INFO - Starting async job submit for node 69c4dba6590270ad7b6da3ec 2026-03-28 07:09:39,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:09:39,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dba6590270ad7b6da3ec HTTP/1.1" 200 None 2026-03-28 07:09:39,289 - lava_callback - DEBUG - Processing node 69c4dba6590270ad7b6da3ec: baseline-arm-clabbe 2026-03-28 07:09:39,289 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dba6590270ad7b6da3ec 2026-03-28 07:09:39,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:09:39,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dba6590270ad7b6da3ec HTTP/1.1" 200 None 2026-03-28 07:09:39,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:09:39,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:09:39,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:09:39,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dba6590270ad7b6da3ec HTTP/1.1" 200 None 2026-03-28 07:09:39,348 - lava_callback - ERROR - Error processing callback for node 69c4dba6590270ad7b6da3ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36386 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:10:19,169 - lava_callback - INFO - Received callback for node 69c4dbcb590270ad7b6da3fc INFO: 172.21.0.1:36394 - "POST /node/69c4dbcb590270ad7b6da3fc HTTP/1.0" 202 Accepted 2026-03-28 07:10:19,175 - lava_callback - INFO - Starting async job submit for node 69c4dbcb590270ad7b6da3fc 2026-03-28 07:10:19,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:10:19,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dbcb590270ad7b6da3fc HTTP/1.1" 200 None 2026-03-28 07:10:19,191 - lava_callback - DEBUG - Processing node 69c4dbcb590270ad7b6da3fc: baseline-arm-clabbe 2026-03-28 07:10:19,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dbcb590270ad7b6da3fc 2026-03-28 07:10:19,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:10:19,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dbcb590270ad7b6da3fc HTTP/1.1" 200 None 2026-03-28 07:10:19,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:10:19,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:10:19,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:10:19,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dbcb590270ad7b6da3fc HTTP/1.1" 200 None 2026-03-28 07:10:19,249 - lava_callback - ERROR - Error processing callback for node 69c4dbcb590270ad7b6da3fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50362 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:11:07,273 - lava_callback - INFO - Received callback for node 69c4dbf5590270ad7b6da417 INFO: 172.21.0.1:50378 - "POST /node/69c4dbf5590270ad7b6da417 HTTP/1.0" 202 Accepted 2026-03-28 07:11:07,279 - lava_callback - INFO - Starting async job submit for node 69c4dbf5590270ad7b6da417 2026-03-28 07:11:07,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:11:07,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dbf5590270ad7b6da417 HTTP/1.1" 200 None 2026-03-28 07:11:07,295 - lava_callback - DEBUG - Processing node 69c4dbf5590270ad7b6da417: baseline-arm64-baylibre 2026-03-28 07:11:07,295 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dbf5590270ad7b6da417 2026-03-28 07:11:07,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:11:07,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dbf5590270ad7b6da417 HTTP/1.1" 200 None 2026-03-28 07:11:07,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:11:07,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:11:07,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:11:07,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dbf5590270ad7b6da417 HTTP/1.1" 200 None 2026-03-28 07:11:07,353 - lava_callback - ERROR - Error processing callback for node 69c4dbf5590270ad7b6da417: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56842 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:12:06,985 - lava_callback - INFO - Received callback for node 69c4dc45590270ad7b6da41d INFO: 172.21.0.1:56848 - "POST /node/69c4dc45590270ad7b6da41d HTTP/1.0" 202 Accepted 2026-03-28 07:12:06,995 - lava_callback - INFO - Starting async job submit for node 69c4dc45590270ad7b6da41d 2026-03-28 07:12:06,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:12:07,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dc45590270ad7b6da41d HTTP/1.1" 200 None 2026-03-28 07:12:07,013 - lava_callback - DEBUG - Processing node 69c4dc45590270ad7b6da41d: baseline-arm64-baylibre 2026-03-28 07:12:07,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dc45590270ad7b6da41d 2026-03-28 07:12:07,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:12:07,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dc45590270ad7b6da41d HTTP/1.1" 200 None 2026-03-28 07:12:07,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:12:07,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:12:07,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:12:07,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dc45590270ad7b6da41d HTTP/1.1" 200 None 2026-03-28 07:12:07,106 - lava_callback - ERROR - Error processing callback for node 69c4dc45590270ad7b6da41d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48292 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:13:27,251 - lava_callback - INFO - Received callback for node 69c4dc94590270ad7b6da42b INFO: 172.21.0.1:48308 - "POST /node/69c4dc94590270ad7b6da42b HTTP/1.0" 202 Accepted 2026-03-28 07:13:27,256 - lava_callback - INFO - Starting async job submit for node 69c4dc94590270ad7b6da42b 2026-03-28 07:13:27,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:13:27,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dc94590270ad7b6da42b HTTP/1.1" 200 None 2026-03-28 07:13:27,272 - lava_callback - DEBUG - Processing node 69c4dc94590270ad7b6da42b: baseline-arm64-baylibre 2026-03-28 07:13:27,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dc94590270ad7b6da42b 2026-03-28 07:13:27,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:13:27,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dc94590270ad7b6da42b HTTP/1.1" 200 None 2026-03-28 07:13:27,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:13:27,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:13:27,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:13:27,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dc94590270ad7b6da42b HTTP/1.1" 200 None 2026-03-28 07:13:27,329 - lava_callback - ERROR - Error processing callback for node 69c4dc94590270ad7b6da42b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46262 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:15:07,188 - lava_callback - INFO - Received callback for node 69c4dce9590270ad7b6da449 INFO: 172.21.0.1:46266 - "POST /node/69c4dce9590270ad7b6da449 HTTP/1.0" 202 Accepted 2026-03-28 07:15:07,194 - lava_callback - INFO - Starting async job submit for node 69c4dce9590270ad7b6da449 2026-03-28 07:15:07,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:15:07,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dce9590270ad7b6da449 HTTP/1.1" 200 None 2026-03-28 07:15:07,209 - lava_callback - DEBUG - Processing node 69c4dce9590270ad7b6da449: baseline-arm64-baylibre 2026-03-28 07:15:07,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dce9590270ad7b6da449 2026-03-28 07:15:07,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:15:07,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dce9590270ad7b6da449 HTTP/1.1" 200 None 2026-03-28 07:15:07,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:15:07,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:15:07,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:15:07,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dce9590270ad7b6da449 HTTP/1.1" 200 None 2026-03-28 07:15:07,264 - lava_callback - ERROR - Error processing callback for node 69c4dce9590270ad7b6da449: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38738 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:16:27,353 - lava_callback - INFO - Received callback for node 69c4dd39590270ad7b6da46d INFO: 172.21.0.1:38750 - "POST /node/69c4dd39590270ad7b6da46d HTTP/1.0" 202 Accepted 2026-03-28 07:16:27,358 - lava_callback - INFO - Starting async job submit for node 69c4dd39590270ad7b6da46d 2026-03-28 07:16:27,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:16:27,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dd39590270ad7b6da46d HTTP/1.1" 200 None 2026-03-28 07:16:27,373 - lava_callback - DEBUG - Processing node 69c4dd39590270ad7b6da46d: baseline-arm64-baylibre 2026-03-28 07:16:27,374 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dd39590270ad7b6da46d 2026-03-28 07:16:27,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:16:27,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dd39590270ad7b6da46d HTTP/1.1" 200 None 2026-03-28 07:16:27,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:16:27,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:16:27,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:16:27,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dd39590270ad7b6da46d HTTP/1.1" 200 None 2026-03-28 07:16:27,437 - lava_callback - ERROR - Error processing callback for node 69c4dd39590270ad7b6da46d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56404 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:24:27,937 - lava_callback - INFO - Received callback for node 69c4df1d590270ad7b6da4b7 INFO: 172.21.0.1:56414 - "POST /node/69c4df1d590270ad7b6da4b7 HTTP/1.0" 202 Accepted 2026-03-28 07:24:27,943 - lava_callback - INFO - Starting async job submit for node 69c4df1d590270ad7b6da4b7 2026-03-28 07:24:27,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:24:27,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4df1d590270ad7b6da4b7 HTTP/1.1" 200 None 2026-03-28 07:24:27,959 - lava_callback - DEBUG - Processing node 69c4df1d590270ad7b6da4b7: baseline-arm64-baylibre 2026-03-28 07:24:27,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4df1d590270ad7b6da4b7 2026-03-28 07:24:27,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:24:27,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4df1d590270ad7b6da4b7 HTTP/1.1" 200 None 2026-03-28 07:24:27,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:24:28,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:24:28,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:24:28,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4df1d590270ad7b6da4b7 HTTP/1.1" 200 None 2026-03-28 07:24:28,191 - lava_callback - ERROR - Error processing callback for node 69c4df1d590270ad7b6da4b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44558 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:25:47,729 - lava_callback - INFO - Received callback for node 69c4df68590270ad7b6da4bc INFO: 172.21.0.1:44572 - "POST /node/69c4df68590270ad7b6da4bc HTTP/1.0" 202 Accepted 2026-03-28 07:25:47,735 - lava_callback - INFO - Starting async job submit for node 69c4df68590270ad7b6da4bc 2026-03-28 07:25:47,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:25:47,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4df68590270ad7b6da4bc HTTP/1.1" 200 None 2026-03-28 07:25:47,752 - lava_callback - DEBUG - Processing node 69c4df68590270ad7b6da4bc: baseline-arm64-baylibre 2026-03-28 07:25:47,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4df68590270ad7b6da4bc 2026-03-28 07:25:47,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:25:47,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4df68590270ad7b6da4bc HTTP/1.1" 200 None 2026-03-28 07:25:47,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:25:47,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:25:47,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:25:47,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4df68590270ad7b6da4bc HTTP/1.1" 200 None 2026-03-28 07:25:47,810 - lava_callback - ERROR - Error processing callback for node 69c4df68590270ad7b6da4bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60614 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:27:07,845 - lava_callback - INFO - Received callback for node 69c4dfbb590270ad7b6da4c5 INFO: 172.21.0.1:60616 - "POST /node/69c4dfbb590270ad7b6da4c5 HTTP/1.0" 202 Accepted 2026-03-28 07:27:07,851 - lava_callback - INFO - Starting async job submit for node 69c4dfbb590270ad7b6da4c5 2026-03-28 07:27:07,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:27:07,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dfbb590270ad7b6da4c5 HTTP/1.1" 200 None 2026-03-28 07:27:07,866 - lava_callback - DEBUG - Processing node 69c4dfbb590270ad7b6da4c5: baseline-arm64-baylibre 2026-03-28 07:27:07,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dfbb590270ad7b6da4c5 2026-03-28 07:27:07,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:27:07,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dfbb590270ad7b6da4c5 HTTP/1.1" 200 None 2026-03-28 07:27:07,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:27:07,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:27:07,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:27:07,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dfbb590270ad7b6da4c5 HTTP/1.1" 200 None 2026-03-28 07:27:07,918 - lava_callback - ERROR - Error processing callback for node 69c4dfbb590270ad7b6da4c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58176 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:29:59,968 - lava_callback - INFO - Received callback for node 69c4e065590270ad7b6da4db INFO: 172.21.0.1:42354 - "POST /node/69c4e065590270ad7b6da4db HTTP/1.0" 202 Accepted 2026-03-28 07:29:59,974 - lava_callback - INFO - Starting async job submit for node 69c4e065590270ad7b6da4db 2026-03-28 07:29:59,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:29:59,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e065590270ad7b6da4db HTTP/1.1" 200 None 2026-03-28 07:29:59,989 - lava_callback - DEBUG - Processing node 69c4e065590270ad7b6da4db: baseline-arm64-clabbe 2026-03-28 07:29:59,990 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e065590270ad7b6da4db 2026-03-28 07:29:59,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:30:00,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e065590270ad7b6da4db HTTP/1.1" 200 None 2026-03-28 07:30:00,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:30:00,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:30:00,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:30:00,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e065590270ad7b6da4db HTTP/1.1" 200 None 2026-03-28 07:30:00,075 - lava_callback - ERROR - Error processing callback for node 69c4e065590270ad7b6da4db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59080 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:31:20,026 - lava_callback - INFO - Received callback for node 69c4e0b7590270ad7b6da4e2 INFO: 172.21.0.1:39178 - "POST /node/69c4e0b7590270ad7b6da4e2 HTTP/1.0" 202 Accepted 2026-03-28 07:31:20,032 - lava_callback - INFO - Starting async job submit for node 69c4e0b7590270ad7b6da4e2 2026-03-28 07:31:20,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:31:20,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e0b7590270ad7b6da4e2 HTTP/1.1" 200 None 2026-03-28 07:31:20,048 - lava_callback - DEBUG - Processing node 69c4e0b7590270ad7b6da4e2: baseline-arm64-clabbe 2026-03-28 07:31:20,048 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e0b7590270ad7b6da4e2 2026-03-28 07:31:20,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:31:20,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e0b7590270ad7b6da4e2 HTTP/1.1" 200 None 2026-03-28 07:31:20,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:31:20,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:31:20,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:31:20,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e0b7590270ad7b6da4e2 HTTP/1.1" 200 None 2026-03-28 07:31:20,106 - lava_callback - ERROR - Error processing callback for node 69c4e0b7590270ad7b6da4e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32934 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:32:40,100 - lava_callback - INFO - Received callback for node 69c4e107590270ad7b6da4ee INFO: 172.21.0.1:47778 - "POST /node/69c4e107590270ad7b6da4ee HTTP/1.0" 202 Accepted 2026-03-28 07:32:40,106 - lava_callback - INFO - Starting async job submit for node 69c4e107590270ad7b6da4ee 2026-03-28 07:32:40,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:32:40,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e107590270ad7b6da4ee HTTP/1.1" 200 None 2026-03-28 07:32:40,122 - lava_callback - DEBUG - Processing node 69c4e107590270ad7b6da4ee: baseline-arm64-clabbe 2026-03-28 07:32:40,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e107590270ad7b6da4ee 2026-03-28 07:32:40,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:32:40,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e107590270ad7b6da4ee HTTP/1.1" 200 None 2026-03-28 07:32:40,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:32:40,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:32:40,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:32:40,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e107590270ad7b6da4ee HTTP/1.1" 200 None 2026-03-28 07:32:40,187 - lava_callback - ERROR - Error processing callback for node 69c4e107590270ad7b6da4ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36362 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:34:00,176 - lava_callback - INFO - Received callback for node 69c4e158590270ad7b6da4fc INFO: 172.21.0.1:44942 - "POST /node/69c4e158590270ad7b6da4fc HTTP/1.0" 202 Accepted 2026-03-28 07:34:00,181 - lava_callback - INFO - Starting async job submit for node 69c4e158590270ad7b6da4fc 2026-03-28 07:34:00,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:34:00,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e158590270ad7b6da4fc HTTP/1.1" 200 None 2026-03-28 07:34:00,198 - lava_callback - DEBUG - Processing node 69c4e158590270ad7b6da4fc: baseline-arm64-clabbe 2026-03-28 07:34:00,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e158590270ad7b6da4fc 2026-03-28 07:34:00,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:34:00,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e158590270ad7b6da4fc HTTP/1.1" 200 None 2026-03-28 07:34:00,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:34:00,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:34:00,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:34:00,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e158590270ad7b6da4fc HTTP/1.1" 200 None 2026-03-28 07:34:00,257 - lava_callback - ERROR - Error processing callback for node 69c4e158590270ad7b6da4fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41784 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:35:20,209 - lava_callback - INFO - Received callback for node 69c4e1a8590270ad7b6da503 INFO: 172.21.0.1:37904 - "POST /node/69c4e1a8590270ad7b6da503 HTTP/1.0" 202 Accepted 2026-03-28 07:35:20,215 - lava_callback - INFO - Starting async job submit for node 69c4e1a8590270ad7b6da503 2026-03-28 07:35:20,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:35:20,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e1a8590270ad7b6da503 HTTP/1.1" 200 None 2026-03-28 07:35:20,237 - lava_callback - DEBUG - Processing node 69c4e1a8590270ad7b6da503: baseline-arm64-clabbe 2026-03-28 07:35:20,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e1a8590270ad7b6da503 2026-03-28 07:35:20,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:35:20,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e1a8590270ad7b6da503 HTTP/1.1" 200 None 2026-03-28 07:35:20,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:35:20,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:35:20,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:35:20,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e1a8590270ad7b6da503 HTTP/1.1" 200 None 2026-03-28 07:35:20,290 - lava_callback - ERROR - Error processing callback for node 69c4e1a8590270ad7b6da503: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52094 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:36:40,242 - lava_callback - INFO - Received callback for node 69c4e1fa590270ad7b6da50f INFO: 172.21.0.1:34124 - "POST /node/69c4e1fa590270ad7b6da50f HTTP/1.0" 202 Accepted 2026-03-28 07:36:40,248 - lava_callback - INFO - Starting async job submit for node 69c4e1fa590270ad7b6da50f 2026-03-28 07:36:40,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:36:40,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e1fa590270ad7b6da50f HTTP/1.1" 200 None 2026-03-28 07:36:40,275 - lava_callback - DEBUG - Processing node 69c4e1fa590270ad7b6da50f: baseline-arm64-clabbe 2026-03-28 07:36:40,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e1fa590270ad7b6da50f 2026-03-28 07:36:40,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:36:40,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e1fa590270ad7b6da50f HTTP/1.1" 200 None 2026-03-28 07:36:40,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:36:40,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:36:40,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:36:40,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e1fa590270ad7b6da50f HTTP/1.1" 200 None 2026-03-28 07:36:40,330 - lava_callback - ERROR - Error processing callback for node 69c4e1fa590270ad7b6da50f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51992 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:52000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57536 - "GET /.well-known/security.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:37288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46466 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:37:48,499 - lava_callback - INFO - Received callback for node 69c4e242590270ad7b6da51b INFO: 172.21.0.1:46482 - "POST /node/69c4e242590270ad7b6da51b HTTP/1.0" 202 Accepted 2026-03-28 07:37:48,505 - lava_callback - INFO - Starting async job submit for node 69c4e242590270ad7b6da51b 2026-03-28 07:37:48,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:37:48,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e242590270ad7b6da51b HTTP/1.1" 200 None 2026-03-28 07:37:48,520 - lava_callback - DEBUG - Processing node 69c4e242590270ad7b6da51b: baseline-arm64-baylibre 2026-03-28 07:37:48,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e242590270ad7b6da51b 2026-03-28 07:37:48,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:37:48,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e242590270ad7b6da51b HTTP/1.1" 200 None 2026-03-28 07:37:48,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:37:48,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:37:48,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:37:48,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e242590270ad7b6da51b HTTP/1.1" 200 None 2026-03-28 07:37:48,589 - lava_callback - ERROR - Error processing callback for node 69c4e242590270ad7b6da51b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33798 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:39:20,376 - lava_callback - INFO - Received callback for node 69c4e29d590270ad7b6da528 INFO: 172.21.0.1:45866 - "POST /node/69c4e29d590270ad7b6da528 HTTP/1.0" 202 Accepted 2026-03-28 07:39:20,381 - lava_callback - INFO - Starting async job submit for node 69c4e29d590270ad7b6da528 2026-03-28 07:39:20,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:39:20,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e29d590270ad7b6da528 HTTP/1.1" 200 None 2026-03-28 07:39:20,399 - lava_callback - DEBUG - Processing node 69c4e29d590270ad7b6da528: baseline-arm64-clabbe 2026-03-28 07:39:20,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e29d590270ad7b6da528 2026-03-28 07:39:20,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:39:20,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e29d590270ad7b6da528 HTTP/1.1" 200 None 2026-03-28 07:39:20,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:39:20,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:39:20,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:39:20,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e29d590270ad7b6da528 HTTP/1.1" 200 None 2026-03-28 07:39:20,489 - lava_callback - ERROR - Error processing callback for node 69c4e29d590270ad7b6da528: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39284 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:40:28,546 - lava_callback - INFO - Received callback for node 69c4e2e5590270ad7b6da530 INFO: 172.21.0.1:39300 - "POST /node/69c4e2e5590270ad7b6da530 HTTP/1.0" 202 Accepted 2026-03-28 07:40:28,551 - lava_callback - INFO - Starting async job submit for node 69c4e2e5590270ad7b6da530 2026-03-28 07:40:28,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:40:28,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e2e5590270ad7b6da530 HTTP/1.1" 200 None 2026-03-28 07:40:28,567 - lava_callback - DEBUG - Processing node 69c4e2e5590270ad7b6da530: baseline-arm64-baylibre 2026-03-28 07:40:28,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e2e5590270ad7b6da530 2026-03-28 07:40:28,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:40:28,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e2e5590270ad7b6da530 HTTP/1.1" 200 None 2026-03-28 07:40:28,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:40:28,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:40:28,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:40:28,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e2e5590270ad7b6da530 HTTP/1.1" 200 None 2026-03-28 07:40:28,648 - lava_callback - ERROR - Error processing callback for node 69c4e2e5590270ad7b6da530: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45090 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:42:00,476 - lava_callback - INFO - Received callback for node 69c4e33e590270ad7b6da540 INFO: 172.21.0.1:57778 - "POST /node/69c4e33e590270ad7b6da540 HTTP/1.0" 202 Accepted 2026-03-28 07:42:00,481 - lava_callback - INFO - Starting async job submit for node 69c4e33e590270ad7b6da540 2026-03-28 07:42:00,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:00,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e33e590270ad7b6da540 HTTP/1.1" 200 None 2026-03-28 07:42:00,497 - lava_callback - DEBUG - Processing node 69c4e33e590270ad7b6da540: baseline-arm64-clabbe 2026-03-28 07:42:00,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e33e590270ad7b6da540 2026-03-28 07:42:00,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:00,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e33e590270ad7b6da540 HTTP/1.1" 200 None 2026-03-28 07:42:00,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:00,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:42:00,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:00,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e33e590270ad7b6da540 HTTP/1.1" 200 None 2026-03-28 07:42:00,549 - lava_callback - ERROR - Error processing callback for node 69c4e33e590270ad7b6da540: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36130 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:42:20,833 - lava_callback - INFO - Received callback for node 69c7802e29015d0f959933bd INFO: 172.21.0.1:47500 - "POST /node/69c7802e29015d0f959933bd HTTP/1.0" 202 Accepted 2026-03-28 07:42:20,840 - lava_callback - INFO - Starting async job submit for node 69c7802e29015d0f959933bd 2026-03-28 07:42:20,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:20,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7802e29015d0f959933bd HTTP/1.1" 200 None 2026-03-28 07:42:20,904 - lava_callback - DEBUG - Processing node 69c7802e29015d0f959933bd: baseline-nfs-arm64 2026-03-28 07:42:21,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:42:21,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:42:21,614 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7802e29015d0f959933bd/log.txt.gz 2026-03-28 07:42:21,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:42:22,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:42:22,101 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7802e29015d0f959933bd/lava-logs.yaml 2026-03-28 07:42:22,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:42:22,294 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:42:22,296 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7802e29015d0f959933bd/lava_callback.json.gz 2026-03-28 07:42:22,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:22,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7802e29015d0f959933bd HTTP/1.1" 200 None 2026-03-28 07:42:22,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:22,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 07:42:22,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:22,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7802e29015d0f959933bd HTTP/1.1" 200 None 2026-03-28 07:42:22,436 - lava_callback - INFO - Completed processing callback for node 69c7802e29015d0f959933bd INFO: 172.21.0.1:47510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33976 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:42:36,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:36,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43330 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:43:20,533 - lava_callback - INFO - Received callback for node 69c4e38f590270ad7b6da54a INFO: 172.21.0.1:52006 - "POST /node/69c4e38f590270ad7b6da54a HTTP/1.0" 202 Accepted 2026-03-28 07:43:20,538 - lava_callback - INFO - Starting async job submit for node 69c4e38f590270ad7b6da54a 2026-03-28 07:43:20,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:43:20,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e38f590270ad7b6da54a HTTP/1.1" 200 None 2026-03-28 07:43:20,554 - lava_callback - DEBUG - Processing node 69c4e38f590270ad7b6da54a: baseline-arm64-clabbe 2026-03-28 07:43:20,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e38f590270ad7b6da54a 2026-03-28 07:43:20,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:43:20,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e38f590270ad7b6da54a HTTP/1.1" 200 None 2026-03-28 07:43:20,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:43:20,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:43:20,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:43:20,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e38f590270ad7b6da54a HTTP/1.1" 200 None 2026-03-28 07:43:20,611 - lava_callback - ERROR - Error processing callback for node 69c4e38f590270ad7b6da54a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48056 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:44:28,820 - lava_callback - INFO - Received callback for node 69c4e3d9590270ad7b6da553 INFO: 172.21.0.1:48072 - "POST /node/69c4e3d9590270ad7b6da553 HTTP/1.0" 202 Accepted 2026-03-28 07:44:28,826 - lava_callback - INFO - Starting async job submit for node 69c4e3d9590270ad7b6da553 2026-03-28 07:44:28,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:44:28,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e3d9590270ad7b6da553 HTTP/1.1" 200 None 2026-03-28 07:44:28,841 - lava_callback - DEBUG - Processing node 69c4e3d9590270ad7b6da553: baseline-arm64-baylibre 2026-03-28 07:44:28,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e3d9590270ad7b6da553 2026-03-28 07:44:28,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:44:28,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e3d9590270ad7b6da553 HTTP/1.1" 200 None 2026-03-28 07:44:28,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:44:28,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:44:28,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:44:28,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e3d9590270ad7b6da553 HTTP/1.1" 200 None 2026-03-28 07:44:28,903 - lava_callback - ERROR - Error processing callback for node 69c4e3d9590270ad7b6da553: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54664 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:45:48,846 - lava_callback - INFO - Received callback for node 69c4e429590270ad7b6da560 INFO: 172.21.0.1:54670 - "POST /node/69c4e429590270ad7b6da560 HTTP/1.0" 202 Accepted 2026-03-28 07:45:48,851 - lava_callback - INFO - Starting async job submit for node 69c4e429590270ad7b6da560 2026-03-28 07:45:48,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:45:48,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e429590270ad7b6da560 HTTP/1.1" 200 None 2026-03-28 07:45:48,867 - lava_callback - DEBUG - Processing node 69c4e429590270ad7b6da560: baseline-arm64-baylibre 2026-03-28 07:45:48,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e429590270ad7b6da560 2026-03-28 07:45:48,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:45:48,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e429590270ad7b6da560 HTTP/1.1" 200 None 2026-03-28 07:45:48,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:45:48,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:45:48,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:45:48,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e429590270ad7b6da560 HTTP/1.1" 200 None 2026-03-28 07:45:48,935 - lava_callback - ERROR - Error processing callback for node 69c4e429590270ad7b6da560: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41826 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:47:20,686 - lava_callback - INFO - Received callback for node 69c4e47f590270ad7b6da56b INFO: 172.21.0.1:36864 - "POST /node/69c4e47f590270ad7b6da56b HTTP/1.0" 202 Accepted 2026-03-28 07:47:20,692 - lava_callback - INFO - Starting async job submit for node 69c4e47f590270ad7b6da56b 2026-03-28 07:47:20,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:47:20,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e47f590270ad7b6da56b HTTP/1.1" 200 None 2026-03-28 07:47:20,707 - lava_callback - DEBUG - Processing node 69c4e47f590270ad7b6da56b: baseline-arm64-clabbe 2026-03-28 07:47:20,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e47f590270ad7b6da56b 2026-03-28 07:47:20,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:47:20,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e47f590270ad7b6da56b HTTP/1.1" 200 None 2026-03-28 07:47:20,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:47:20,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:47:20,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:47:20,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e47f590270ad7b6da56b HTTP/1.1" 200 None 2026-03-28 07:47:20,818 - lava_callback - ERROR - Error processing callback for node 69c4e47f590270ad7b6da56b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60716 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:48:40,770 - lava_callback - INFO - Received callback for node 69c4e4cf590270ad7b6da575 INFO: 172.21.0.1:38476 - "POST /node/69c4e4cf590270ad7b6da575 HTTP/1.0" 202 Accepted 2026-03-28 07:48:40,777 - lava_callback - INFO - Starting async job submit for node 69c4e4cf590270ad7b6da575 2026-03-28 07:48:40,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:48:40,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e4cf590270ad7b6da575 HTTP/1.1" 200 None 2026-03-28 07:48:40,795 - lava_callback - DEBUG - Processing node 69c4e4cf590270ad7b6da575: baseline-arm64-clabbe 2026-03-28 07:48:40,795 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e4cf590270ad7b6da575 2026-03-28 07:48:40,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:48:40,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e4cf590270ad7b6da575 HTTP/1.1" 200 None 2026-03-28 07:48:40,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:48:40,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:48:40,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:48:40,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e4cf590270ad7b6da575 HTTP/1.1" 200 None 2026-03-28 07:48:40,864 - lava_callback - ERROR - Error processing callback for node 69c4e4cf590270ad7b6da575: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33606 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:49:49,235 - lava_callback - INFO - Received callback for node 69c4e519590270ad7b6da581 INFO: 172.21.0.1:33612 - "POST /node/69c4e519590270ad7b6da581 HTTP/1.0" 202 Accepted 2026-03-28 07:49:49,241 - lava_callback - INFO - Starting async job submit for node 69c4e519590270ad7b6da581 2026-03-28 07:49:49,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:49:49,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e519590270ad7b6da581 HTTP/1.1" 200 None 2026-03-28 07:49:49,257 - lava_callback - DEBUG - Processing node 69c4e519590270ad7b6da581: baseline-arm64-baylibre 2026-03-28 07:49:49,258 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e519590270ad7b6da581 2026-03-28 07:49:49,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:49:49,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e519590270ad7b6da581 HTTP/1.1" 200 None 2026-03-28 07:49:49,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:49:49,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:49:49,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:49:49,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e519590270ad7b6da581 HTTP/1.1" 200 None 2026-03-28 07:49:49,314 - lava_callback - ERROR - Error processing callback for node 69c4e519590270ad7b6da581: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39154 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:51:20,909 - lava_callback - INFO - Received callback for node 69c4e571590270ad7b6da58d INFO: 172.21.0.1:37472 - "POST /node/69c4e571590270ad7b6da58d HTTP/1.0" 202 Accepted 2026-03-28 07:51:20,915 - lava_callback - INFO - Starting async job submit for node 69c4e571590270ad7b6da58d 2026-03-28 07:51:20,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:51:20,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e571590270ad7b6da58d HTTP/1.1" 200 None 2026-03-28 07:51:20,933 - lava_callback - DEBUG - Processing node 69c4e571590270ad7b6da58d: baseline-arm64-clabbe 2026-03-28 07:51:20,933 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e571590270ad7b6da58d 2026-03-28 07:51:20,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:51:20,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e571590270ad7b6da58d HTTP/1.1" 200 None 2026-03-28 07:51:20,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:51:20,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:51:20,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:51:20,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e571590270ad7b6da58d HTTP/1.1" 200 None 2026-03-28 07:51:20,991 - lava_callback - ERROR - Error processing callback for node 69c4e571590270ad7b6da58d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:52:40,994 - lava_callback - INFO - Received callback for node 69c4e5c2590270ad7b6da59a INFO: 172.21.0.1:52426 - "POST /node/69c4e5c2590270ad7b6da59a HTTP/1.0" 202 Accepted 2026-03-28 07:52:41,000 - lava_callback - INFO - Starting async job submit for node 69c4e5c2590270ad7b6da59a 2026-03-28 07:52:41,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:52:41,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e5c2590270ad7b6da59a HTTP/1.1" 200 None 2026-03-28 07:52:41,017 - lava_callback - DEBUG - Processing node 69c4e5c2590270ad7b6da59a: baseline-arm64-clabbe 2026-03-28 07:52:41,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e5c2590270ad7b6da59a 2026-03-28 07:52:41,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:52:41,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e5c2590270ad7b6da59a HTTP/1.1" 200 None 2026-03-28 07:52:41,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:52:41,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:52:41,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:52:41,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e5c2590270ad7b6da59a HTTP/1.1" 200 None 2026-03-28 07:52:41,121 - lava_callback - ERROR - Error processing callback for node 69c4e5c2590270ad7b6da59a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49574 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:54:01,084 - lava_callback - INFO - Received callback for node 69c4e611590270ad7b6da5a6 INFO: 172.21.0.1:39922 - "POST /node/69c4e611590270ad7b6da5a6 HTTP/1.0" 202 Accepted 2026-03-28 07:54:01,089 - lava_callback - INFO - Starting async job submit for node 69c4e611590270ad7b6da5a6 2026-03-28 07:54:01,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:54:01,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e611590270ad7b6da5a6 HTTP/1.1" 200 None 2026-03-28 07:54:01,105 - lava_callback - DEBUG - Processing node 69c4e611590270ad7b6da5a6: baseline-arm64-clabbe 2026-03-28 07:54:01,106 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e611590270ad7b6da5a6 2026-03-28 07:54:01,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:54:01,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e611590270ad7b6da5a6 HTTP/1.1" 200 None 2026-03-28 07:54:01,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:54:01,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:54:01,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:54:01,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e611590270ad7b6da5a6 HTTP/1.1" 200 None 2026-03-28 07:54:01,157 - lava_callback - ERROR - Error processing callback for node 69c4e611590270ad7b6da5a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:55:21,053 - lava_callback - INFO - Received callback for node 69c4e662590270ad7b6da5b1 INFO: 172.21.0.1:46578 - "POST /node/69c4e662590270ad7b6da5b1 HTTP/1.0" 202 Accepted 2026-03-28 07:55:21,059 - lava_callback - INFO - Starting async job submit for node 69c4e662590270ad7b6da5b1 2026-03-28 07:55:21,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:55:21,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e662590270ad7b6da5b1 HTTP/1.1" 200 None 2026-03-28 07:55:21,075 - lava_callback - DEBUG - Processing node 69c4e662590270ad7b6da5b1: baseline-arm64-clabbe 2026-03-28 07:55:21,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e662590270ad7b6da5b1 2026-03-28 07:55:21,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:55:21,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e662590270ad7b6da5b1 HTTP/1.1" 200 None 2026-03-28 07:55:21,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:55:21,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:55:21,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:55:21,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e662590270ad7b6da5b1 HTTP/1.1" 200 None 2026-03-28 07:55:21,141 - lava_callback - ERROR - Error processing callback for node 69c4e662590270ad7b6da5b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42162 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:56:41,170 - lava_callback - INFO - Received callback for node 69c4e6b5590270ad7b6da5b8 INFO: 172.21.0.1:53544 - "POST /node/69c4e6b5590270ad7b6da5b8 HTTP/1.0" 202 Accepted 2026-03-28 07:56:41,176 - lava_callback - INFO - Starting async job submit for node 69c4e6b5590270ad7b6da5b8 2026-03-28 07:56:41,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:56:41,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e6b5590270ad7b6da5b8 HTTP/1.1" 200 None 2026-03-28 07:56:41,213 - lava_callback - DEBUG - Processing node 69c4e6b5590270ad7b6da5b8: baseline-arm64-clabbe 2026-03-28 07:56:41,214 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e6b5590270ad7b6da5b8 2026-03-28 07:56:41,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:56:41,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e6b5590270ad7b6da5b8 HTTP/1.1" 200 None 2026-03-28 07:56:41,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:56:41,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:56:41,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:56:41,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e6b5590270ad7b6da5b8 HTTP/1.1" 200 None 2026-03-28 07:56:41,293 - lava_callback - ERROR - Error processing callback for node 69c4e6b5590270ad7b6da5b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48432 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:57:49,486 - lava_callback - INFO - Received callback for node 69c4e6fb590270ad7b6da5c4 INFO: 172.21.0.1:48440 - "POST /node/69c4e6fb590270ad7b6da5c4 HTTP/1.0" 202 Accepted 2026-03-28 07:57:49,492 - lava_callback - INFO - Starting async job submit for node 69c4e6fb590270ad7b6da5c4 2026-03-28 07:57:49,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:57:49,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e6fb590270ad7b6da5c4 HTTP/1.1" 200 None 2026-03-28 07:57:49,511 - lava_callback - DEBUG - Processing node 69c4e6fb590270ad7b6da5c4: baseline-arm64-baylibre 2026-03-28 07:57:49,511 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e6fb590270ad7b6da5c4 2026-03-28 07:57:49,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:57:49,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e6fb590270ad7b6da5c4 HTTP/1.1" 200 None 2026-03-28 07:57:49,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:57:49,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 07:57:49,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:57:49,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e6fb590270ad7b6da5c4 HTTP/1.1" 200 None 2026-03-28 07:57:49,567 - lava_callback - ERROR - Error processing callback for node 69c4e6fb590270ad7b6da5c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50460 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:59:21,278 - lava_callback - INFO - Received callback for node 69c4e755590270ad7b6da5cf INFO: 172.21.0.1:50416 - "POST /node/69c4e755590270ad7b6da5cf HTTP/1.0" 202 Accepted 2026-03-28 07:59:21,283 - lava_callback - INFO - Starting async job submit for node 69c4e755590270ad7b6da5cf 2026-03-28 07:59:21,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:59:21,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e755590270ad7b6da5cf HTTP/1.1" 200 None 2026-03-28 07:59:21,299 - lava_callback - DEBUG - Processing node 69c4e755590270ad7b6da5cf: baseline-arm64-clabbe 2026-03-28 07:59:21,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e755590270ad7b6da5cf 2026-03-28 07:59:21,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:59:21,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e755590270ad7b6da5cf HTTP/1.1" 200 None 2026-03-28 07:59:21,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:59:21,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:59:21,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:59:21,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e755590270ad7b6da5cf HTTP/1.1" 200 None 2026-03-28 07:59:21,357 - lava_callback - ERROR - Error processing callback for node 69c4e755590270ad7b6da5cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41042 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:00:45,177 - lava_callback - INFO - Received callback for node 69c72d0329015d0f95992389 INFO: 172.21.0.1:41052 - "POST /node/69c72d0329015d0f95992389 HTTP/1.0" 202 Accepted 2026-03-28 08:00:45,183 - lava_callback - INFO - Starting async job submit for node 69c72d0329015d0f95992389 2026-03-28 08:00:45,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:00:45,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0329015d0f95992389 HTTP/1.1" 200 None 2026-03-28 08:00:45,245 - lava_callback - DEBUG - Processing node 69c72d0329015d0f95992389: baseline-arm64-broonie 2026-03-28 08:00:45,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:00:45,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:00:45,628 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0329015d0f95992389/log.txt.gz 2026-03-28 08:00:45,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:00:45,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:00:45,975 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0329015d0f95992389/lava-logs.yaml 2026-03-28 08:00:45,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:00:46,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:00:46,308 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0329015d0f95992389/lava_callback.json.gz 2026-03-28 08:00:46,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:00:46,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0329015d0f95992389 HTTP/1.1" 200 None 2026-03-28 08:00:46,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:00:46,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 08:00:46,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:00:46,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0329015d0f95992389 HTTP/1.1" 200 None 2026-03-28 08:00:46,394 - lava_callback - INFO - Completed processing callback for node 69c72d0329015d0f95992389 INFO: 172.21.0.1:41060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35034 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:01:01,399 - lava_callback - INFO - Received callback for node 69c4e7a7590270ad7b6da5d9 INFO: 172.21.0.1:33790 - "POST /node/69c4e7a7590270ad7b6da5d9 HTTP/1.0" 202 Accepted 2026-03-28 08:01:01,405 - lava_callback - INFO - Starting async job submit for node 69c4e7a7590270ad7b6da5d9 2026-03-28 08:01:01,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:01:01,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e7a7590270ad7b6da5d9 HTTP/1.1" 200 None 2026-03-28 08:01:01,420 - lava_callback - DEBUG - Processing node 69c4e7a7590270ad7b6da5d9: baseline-arm64-clabbe 2026-03-28 08:01:01,421 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e7a7590270ad7b6da5d9 2026-03-28 08:01:01,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:01:01,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e7a7590270ad7b6da5d9 HTTP/1.1" 200 None 2026-03-28 08:01:01,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:01:01,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 08:01:01,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:01:01,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e7a7590270ad7b6da5d9 HTTP/1.1" 200 None 2026-03-28 08:01:01,475 - lava_callback - ERROR - Error processing callback for node 69c4e7a7590270ad7b6da5d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33802 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:01:06,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:01:06,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55608 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:55620 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:02:09,722 - lava_callback - INFO - Received callback for node 69c4e7f3590270ad7b6da5e0 INFO: 172.21.0.1:59924 - "POST /node/69c4e7f3590270ad7b6da5e0 HTTP/1.0" 202 Accepted 2026-03-28 08:02:09,728 - lava_callback - INFO - Starting async job submit for node 69c4e7f3590270ad7b6da5e0 2026-03-28 08:02:09,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:02:09,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e7f3590270ad7b6da5e0 HTTP/1.1" 200 None 2026-03-28 08:02:09,745 - lava_callback - DEBUG - Processing node 69c4e7f3590270ad7b6da5e0: baseline-arm64-baylibre 2026-03-28 08:02:09,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e7f3590270ad7b6da5e0 2026-03-28 08:02:09,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:02:09,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e7f3590270ad7b6da5e0 HTTP/1.1" 200 None 2026-03-28 08:02:09,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:02:09,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:02:09,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:02:09,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e7f3590270ad7b6da5e0 HTTP/1.1" 200 None 2026-03-28 08:02:09,804 - lava_callback - ERROR - Error processing callback for node 69c4e7f3590270ad7b6da5e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52398 - "GET /wiki HTTP/1.0" 404 Not Found INFO: 172.21.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54670 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:03:05,346 - lava_callback - INFO - Received callback for node 69c7854e29015d0f959934a9 INFO: 172.21.0.1:33768 - "POST /node/69c7854e29015d0f959934a9 HTTP/1.0" 202 Accepted 2026-03-28 08:03:05,352 - lava_callback - INFO - Starting async job submit for node 69c7854e29015d0f959934a9 2026-03-28 08:03:05,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:05,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7854e29015d0f959934a9 HTTP/1.1" 200 None 2026-03-28 08:03:05,390 - lava_callback - DEBUG - Processing node 69c7854e29015d0f959934a9: baseline-arm64-broonie 2026-03-28 08:03:05,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:03:05,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:03:05,925 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7854e29015d0f959934a9/log.txt.gz 2026-03-28 08:03:06,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:33772 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:03:10,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:03:10,857 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7854e29015d0f959934a9/lava-logs.yaml 2026-03-28 08:03:10,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:03:11,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:03:11,022 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7854e29015d0f959934a9/lava_callback.json.gz 2026-03-28 08:03:11,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:11,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7854e29015d0f959934a9 HTTP/1.1" 200 None 2026-03-28 08:03:11,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:11,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 08:03:11,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:11,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7854e29015d0f959934a9 HTTP/1.1" 200 None 2026-03-28 08:03:11,125 - lava_callback - INFO - Completed processing callback for node 69c7854e29015d0f959934a9 INFO: 172.21.0.1:43702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52240 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:03:29,830 - lava_callback - INFO - Received callback for node 69c4e840590270ad7b6da5eb INFO: 172.21.0.1:40932 - "POST /node/69c4e840590270ad7b6da5eb HTTP/1.0" 202 Accepted 2026-03-28 08:03:29,835 - lava_callback - INFO - Starting async job submit for node 69c4e840590270ad7b6da5eb 2026-03-28 08:03:29,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:29,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e840590270ad7b6da5eb HTTP/1.1" 200 None 2026-03-28 08:03:29,852 - lava_callback - DEBUG - Processing node 69c4e840590270ad7b6da5eb: baseline-arm64-baylibre 2026-03-28 08:03:29,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e840590270ad7b6da5eb 2026-03-28 08:03:29,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:29,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e840590270ad7b6da5eb HTTP/1.1" 200 None 2026-03-28 08:03:29,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:29,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:03:29,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:29,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e840590270ad7b6da5eb HTTP/1.1" 200 None 2026-03-28 08:03:29,907 - lava_callback - ERROR - Error processing callback for node 69c4e840590270ad7b6da5eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40938 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:03:36,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:36,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39796 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:04:49,803 - lava_callback - INFO - Received callback for node 69c4e88f590270ad7b6da5f1 INFO: 172.21.0.1:41882 - "POST /node/69c4e88f590270ad7b6da5f1 HTTP/1.0" 202 Accepted 2026-03-28 08:04:49,808 - lava_callback - INFO - Starting async job submit for node 69c4e88f590270ad7b6da5f1 2026-03-28 08:04:49,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:04:49,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e88f590270ad7b6da5f1 HTTP/1.1" 200 None 2026-03-28 08:04:49,824 - lava_callback - DEBUG - Processing node 69c4e88f590270ad7b6da5f1: baseline-arm64-baylibre 2026-03-28 08:04:49,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e88f590270ad7b6da5f1 2026-03-28 08:04:49,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:04:49,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e88f590270ad7b6da5f1 HTTP/1.1" 200 None 2026-03-28 08:04:49,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:04:49,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:04:49,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:04:49,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e88f590270ad7b6da5f1 HTTP/1.1" 200 None 2026-03-28 08:04:49,879 - lava_callback - ERROR - Error processing callback for node 69c4e88f590270ad7b6da5f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43450 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:06:09,883 - lava_callback - INFO - Received callback for node 69c4e8e2590270ad7b6da5fb INFO: 172.21.0.1:38884 - "POST /node/69c4e8e2590270ad7b6da5fb HTTP/1.0" 202 Accepted 2026-03-28 08:06:09,888 - lava_callback - INFO - Starting async job submit for node 69c4e8e2590270ad7b6da5fb 2026-03-28 08:06:09,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:06:09,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e8e2590270ad7b6da5fb HTTP/1.1" 200 None 2026-03-28 08:06:09,904 - lava_callback - DEBUG - Processing node 69c4e8e2590270ad7b6da5fb: baseline-arm64-baylibre 2026-03-28 08:06:09,905 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e8e2590270ad7b6da5fb 2026-03-28 08:06:09,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:06:09,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e8e2590270ad7b6da5fb HTTP/1.1" 200 None 2026-03-28 08:06:09,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:06:09,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:06:09,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:06:09,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e8e2590270ad7b6da5fb HTTP/1.1" 200 None 2026-03-28 08:06:09,965 - lava_callback - ERROR - Error processing callback for node 69c4e8e2590270ad7b6da5fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45036 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:07:29,983 - lava_callback - INFO - Received callback for node 69c4e932590270ad7b6da607 INFO: 172.21.0.1:53694 - "POST /node/69c4e932590270ad7b6da607 HTTP/1.0" 202 Accepted 2026-03-28 08:07:29,988 - lava_callback - INFO - Starting async job submit for node 69c4e932590270ad7b6da607 2026-03-28 08:07:29,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:07:30,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e932590270ad7b6da607 HTTP/1.1" 200 None 2026-03-28 08:07:30,004 - lava_callback - DEBUG - Processing node 69c4e932590270ad7b6da607: baseline-arm64-baylibre 2026-03-28 08:07:30,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e932590270ad7b6da607 2026-03-28 08:07:30,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:07:30,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e932590270ad7b6da607 HTTP/1.1" 200 None 2026-03-28 08:07:30,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:07:30,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:07:30,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:07:30,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e932590270ad7b6da607 HTTP/1.1" 200 None 2026-03-28 08:07:30,056 - lava_callback - ERROR - Error processing callback for node 69c4e932590270ad7b6da607: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39742 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:08:50,159 - lava_callback - INFO - Received callback for node 69c4e980590270ad7b6da60d INFO: 172.21.0.1:36710 - "POST /node/69c4e980590270ad7b6da60d HTTP/1.0" 202 Accepted 2026-03-28 08:08:50,165 - lava_callback - INFO - Starting async job submit for node 69c4e980590270ad7b6da60d 2026-03-28 08:08:50,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:08:50,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e980590270ad7b6da60d HTTP/1.1" 200 None 2026-03-28 08:08:50,181 - lava_callback - DEBUG - Processing node 69c4e980590270ad7b6da60d: baseline-arm64-baylibre 2026-03-28 08:08:50,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e980590270ad7b6da60d 2026-03-28 08:08:50,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:08:50,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e980590270ad7b6da60d HTTP/1.1" 200 None 2026-03-28 08:08:50,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:08:50,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:08:50,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:08:50,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e980590270ad7b6da60d HTTP/1.1" 200 None 2026-03-28 08:08:50,245 - lava_callback - ERROR - Error processing callback for node 69c4e980590270ad7b6da60d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44682 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:10:01,905 - lava_callback - INFO - Received callback for node 69c4e9d6590270ad7b6da617 INFO: 172.21.0.1:37390 - "POST /node/69c4e9d6590270ad7b6da617 HTTP/1.0" 202 Accepted 2026-03-28 08:10:01,911 - lava_callback - INFO - Starting async job submit for node 69c4e9d6590270ad7b6da617 2026-03-28 08:10:01,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:10:01,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e9d6590270ad7b6da617 HTTP/1.1" 200 None 2026-03-28 08:10:01,935 - lava_callback - DEBUG - Processing node 69c4e9d6590270ad7b6da617: baseline-arm64-clabbe 2026-03-28 08:10:01,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e9d6590270ad7b6da617 2026-03-28 08:10:01,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:10:01,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e9d6590270ad7b6da617 HTTP/1.1" 200 None 2026-03-28 08:10:01,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:10:01,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:10:01,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:10:02,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e9d6590270ad7b6da617 HTTP/1.1" 200 None 2026-03-28 08:10:02,015 - lava_callback - ERROR - Error processing callback for node 69c4e9d6590270ad7b6da617: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52352 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:11:30,239 - lava_callback - INFO - Received callback for node 69c4ea21590270ad7b6da622 INFO: 172.21.0.1:49122 - "POST /node/69c4ea21590270ad7b6da622 HTTP/1.0" 202 Accepted 2026-03-28 08:11:30,245 - lava_callback - INFO - Starting async job submit for node 69c4ea21590270ad7b6da622 2026-03-28 08:11:30,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:11:30,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ea21590270ad7b6da622 HTTP/1.1" 200 None 2026-03-28 08:11:30,261 - lava_callback - DEBUG - Processing node 69c4ea21590270ad7b6da622: baseline-arm64-baylibre 2026-03-28 08:11:30,261 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4ea21590270ad7b6da622 2026-03-28 08:11:30,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:11:30,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ea21590270ad7b6da622 HTTP/1.1" 200 None 2026-03-28 08:11:30,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:11:30,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:11:30,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:11:30,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ea21590270ad7b6da622 HTTP/1.1" 200 None 2026-03-28 08:11:30,330 - lava_callback - ERROR - Error processing callback for node 69c4ea21590270ad7b6da622: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42032 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:13:01,913 - lava_callback - INFO - Received callback for node 69c4ea77590270ad7b6da62b INFO: 172.21.0.1:50122 - "POST /node/69c4ea77590270ad7b6da62b HTTP/1.0" 202 Accepted 2026-03-28 08:13:01,922 - lava_callback - INFO - Starting async job submit for node 69c4ea77590270ad7b6da62b 2026-03-28 08:13:01,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:13:01,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ea77590270ad7b6da62b HTTP/1.1" 200 None 2026-03-28 08:13:01,944 - lava_callback - DEBUG - Processing node 69c4ea77590270ad7b6da62b: baseline-arm64-clabbe 2026-03-28 08:13:01,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4ea77590270ad7b6da62b 2026-03-28 08:13:01,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:13:01,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ea77590270ad7b6da62b HTTP/1.1" 200 None 2026-03-28 08:13:01,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:13:01,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:13:01,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:13:02,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ea77590270ad7b6da62b HTTP/1.1" 200 None 2026-03-28 08:13:02,026 - lava_callback - ERROR - Error processing callback for node 69c4ea77590270ad7b6da62b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1] INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.21.0.1:49050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39716 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:17:04,684 - lava_callback - INFO - Received callback for node 69c47aec590270ad7b6d6974 2026-03-28 00:17:04,694 - lava_callback - INFO - Starting async job submit for node 69c47aec590270ad7b6d6974 INFO: 172.21.0.1:54576 - "POST /node/69c47aec590270ad7b6d6974 HTTP/1.0" 202 Accepted 2026-03-28 00:17:04,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:17:04,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47aec590270ad7b6d6974 HTTP/1.1" 200 None 2026-03-28 00:17:04,737 - lava_callback - DEBUG - Processing node 69c47aec590270ad7b6d6974: baseline-arm64-baylibre 2026-03-28 00:17:04,743 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47aec590270ad7b6d6974 2026-03-28 00:17:04,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:17:04,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47aec590270ad7b6d6974 HTTP/1.1" 200 None 2026-03-28 00:17:04,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:17:04,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:17:04,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:17:04,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47aec590270ad7b6d6974 HTTP/1.1" 200 None 2026-03-28 00:17:04,871 - lava_callback - ERROR - Error processing callback for node 69c47aec590270ad7b6d6974: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55330 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:18:24,434 - lava_callback - INFO - Received callback for node 69c47b3e590270ad7b6d6990 INFO: 172.21.0.1:41098 - "POST /node/69c47b3e590270ad7b6d6990 HTTP/1.0" 202 Accepted 2026-03-28 00:18:24,440 - lava_callback - INFO - Starting async job submit for node 69c47b3e590270ad7b6d6990 2026-03-28 00:18:24,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:18:24,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3e590270ad7b6d6990 HTTP/1.1" 200 None 2026-03-28 00:18:24,457 - lava_callback - DEBUG - Processing node 69c47b3e590270ad7b6d6990: baseline-arm64-baylibre 2026-03-28 00:18:24,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47b3e590270ad7b6d6990 2026-03-28 00:18:24,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:18:24,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3e590270ad7b6d6990 HTTP/1.1" 200 None 2026-03-28 00:18:24,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:18:24,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:18:24,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:18:24,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47b3e590270ad7b6d6990 HTTP/1.1" 200 None 2026-03-28 00:18:24,509 - lava_callback - ERROR - Error processing callback for node 69c47b3e590270ad7b6d6990: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55790 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:20:04,539 - lava_callback - INFO - Received callback for node 69c47ba1590270ad7b6d69d4 INFO: 172.21.0.1:40662 - "POST /node/69c47ba1590270ad7b6d69d4 HTTP/1.0" 202 Accepted 2026-03-28 00:20:04,545 - lava_callback - INFO - Starting async job submit for node 69c47ba1590270ad7b6d69d4 2026-03-28 00:20:04,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:20:04,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47ba1590270ad7b6d69d4 HTTP/1.1" 200 None 2026-03-28 00:20:04,561 - lava_callback - DEBUG - Processing node 69c47ba1590270ad7b6d69d4: baseline-arm64-baylibre 2026-03-28 00:20:04,562 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47ba1590270ad7b6d69d4 2026-03-28 00:20:04,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:20:04,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47ba1590270ad7b6d69d4 HTTP/1.1" 200 None 2026-03-28 00:20:04,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:20:04,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:20:04,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:20:04,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47ba1590270ad7b6d69d4 HTTP/1.1" 200 None 2026-03-28 00:20:04,618 - lava_callback - ERROR - Error processing callback for node 69c47ba1590270ad7b6d69d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35934 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:21:24,603 - lava_callback - INFO - Received callback for node 69c47bf5590270ad7b6d69e3 INFO: 172.21.0.1:59576 - "POST /node/69c47bf5590270ad7b6d69e3 HTTP/1.0" 202 Accepted 2026-03-28 00:21:24,608 - lava_callback - INFO - Starting async job submit for node 69c47bf5590270ad7b6d69e3 2026-03-28 00:21:24,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:21:24,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47bf5590270ad7b6d69e3 HTTP/1.1" 200 None 2026-03-28 00:21:24,625 - lava_callback - DEBUG - Processing node 69c47bf5590270ad7b6d69e3: baseline-arm64-baylibre 2026-03-28 00:21:24,625 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47bf5590270ad7b6d69e3 2026-03-28 00:21:24,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:21:24,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47bf5590270ad7b6d69e3 HTTP/1.1" 200 None 2026-03-28 00:21:24,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:21:24,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:21:24,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:21:24,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47bf5590270ad7b6d69e3 HTTP/1.1" 200 None 2026-03-28 00:21:24,739 - lava_callback - ERROR - Error processing callback for node 69c47bf5590270ad7b6d69e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43086 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:22:44,669 - lava_callback - INFO - Received callback for node 69c47c46590270ad7b6d69f3 INFO: 172.21.0.1:42872 - "POST /node/69c47c46590270ad7b6d69f3 HTTP/1.0" 202 Accepted 2026-03-28 00:22:44,675 - lava_callback - INFO - Starting async job submit for node 69c47c46590270ad7b6d69f3 2026-03-28 00:22:44,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:22:44,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47c46590270ad7b6d69f3 HTTP/1.1" 200 None 2026-03-28 00:22:44,702 - lava_callback - DEBUG - Processing node 69c47c46590270ad7b6d69f3: baseline-arm64-baylibre 2026-03-28 00:22:44,702 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47c46590270ad7b6d69f3 2026-03-28 00:22:44,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:22:44,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47c46590270ad7b6d69f3 HTTP/1.1" 200 None 2026-03-28 00:22:44,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:22:44,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:22:44,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:22:44,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47c46590270ad7b6d69f3 HTTP/1.1" 200 None 2026-03-28 00:22:44,761 - lava_callback - ERROR - Error processing callback for node 69c47c46590270ad7b6d69f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52052 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:24:04,723 - lava_callback - INFO - Received callback for node 69c47c98590270ad7b6d69fe INFO: 172.21.0.1:57702 - "POST /node/69c47c98590270ad7b6d69fe HTTP/1.0" 202 Accepted 2026-03-28 00:24:04,729 - lava_callback - INFO - Starting async job submit for node 69c47c98590270ad7b6d69fe 2026-03-28 00:24:04,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:24:04,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47c98590270ad7b6d69fe HTTP/1.1" 200 None 2026-03-28 00:24:04,745 - lava_callback - DEBUG - Processing node 69c47c98590270ad7b6d69fe: baseline-arm64-baylibre 2026-03-28 00:24:04,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47c98590270ad7b6d69fe 2026-03-28 00:24:04,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:24:04,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47c98590270ad7b6d69fe HTTP/1.1" 200 None 2026-03-28 00:24:04,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:24:04,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:24:04,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:24:04,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47c98590270ad7b6d69fe HTTP/1.1" 200 None 2026-03-28 00:24:04,803 - lava_callback - ERROR - Error processing callback for node 69c47c98590270ad7b6d69fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54124 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50276 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:25:41,806 - lava_callback - INFO - Received callback for node 69c47cf0590270ad7b6d6a14 INFO: 172.21.0.1:39308 - "POST /node/69c47cf0590270ad7b6d6a14 HTTP/1.0" 202 Accepted 2026-03-28 00:25:41,811 - lava_callback - INFO - Starting async job submit for node 69c47cf0590270ad7b6d6a14 2026-03-28 00:25:41,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:25:41,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47cf0590270ad7b6d6a14 HTTP/1.1" 200 None 2026-03-28 00:25:41,842 - lava_callback - DEBUG - Processing node 69c47cf0590270ad7b6d6a14: baseline-arm64-clabbe 2026-03-28 00:25:41,842 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47cf0590270ad7b6d6a14 2026-03-28 00:25:41,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:25:41,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47cf0590270ad7b6d6a14 HTTP/1.1" 200 None 2026-03-28 00:25:41,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:25:41,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:25:41,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:25:41,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47cf0590270ad7b6d6a14 HTTP/1.1" 200 None 2026-03-28 00:25:41,902 - lava_callback - ERROR - Error processing callback for node 69c47cf0590270ad7b6d6a14: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47436 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:27:01,780 - lava_callback - INFO - Received callback for node 69c47d4f590270ad7b6d6a22 INFO: 172.21.0.1:46310 - "POST /node/69c47d4f590270ad7b6d6a22 HTTP/1.0" 202 Accepted 2026-03-28 00:27:01,786 - lava_callback - INFO - Starting async job submit for node 69c47d4f590270ad7b6d6a22 2026-03-28 00:27:01,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:27:01,801 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47d4f590270ad7b6d6a22 HTTP/1.1" 200 None 2026-03-28 00:27:01,802 - lava_callback - DEBUG - Processing node 69c47d4f590270ad7b6d6a22: baseline-arm64-clabbe 2026-03-28 00:27:01,802 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47d4f590270ad7b6d6a22 2026-03-28 00:27:01,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:27:01,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47d4f590270ad7b6d6a22 HTTP/1.1" 200 None 2026-03-28 00:27:01,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:27:01,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:27:01,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:27:01,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47d4f590270ad7b6d6a22 HTTP/1.1" 200 None 2026-03-28 00:27:01,868 - lava_callback - ERROR - Error processing callback for node 69c47d4f590270ad7b6d6a22: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33202 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:28:21,871 - lava_callback - INFO - Received callback for node 69c47da1590270ad7b6d6a34 INFO: 172.21.0.1:39882 - "POST /node/69c47da1590270ad7b6d6a34 HTTP/1.0" 202 Accepted 2026-03-28 00:28:21,877 - lava_callback - INFO - Starting async job submit for node 69c47da1590270ad7b6d6a34 2026-03-28 00:28:21,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:28:21,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47da1590270ad7b6d6a34 HTTP/1.1" 200 None 2026-03-28 00:28:21,918 - lava_callback - DEBUG - Processing node 69c47da1590270ad7b6d6a34: baseline-arm64-clabbe 2026-03-28 00:28:21,918 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47da1590270ad7b6d6a34 2026-03-28 00:28:21,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:28:21,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47da1590270ad7b6d6a34 HTTP/1.1" 200 None 2026-03-28 00:28:21,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:28:21,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:28:21,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:28:21,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47da1590270ad7b6d6a34 HTTP/1.1" 200 None 2026-03-28 00:28:21,982 - lava_callback - ERROR - Error processing callback for node 69c47da1590270ad7b6d6a34: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41778 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:30:01,890 - lava_callback - INFO - Received callback for node 69c47df5590270ad7b6d8fdc INFO: 172.21.0.1:43460 - "POST /node/69c47df5590270ad7b6d8fdc HTTP/1.0" 202 Accepted 2026-03-28 00:30:01,895 - lava_callback - INFO - Starting async job submit for node 69c47df5590270ad7b6d8fdc 2026-03-28 00:30:01,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:30:01,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47df5590270ad7b6d8fdc HTTP/1.1" 200 None 2026-03-28 00:30:01,914 - lava_callback - DEBUG - Processing node 69c47df5590270ad7b6d8fdc: baseline-arm64-clabbe 2026-03-28 00:30:01,914 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47df5590270ad7b6d8fdc 2026-03-28 00:30:01,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:30:01,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47df5590270ad7b6d8fdc HTTP/1.1" 200 None 2026-03-28 00:30:01,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:30:01,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:30:01,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:30:02,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47df5590270ad7b6d8fdc HTTP/1.1" 200 None 2026-03-28 00:30:02,146 - lava_callback - ERROR - Error processing callback for node 69c47df5590270ad7b6d8fdc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57928 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:31:21,961 - lava_callback - INFO - Received callback for node 69c47e4a590270ad7b6d8ff5 INFO: 172.21.0.1:41034 - "POST /node/69c47e4a590270ad7b6d8ff5 HTTP/1.0" 202 Accepted 2026-03-28 00:31:21,967 - lava_callback - INFO - Starting async job submit for node 69c47e4a590270ad7b6d8ff5 2026-03-28 00:31:21,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:31:21,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e4a590270ad7b6d8ff5 HTTP/1.1" 200 None 2026-03-28 00:31:21,983 - lava_callback - DEBUG - Processing node 69c47e4a590270ad7b6d8ff5: baseline-arm64-clabbe 2026-03-28 00:31:21,983 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47e4a590270ad7b6d8ff5 2026-03-28 00:31:21,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:31:21,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e4a590270ad7b6d8ff5 HTTP/1.1" 200 None 2026-03-28 00:31:21,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:31:22,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:31:22,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:31:22,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47e4a590270ad7b6d8ff5 HTTP/1.1" 200 None 2026-03-28 00:31:22,048 - lava_callback - ERROR - Error processing callback for node 69c47e4a590270ad7b6d8ff5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52410 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50750 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:32:41,983 - lava_callback - INFO - Received callback for node 69c47e9d590270ad7b6d900d INFO: 172.21.0.1:37132 - "POST /node/69c47e9d590270ad7b6d900d HTTP/1.0" 202 Accepted 2026-03-28 00:32:41,989 - lava_callback - INFO - Starting async job submit for node 69c47e9d590270ad7b6d900d 2026-03-28 00:32:41,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:32:42,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e9d590270ad7b6d900d HTTP/1.1" 200 None 2026-03-28 00:32:42,006 - lava_callback - DEBUG - Processing node 69c47e9d590270ad7b6d900d: baseline-arm64-clabbe 2026-03-28 00:32:42,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47e9d590270ad7b6d900d 2026-03-28 00:32:42,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:32:42,020 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47e9d590270ad7b6d900d HTTP/1.1" 200 None 2026-03-28 00:32:42,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:32:42,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 00:32:42,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:32:42,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47e9d590270ad7b6d900d HTTP/1.1" 200 None 2026-03-28 00:32:42,062 - lava_callback - ERROR - Error processing callback for node 69c47e9d590270ad7b6d900d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37654 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:37:05,561 - lava_callback - INFO - Received callback for node 69c47fa4590270ad7b6d902b INFO: 172.21.0.1:45762 - "POST /node/69c47fa4590270ad7b6d902b HTTP/1.0" 202 Accepted 2026-03-28 00:37:05,570 - lava_callback - INFO - Starting async job submit for node 69c47fa4590270ad7b6d902b 2026-03-28 00:37:05,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:05,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47fa4590270ad7b6d902b HTTP/1.1" 200 None 2026-03-28 00:37:05,600 - lava_callback - DEBUG - Processing node 69c47fa4590270ad7b6d902b: baseline-arm-baylibre 2026-03-28 00:37:05,600 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47fa4590270ad7b6d902b 2026-03-28 00:37:05,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:05,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47fa4590270ad7b6d902b HTTP/1.1" 200 None 2026-03-28 00:37:05,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:05,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:37:05,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:05,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47fa4590270ad7b6d902b HTTP/1.1" 200 None 2026-03-28 00:37:05,692 - lava_callback - ERROR - Error processing callback for node 69c47fa4590270ad7b6d902b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39708 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:37:28,979 - lava_callback - INFO - Received callback for node 69c722a829015d0f9598f82e INFO: 172.21.0.1:39716 - "POST /node/69c722a829015d0f9598f82e HTTP/1.0" 202 Accepted 2026-03-28 00:37:28,988 - lava_callback - INFO - Starting async job submit for node 69c722a829015d0f9598f82e 2026-03-28 00:37:29,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:29,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722a829015d0f9598f82e HTTP/1.1" 200 None 2026-03-28 00:37:29,052 - lava_callback - DEBUG - Processing node 69c722a829015d0f9598f82e: baseline-x86 2026-03-28 00:37:29,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:29,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:29,781 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a829015d0f9598f82e/log.txt.gz 2026-03-28 00:37:29,959 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:30,664 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:30,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a829015d0f9598f82e/lava-logs.yaml 2026-03-28 00:37:30,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:31,129 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:31,131 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a829015d0f9598f82e/lava_callback.json.gz 2026-03-28 00:37:31,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:31,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722a829015d0f9598f82e HTTP/1.1" 200 None 2026-03-28 00:37:31,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:31,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 00:37:31,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:31,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c722a829015d0f9598f82e HTTP/1.1" 200 None 2026-03-28 00:37:31,266 - lava_callback - INFO - Completed processing callback for node 69c722a829015d0f9598f82e 2026-03-28 00:37:34,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:34,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43366 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:37:39,640 - lava_callback - INFO - Received callback for node 69c722ab29015d0f9598f837 INFO: 172.21.0.1:43380 - "POST /node/69c722ab29015d0f9598f837 HTTP/1.0" 202 Accepted 2026-03-28 00:37:39,647 - lava_callback - INFO - Starting async job submit for node 69c722ab29015d0f9598f837 2026-03-28 00:37:39,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:39,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722ab29015d0f9598f837 HTTP/1.1" 200 None 2026-03-28 00:37:39,681 - lava_callback - DEBUG - Processing node 69c722ab29015d0f9598f837: baseline-x86 2026-03-28 00:37:39,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:39,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:39,965 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c722ab29015d0f9598f837/log.txt.gz 2026-03-28 00:37:40,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:40,236 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:40,238 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c722ab29015d0f9598f837/lava-logs.yaml 2026-03-28 00:37:40,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:37:40,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:37:40,373 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c722ab29015d0f9598f837/lava_callback.json.gz 2026-03-28 00:37:40,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:40,405 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722ab29015d0f9598f837 HTTP/1.1" 200 None 2026-03-28 00:37:40,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:40,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 00:37:40,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:40,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c722ab29015d0f9598f837 HTTP/1.1" 200 None 2026-03-28 00:37:40,452 - lava_callback - INFO - Completed processing callback for node 69c722ab29015d0f9598f837 2026-03-28 00:37:45,475 - lava_callback - INFO - Received callback for node 69c47fcb590270ad7b6d903e INFO: 172.21.0.1:35054 - "POST /node/69c47fcb590270ad7b6d903e HTTP/1.0" 202 Accepted 2026-03-28 00:37:45,481 - lava_callback - INFO - Starting async job submit for node 69c47fcb590270ad7b6d903e 2026-03-28 00:37:45,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:45,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47fcb590270ad7b6d903e HTTP/1.1" 200 None 2026-03-28 00:37:45,497 - lava_callback - DEBUG - Processing node 69c47fcb590270ad7b6d903e: baseline-arm-baylibre 2026-03-28 00:37:45,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47fcb590270ad7b6d903e 2026-03-28 00:37:45,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:45,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47fcb590270ad7b6d903e HTTP/1.1" 200 None 2026-03-28 00:37:45,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:45,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:37:45,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:37:45,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47fcb590270ad7b6d903e HTTP/1.1" 200 None 2026-03-28 00:37:45,550 - lava_callback - ERROR - Error processing callback for node 69c47fcb590270ad7b6d903e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59470 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:38:04,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:04,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47962 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:38:25,559 - lava_callback - INFO - Received callback for node 69c47ff1590270ad7b6d9049 INFO: 172.21.0.1:54416 - "POST /node/69c47ff1590270ad7b6d9049 HTTP/1.0" 202 Accepted 2026-03-28 00:38:25,566 - lava_callback - INFO - Starting async job submit for node 69c47ff1590270ad7b6d9049 2026-03-28 00:38:25,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:25,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47ff1590270ad7b6d9049 HTTP/1.1" 200 None 2026-03-28 00:38:25,609 - lava_callback - DEBUG - Processing node 69c47ff1590270ad7b6d9049: baseline-arm-baylibre 2026-03-28 00:38:25,610 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c47ff1590270ad7b6d9049 2026-03-28 00:38:25,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:25,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47ff1590270ad7b6d9049 HTTP/1.1" 200 None 2026-03-28 00:38:25,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:25,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:38:25,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:25,694 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c47ff1590270ad7b6d9049 HTTP/1.1" 200 None 2026-03-28 00:38:25,696 - lava_callback - ERROR - Error processing callback for node 69c47ff1590270ad7b6d9049: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43564 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:38:36,786 - lava_callback - INFO - Received callback for node 69c722a929015d0f9598f831 INFO: 172.21.0.1:43578 - "POST /node/69c722a929015d0f9598f831 HTTP/1.0" 202 Accepted 2026-03-28 00:38:36,795 - lava_callback - INFO - Starting async job submit for node 69c722a929015d0f9598f831 2026-03-28 00:38:36,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:36,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722a929015d0f9598f831 HTTP/1.1" 200 None 2026-03-28 00:38:36,862 - lava_callback - DEBUG - Processing node 69c722a929015d0f9598f831: baseline-x86 2026-03-28 00:38:37,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:38:37,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:38:37,669 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a929015d0f9598f831/log.txt.gz 2026-03-28 00:38:37,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:38:38,373 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:38:38,375 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a929015d0f9598f831/lava-logs.yaml 2026-03-28 00:38:38,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:38:38,567 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:38:38,569 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c722a929015d0f9598f831/lava_callback.json.gz 2026-03-28 00:38:38,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:38,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722a929015d0f9598f831 HTTP/1.1" 200 None 2026-03-28 00:38:38,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:38,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 00:38:38,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:38:38,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c722a929015d0f9598f831 HTTP/1.1" 200 None 2026-03-28 00:38:38,661 - lava_callback - INFO - Completed processing callback for node 69c722a929015d0f9598f831 INFO: 172.21.0.1:41242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34862 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:39:04,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:04,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 00:39:05,527 - lava_callback - INFO - Received callback for node 69c48017590270ad7b6d9055 INFO: 172.21.0.1:55452 - "POST /node/69c48017590270ad7b6d9055 HTTP/1.0" 202 Accepted 2026-03-28 00:39:05,533 - lava_callback - INFO - Starting async job submit for node 69c48017590270ad7b6d9055 2026-03-28 00:39:05,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:05,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48017590270ad7b6d9055 HTTP/1.1" 200 None 2026-03-28 00:39:05,549 - lava_callback - DEBUG - Processing node 69c48017590270ad7b6d9055: baseline-arm-baylibre 2026-03-28 00:39:05,549 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48017590270ad7b6d9055 2026-03-28 00:39:05,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:05,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48017590270ad7b6d9055 HTTP/1.1" 200 None 2026-03-28 00:39:05,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:05,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:39:05,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:05,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48017590270ad7b6d9055 HTTP/1.1" 200 None 2026-03-28 00:39:05,606 - lava_callback - ERROR - Error processing callback for node 69c48017590270ad7b6d9055: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43830 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:39:45,930 - lava_callback - INFO - Received callback for node 69c4803d590270ad7b6d9064 INFO: 172.21.0.1:56868 - "POST /node/69c4803d590270ad7b6d9064 HTTP/1.0" 202 Accepted 2026-03-28 00:39:45,936 - lava_callback - INFO - Starting async job submit for node 69c4803d590270ad7b6d9064 2026-03-28 00:39:45,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:45,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4803d590270ad7b6d9064 HTTP/1.1" 200 None 2026-03-28 00:39:45,956 - lava_callback - DEBUG - Processing node 69c4803d590270ad7b6d9064: baseline-arm-baylibre 2026-03-28 00:39:45,956 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4803d590270ad7b6d9064 2026-03-28 00:39:45,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:45,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4803d590270ad7b6d9064 HTTP/1.1" 200 None 2026-03-28 00:39:45,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:45,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:39:45,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:39:46,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4803d590270ad7b6d9064 HTTP/1.1" 200 None 2026-03-28 00:39:46,008 - lava_callback - ERROR - Error processing callback for node 69c4803d590270ad7b6d9064: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60396 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:40:05,612 - lava_callback - INFO - Received callback for node 69c48062590270ad7b6d9071 INFO: 172.21.0.1:42430 - "POST /node/69c48062590270ad7b6d9071 HTTP/1.0" 202 Accepted 2026-03-28 00:40:05,618 - lava_callback - INFO - Starting async job submit for node 69c48062590270ad7b6d9071 2026-03-28 00:40:05,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:05,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48062590270ad7b6d9071 HTTP/1.1" 200 None 2026-03-28 00:40:05,634 - lava_callback - DEBUG - Processing node 69c48062590270ad7b6d9071: baseline-arm-baylibre 2026-03-28 00:40:05,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48062590270ad7b6d9071 2026-03-28 00:40:05,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:05,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48062590270ad7b6d9071 HTTP/1.1" 200 None 2026-03-28 00:40:05,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:05,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:40:05,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:05,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48062590270ad7b6d9071 HTTP/1.1" 200 None 2026-03-28 00:40:05,690 - lava_callback - ERROR - Error processing callback for node 69c48062590270ad7b6d9071: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43632 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:40:25,591 - lava_callback - INFO - Received callback for node 69c48075590270ad7b6d909d INFO: 172.21.0.1:59354 - "POST /node/69c48075590270ad7b6d909d HTTP/1.0" 202 Accepted 2026-03-28 00:40:25,597 - lava_callback - INFO - Starting async job submit for node 69c48075590270ad7b6d909d 2026-03-28 00:40:25,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:25,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48075590270ad7b6d909d HTTP/1.1" 200 None 2026-03-28 00:40:25,619 - lava_callback - DEBUG - Processing node 69c48075590270ad7b6d909d: baseline-arm-baylibre 2026-03-28 00:40:25,619 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48075590270ad7b6d909d 2026-03-28 00:40:25,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:25,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48075590270ad7b6d909d HTTP/1.1" 200 None 2026-03-28 00:40:25,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:25,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:25,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:25,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48075590270ad7b6d909d HTTP/1.1" 200 None 2026-03-28 00:40:25,683 - lava_callback - ERROR - Error processing callback for node 69c48075590270ad7b6d909d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:40:26,275 - lava_callback - INFO - Received callback for node 69c48077590270ad7b6d90a0 INFO: 172.21.0.1:59370 - "POST /node/69c48077590270ad7b6d90a0 HTTP/1.0" 202 Accepted 2026-03-28 00:40:26,281 - lava_callback - INFO - Starting async job submit for node 69c48077590270ad7b6d90a0 2026-03-28 00:40:26,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:26,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48077590270ad7b6d90a0 HTTP/1.1" 200 None 2026-03-28 00:40:26,297 - lava_callback - DEBUG - Processing node 69c48077590270ad7b6d90a0: baseline-arm-baylibre 2026-03-28 00:40:26,297 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48077590270ad7b6d90a0 2026-03-28 00:40:26,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:26,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48077590270ad7b6d90a0 HTTP/1.1" 200 None 2026-03-28 00:40:26,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:26,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:26,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:26,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48077590270ad7b6d90a0 HTTP/1.1" 200 None 2026-03-28 00:40:26,350 - lava_callback - ERROR - Error processing callback for node 69c48077590270ad7b6d90a0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56542 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:40:42,476 - lava_callback - INFO - Received callback for node 69c48085590270ad7b6d90e6 INFO: 172.21.0.1:42796 - "POST /node/69c48085590270ad7b6d90e6 HTTP/1.0" 202 Accepted 2026-03-28 00:40:42,482 - lava_callback - INFO - Starting async job submit for node 69c48085590270ad7b6d90e6 2026-03-28 00:40:42,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:42,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48085590270ad7b6d90e6 HTTP/1.1" 200 None 2026-03-28 00:40:42,500 - lava_callback - DEBUG - Processing node 69c48085590270ad7b6d90e6: baseline-arm-clabbe 2026-03-28 00:40:42,500 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48085590270ad7b6d90e6 2026-03-28 00:40:42,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:42,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48085590270ad7b6d90e6 HTTP/1.1" 200 None 2026-03-28 00:40:42,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:42,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:42,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:42,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48085590270ad7b6d90e6 HTTP/1.1" 200 None 2026-03-28 00:40:42,558 - lava_callback - ERROR - Error processing callback for node 69c48085590270ad7b6d90e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:40:45,597 - lava_callback - INFO - Received callback for node 69c48078590270ad7b6d90a3 INFO: 172.21.0.1:42800 - "POST /node/69c48078590270ad7b6d90a3 HTTP/1.0" 202 Accepted 2026-03-28 00:40:45,602 - lava_callback - INFO - Starting async job submit for node 69c48078590270ad7b6d90a3 2026-03-28 00:40:45,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:45,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48078590270ad7b6d90a3 HTTP/1.1" 200 None 2026-03-28 00:40:45,633 - lava_callback - DEBUG - Processing node 69c48078590270ad7b6d90a3: baseline-arm-baylibre 2026-03-28 00:40:45,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48078590270ad7b6d90a3 2026-03-28 00:40:45,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:45,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48078590270ad7b6d90a3 HTTP/1.1" 200 None 2026-03-28 00:40:45,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:45,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:45,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:45,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48078590270ad7b6d90a3 HTTP/1.1" 200 None 2026-03-28 00:40:45,690 - lava_callback - ERROR - Error processing callback for node 69c48078590270ad7b6d90a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:40:46,263 - lava_callback - INFO - Received callback for node 69c48079590270ad7b6d90a6 INFO: 172.21.0.1:42802 - "POST /node/69c48079590270ad7b6d90a6 HTTP/1.0" 202 Accepted 2026-03-28 00:40:46,268 - lava_callback - INFO - Starting async job submit for node 69c48079590270ad7b6d90a6 2026-03-28 00:40:46,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:46,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48079590270ad7b6d90a6 HTTP/1.1" 200 None 2026-03-28 00:40:46,286 - lava_callback - DEBUG - Processing node 69c48079590270ad7b6d90a6: baseline-arm-baylibre 2026-03-28 00:40:46,286 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48079590270ad7b6d90a6 2026-03-28 00:40:46,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:46,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48079590270ad7b6d90a6 HTTP/1.1" 200 None 2026-03-28 00:40:46,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:46,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:46,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:46,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48079590270ad7b6d90a6 HTTP/1.1" 200 None 2026-03-28 00:40:46,375 - lava_callback - ERROR - Error processing callback for node 69c48079590270ad7b6d90a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42810 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:40:47,006 - lava_callback - INFO - Received callback for node 69c4807c590270ad7b6d90cb INFO: 172.21.0.1:42822 - "POST /node/69c4807c590270ad7b6d90cb HTTP/1.0" 202 Accepted 2026-03-28 00:40:47,011 - lava_callback - INFO - Starting async job submit for node 69c4807c590270ad7b6d90cb 2026-03-28 00:40:47,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4807c590270ad7b6d90cb HTTP/1.1" 200 None 2026-03-28 00:40:47,027 - lava_callback - DEBUG - Processing node 69c4807c590270ad7b6d90cb: baseline-arm-baylibre 2026-03-28 00:40:47,027 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4807c590270ad7b6d90cb 2026-03-28 00:40:47,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4807c590270ad7b6d90cb HTTP/1.1" 200 None 2026-03-28 00:40:47,043 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:47,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4807c590270ad7b6d90cb HTTP/1.1" 200 None 2026-03-28 00:40:47,080 - lava_callback - ERROR - Error processing callback for node 69c4807c590270ad7b6d90cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:40:47,656 - lava_callback - INFO - Received callback for node 69c4807e590270ad7b6d90ce INFO: 172.21.0.1:42830 - "POST /node/69c4807e590270ad7b6d90ce HTTP/1.0" 202 Accepted 2026-03-28 00:40:47,662 - lava_callback - INFO - Starting async job submit for node 69c4807e590270ad7b6d90ce 2026-03-28 00:40:47,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4807e590270ad7b6d90ce HTTP/1.1" 200 None 2026-03-28 00:40:47,681 - lava_callback - DEBUG - Processing node 69c4807e590270ad7b6d90ce: baseline-arm-baylibre 2026-03-28 00:40:47,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4807e590270ad7b6d90ce 2026-03-28 00:40:47,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4807e590270ad7b6d90ce HTTP/1.1" 200 None 2026-03-28 00:40:47,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:40:47,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:47,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4807e590270ad7b6d90ce HTTP/1.1" 200 None 2026-03-28 00:40:47,736 - lava_callback - ERROR - Error processing callback for node 69c4807e590270ad7b6d90ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:40:48,755 - lava_callback - INFO - Received callback for node 69c7231029015d0f9598f86f INFO: 172.21.0.1:42838 - "POST /node/69c7231029015d0f9598f86f HTTP/1.0" 202 Accepted 2026-03-28 00:40:48,761 - lava_callback - INFO - Starting async job submit for node 69c7231029015d0f9598f86f 2026-03-28 00:40:48,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:48,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7231029015d0f9598f86f HTTP/1.1" 200 None 2026-03-28 00:40:48,804 - lava_callback - DEBUG - Processing node 69c7231029015d0f9598f86f: baseline-arm 2026-03-28 00:40:48,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:40:49,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:40:49,174 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7231029015d0f9598f86f/log.txt.gz 2026-03-28 00:40:49,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:40:49,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:40:49,510 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7231029015d0f9598f86f/lava-logs.yaml 2026-03-28 00:40:49,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:40:49,679 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:40:49,681 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7231029015d0f9598f86f/lava_callback.json.gz 2026-03-28 00:40:49,705 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:49,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7231029015d0f9598f86f HTTP/1.1" 200 None 2026-03-28 00:40:49,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:49,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4229015d0f9598f742 HTTP/1.1" 200 None 2026-03-28 00:40:49,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:40:49,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7231029015d0f9598f86f HTTP/1.1" 200 None 2026-03-28 00:40:49,766 - lava_callback - INFO - Completed processing callback for node 69c7231029015d0f9598f86f INFO: 172.21.0.1:40152 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:41:02,364 - lava_callback - INFO - Received callback for node 69c48088590270ad7b6d90f3 INFO: 172.21.0.1:53828 - "POST /node/69c48088590270ad7b6d90f3 HTTP/1.0" 202 Accepted 2026-03-28 00:41:02,369 - lava_callback - INFO - Starting async job submit for node 69c48088590270ad7b6d90f3 2026-03-28 00:41:02,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:02,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48088590270ad7b6d90f3 HTTP/1.1" 200 None 2026-03-28 00:41:02,392 - lava_callback - DEBUG - Processing node 69c48088590270ad7b6d90f3: baseline-arm-clabbe 2026-03-28 00:41:02,393 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48088590270ad7b6d90f3 2026-03-28 00:41:02,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:02,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48088590270ad7b6d90f3 HTTP/1.1" 200 None 2026-03-28 00:41:02,408 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:02,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:41:02,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:02,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48088590270ad7b6d90f3 HTTP/1.1" 200 None 2026-03-28 00:41:02,446 - lava_callback - ERROR - Error processing callback for node 69c48088590270ad7b6d90f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:41:03,046 - lava_callback - INFO - Received callback for node 69c4808a590270ad7b6d90fc INFO: 172.21.0.1:53834 - "POST /node/69c4808a590270ad7b6d90fc HTTP/1.0" 202 Accepted 2026-03-28 00:41:03,052 - lava_callback - INFO - Starting async job submit for node 69c4808a590270ad7b6d90fc 2026-03-28 00:41:03,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808a590270ad7b6d90fc HTTP/1.1" 200 None 2026-03-28 00:41:03,068 - lava_callback - DEBUG - Processing node 69c4808a590270ad7b6d90fc: baseline-arm-clabbe 2026-03-28 00:41:03,068 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4808a590270ad7b6d90fc 2026-03-28 00:41:03,069 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808a590270ad7b6d90fc HTTP/1.1" 200 None 2026-03-28 00:41:03,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:41:03,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4808a590270ad7b6d90fc HTTP/1.1" 200 None 2026-03-28 00:41:03,120 - lava_callback - ERROR - Error processing callback for node 69c4808a590270ad7b6d90fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:41:03,716 - lava_callback - INFO - Received callback for node 69c4808b590270ad7b6d90ff INFO: 172.21.0.1:53850 - "POST /node/69c4808b590270ad7b6d90ff HTTP/1.0" 202 Accepted 2026-03-28 00:41:03,722 - lava_callback - INFO - Starting async job submit for node 69c4808b590270ad7b6d90ff 2026-03-28 00:41:03,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808b590270ad7b6d90ff HTTP/1.1" 200 None 2026-03-28 00:41:03,740 - lava_callback - DEBUG - Processing node 69c4808b590270ad7b6d90ff: baseline-arm-clabbe 2026-03-28 00:41:03,740 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4808b590270ad7b6d90ff 2026-03-28 00:41:03,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808b590270ad7b6d90ff HTTP/1.1" 200 None 2026-03-28 00:41:03,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,768 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:41:03,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:03,790 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4808b590270ad7b6d90ff HTTP/1.1" 200 None 2026-03-28 00:41:03,796 - lava_callback - ERROR - Error processing callback for node 69c4808b590270ad7b6d90ff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:41:04,395 - lava_callback - INFO - Received callback for node 69c4808c590270ad7b6d9102 INFO: 172.21.0.1:53854 - "POST /node/69c4808c590270ad7b6d9102 HTTP/1.0" 202 Accepted 2026-03-28 00:41:04,401 - lava_callback - INFO - Starting async job submit for node 69c4808c590270ad7b6d9102 2026-03-28 00:41:04,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:04,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808c590270ad7b6d9102 HTTP/1.1" 200 None 2026-03-28 00:41:04,417 - lava_callback - DEBUG - Processing node 69c4808c590270ad7b6d9102: baseline-arm-clabbe 2026-03-28 00:41:04,417 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4808c590270ad7b6d9102 2026-03-28 00:41:04,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:04,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808c590270ad7b6d9102 HTTP/1.1" 200 None 2026-03-28 00:41:04,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:04,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:41:04,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:04,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4808c590270ad7b6d9102 HTTP/1.1" 200 None 2026-03-28 00:41:04,476 - lava_callback - ERROR - Error processing callback for node 69c4808c590270ad7b6d9102: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:41:04,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:04,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 00:41:05,078 - lava_callback - INFO - Received callback for node 69c4808f590270ad7b6d910b INFO: 172.21.0.1:53860 - "POST /node/69c4808f590270ad7b6d910b HTTP/1.0" 202 Accepted 2026-03-28 00:41:05,084 - lava_callback - INFO - Starting async job submit for node 69c4808f590270ad7b6d910b 2026-03-28 00:41:05,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808f590270ad7b6d910b HTTP/1.1" 200 None 2026-03-28 00:41:05,099 - lava_callback - DEBUG - Processing node 69c4808f590270ad7b6d910b: baseline-arm-clabbe 2026-03-28 00:41:05,099 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4808f590270ad7b6d910b 2026-03-28 00:41:05,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4808f590270ad7b6d910b HTTP/1.1" 200 None 2026-03-28 00:41:05,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:41:05,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4808f590270ad7b6d910b HTTP/1.1" 200 None 2026-03-28 00:41:05,170 - lava_callback - ERROR - Error processing callback for node 69c4808f590270ad7b6d910b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 00:41:05,779 - lava_callback - INFO - Received callback for node 69c48090590270ad7b6d910e INFO: 172.21.0.1:53866 - "POST /node/69c48090590270ad7b6d910e HTTP/1.0" 202 Accepted 2026-03-28 00:41:05,784 - lava_callback - INFO - Starting async job submit for node 69c48090590270ad7b6d910e 2026-03-28 00:41:05,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48090590270ad7b6d910e HTTP/1.1" 200 None 2026-03-28 00:41:05,823 - lava_callback - DEBUG - Processing node 69c48090590270ad7b6d910e: baseline-arm-kontron 2026-03-28 00:41:05,823 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48090590270ad7b6d910e 2026-03-28 00:41:05,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48090590270ad7b6d910e HTTP/1.1" 200 None 2026-03-28 00:41:05,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 00:41:05,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:05,874 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48090590270ad7b6d910e HTTP/1.1" 200 None 2026-03-28 00:41:05,879 - lava_callback - ERROR - Error processing callback for node 69c48090590270ad7b6d910e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46388 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:41:22,354 - lava_callback - INFO - Received callback for node 69c480ad590270ad7b6d916d INFO: 172.21.0.1:56302 - "POST /node/69c480ad590270ad7b6d916d HTTP/1.0" 202 Accepted 2026-03-28 00:41:22,360 - lava_callback - INFO - Starting async job submit for node 69c480ad590270ad7b6d916d 2026-03-28 00:41:22,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:22,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480ad590270ad7b6d916d HTTP/1.1" 200 None 2026-03-28 00:41:22,376 - lava_callback - DEBUG - Processing node 69c480ad590270ad7b6d916d: baseline-arm-clabbe 2026-03-28 00:41:22,376 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c480ad590270ad7b6d916d 2026-03-28 00:41:22,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:22,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480ad590270ad7b6d916d HTTP/1.1" 200 None 2026-03-28 00:41:22,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:22,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:41:22,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:41:22,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480ad590270ad7b6d916d HTTP/1.1" 200 None 2026-03-28 00:41:22,431 - lava_callback - ERROR - Error processing callback for node 69c480ad590270ad7b6d916d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43646 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:42:02,392 - lava_callback - INFO - Received callback for node 69c480d4590270ad7b6d91bd INFO: 172.21.0.1:37764 - "POST /node/69c480d4590270ad7b6d91bd HTTP/1.0" 202 Accepted 2026-03-28 00:42:02,398 - lava_callback - INFO - Starting async job submit for node 69c480d4590270ad7b6d91bd 2026-03-28 00:42:02,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:02,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480d4590270ad7b6d91bd HTTP/1.1" 200 None 2026-03-28 00:42:02,414 - lava_callback - DEBUG - Processing node 69c480d4590270ad7b6d91bd: baseline-arm-clabbe 2026-03-28 00:42:02,414 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c480d4590270ad7b6d91bd 2026-03-28 00:42:02,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:02,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480d4590270ad7b6d91bd HTTP/1.1" 200 None 2026-03-28 00:42:02,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:02,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:42:02,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:02,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480d4590270ad7b6d91bd HTTP/1.1" 200 None 2026-03-28 00:42:02,477 - lava_callback - ERROR - Error processing callback for node 69c480d4590270ad7b6d91bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47744 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:42:42,412 - lava_callback - INFO - Received callback for node 69c480fb590270ad7b6d91f8 INFO: 172.21.0.1:45518 - "POST /node/69c480fb590270ad7b6d91f8 HTTP/1.0" 202 Accepted 2026-03-28 00:42:42,418 - lava_callback - INFO - Starting async job submit for node 69c480fb590270ad7b6d91f8 2026-03-28 00:42:42,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:42,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480fb590270ad7b6d91f8 HTTP/1.1" 200 None 2026-03-28 00:42:42,434 - lava_callback - DEBUG - Processing node 69c480fb590270ad7b6d91f8: baseline-arm-clabbe 2026-03-28 00:42:42,434 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c480fb590270ad7b6d91f8 2026-03-28 00:42:42,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:42,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c480fb590270ad7b6d91f8 HTTP/1.1" 200 None 2026-03-28 00:42:42,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:42,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:42:42,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:42:42,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c480fb590270ad7b6d91f8 HTTP/1.1" 200 None 2026-03-28 00:42:42,509 - lava_callback - ERROR - Error processing callback for node 69c480fb590270ad7b6d91f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57842 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:43:22,487 - lava_callback - INFO - Received callback for node 69c48121590270ad7b6d9217 INFO: 172.21.0.1:48998 - "POST /node/69c48121590270ad7b6d9217 HTTP/1.0" 202 Accepted 2026-03-28 00:43:22,493 - lava_callback - INFO - Starting async job submit for node 69c48121590270ad7b6d9217 2026-03-28 00:43:22,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:43:22,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48121590270ad7b6d9217 HTTP/1.1" 200 None 2026-03-28 00:43:22,509 - lava_callback - DEBUG - Processing node 69c48121590270ad7b6d9217: baseline-arm-clabbe 2026-03-28 00:43:22,509 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48121590270ad7b6d9217 2026-03-28 00:43:22,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:43:22,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48121590270ad7b6d9217 HTTP/1.1" 200 None 2026-03-28 00:43:22,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:43:22,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 00:43:22,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:43:22,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48121590270ad7b6d9217 HTTP/1.1" 200 None 2026-03-28 00:43:22,585 - lava_callback - ERROR - Error processing callback for node 69c48121590270ad7b6d9217: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40256 - "GET /node/694271c01b61f1c6005f7244%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:39634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53150 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51630 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56726 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52858 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:48:44,274 - lava_callback - INFO - Received callback for node 69c7256529015d0f9598f935 INFO: 172.21.0.1:49826 - "POST /node/69c7256529015d0f9598f935 HTTP/1.0" 202 Accepted 2026-03-28 00:48:44,281 - lava_callback - INFO - Starting async job submit for node 69c7256529015d0f9598f935 2026-03-28 00:48:44,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:48:44,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256529015d0f9598f935 HTTP/1.1" 200 None 2026-03-28 00:48:44,298 - lava_callback - DEBUG - Processing node 69c7256529015d0f9598f935: baseline-arm 2026-03-28 00:48:44,303 - lava_callback - WARNING - Failed to upload log 2026-03-28 00:48:44,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:48:44,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:48:44,449 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256529015d0f9598f935/lava-logs.yaml 2026-03-28 00:48:44,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:48:44,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:48:44,638 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256529015d0f9598f935/lava_callback.json.gz 2026-03-28 00:48:44,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:48:44,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256529015d0f9598f935 HTTP/1.1" 200 None 2026-03-28 00:48:44,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:48:44,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:48:44,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:48:44,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256529015d0f9598f935 HTTP/1.1" 200 None 2026-03-28 00:48:44,706 - lava_callback - INFO - Completed processing callback for node 69c7256529015d0f9598f935 INFO: 172.21.0.1:49840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47948 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:49:04,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:05,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-03-28 00:49:06,137 - lava_callback - INFO - Received callback for node 69c7256629015d0f9598f938 INFO: 172.21.0.1:54616 - "POST /node/69c7256629015d0f9598f938 HTTP/1.0" 202 Accepted 2026-03-28 00:49:06,186 - lava_callback - INFO - Starting async job submit for node 69c7256629015d0f9598f938 2026-03-28 00:49:06,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:06,243 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256629015d0f9598f938 HTTP/1.1" 200 None 2026-03-28 00:49:06,243 - lava_callback - DEBUG - Processing node 69c7256629015d0f9598f938: baseline-arm 2026-03-28 00:49:06,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:06,515 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:06,517 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f938/log.txt.gz INFO: 172.21.0.1:54620 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:49:06,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:06,885 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:06,888 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f938/lava-logs.yaml 2026-03-28 00:49:06,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:07,003 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:07,004 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f938/lava_callback.json.gz 2026-03-28 00:49:07,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:07,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256629015d0f9598f938 HTTP/1.1" 200 None 2026-03-28 00:49:07,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:07,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:49:07,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:07,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256629015d0f9598f938 HTTP/1.1" 200 None 2026-03-28 00:49:07,082 - lava_callback - INFO - Completed processing callback for node 69c7256629015d0f9598f938 INFO: 172.21.0.1:35910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35916 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:49:17,132 - lava_callback - INFO - Received callback for node 69c7256829015d0f9598f942 INFO: 172.21.0.1:35928 - "POST /node/69c7256829015d0f9598f942 HTTP/1.0" 202 Accepted 2026-03-28 00:49:17,185 - lava_callback - INFO - Starting async job submit for node 69c7256829015d0f9598f942 2026-03-28 00:49:17,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:17,236 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256829015d0f9598f942 HTTP/1.1" 200 None 2026-03-28 00:49:17,237 - lava_callback - DEBUG - Processing node 69c7256829015d0f9598f942: baseline-arm 2026-03-28 00:49:17,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:17,573 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:17,574 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256829015d0f9598f942/log.txt.gz 2026-03-28 00:49:17,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:17,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:17,819 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256829015d0f9598f942/lava-logs.yaml 2026-03-28 00:49:17,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:49:17,952 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:49:17,953 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256829015d0f9598f942/lava_callback.json.gz 2026-03-28 00:49:17,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:17,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256829015d0f9598f942 HTTP/1.1" 200 None 2026-03-28 00:49:17,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:18,006 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:49:18,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:18,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256829015d0f9598f942 HTTP/1.1" 200 None 2026-03-28 00:49:18,036 - lava_callback - INFO - Completed processing callback for node 69c7256829015d0f9598f942 INFO: 172.21.0.1:37462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37476 - "GET /node/6986faf86e077e4126a7d395%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-28 00:49:35,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:49:35,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:32972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49414 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:50:00,057 - lava_callback - INFO - Received callback for node 69c7256729015d0f9598f93f INFO: 172.21.0.1:45854 - "POST /node/69c7256729015d0f9598f93f HTTP/1.0" 202 Accepted 2026-03-28 00:50:00,063 - lava_callback - INFO - Starting async job submit for node 69c7256729015d0f9598f93f 2026-03-28 00:50:00,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:00,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256729015d0f9598f93f HTTP/1.1" 200 None 2026-03-28 00:50:00,100 - lava_callback - DEBUG - Processing node 69c7256729015d0f9598f93f: baseline-arm 2026-03-28 00:50:00,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:00,543 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:00,545 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256729015d0f9598f93f/log.txt.gz 2026-03-28 00:50:00,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:00,876 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:00,878 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256729015d0f9598f93f/lava-logs.yaml 2026-03-28 00:50:00,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:01,014 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:01,016 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256729015d0f9598f93f/lava_callback.json.gz 2026-03-28 00:50:01,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:01,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256729015d0f9598f93f HTTP/1.1" 200 None 2026-03-28 00:50:01,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:01,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:01,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:01,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256729015d0f9598f93f HTTP/1.1" 200 None 2026-03-28 00:50:01,102 - lava_callback - INFO - Completed processing callback for node 69c7256729015d0f9598f93f 2026-03-28 00:50:05,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:05,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45870 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:50:09,003 - lava_callback - INFO - Received callback for node 69c7256429015d0f9598f932 INFO: 172.21.0.1:45886 - "POST /node/69c7256429015d0f9598f932 HTTP/1.0" 202 Accepted 2026-03-28 00:50:09,009 - lava_callback - INFO - Starting async job submit for node 69c7256429015d0f9598f932 2026-03-28 00:50:09,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:09,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256429015d0f9598f932 HTTP/1.1" 200 None 2026-03-28 00:50:09,051 - lava_callback - DEBUG - Processing node 69c7256429015d0f9598f932: baseline-arm 2026-03-28 00:50:09,267 - lava_callback - INFO - Received callback for node 69c7257729015d0f9598f974 2026-03-28 00:50:09,273 - lava_callback - INFO - Starting async job submit for node 69c7257729015d0f9598f974 INFO: 172.21.0.1:45894 - "POST /node/69c7257729015d0f9598f974 HTTP/1.0" 202 Accepted 2026-03-28 00:50:09,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:09,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:09,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257729015d0f9598f974 HTTP/1.1" 200 None 2026-03-28 00:50:09,320 - lava_callback - DEBUG - Processing node 69c7257729015d0f9598f974: baseline-arm-broonie 2026-03-28 00:50:09,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:09,489 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256429015d0f9598f932/log.txt.gz 2026-03-28 00:50:09,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:09,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:09,786 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:09,788 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257729015d0f9598f974/log.txt.gz 2026-03-28 00:50:09,955 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:09,956 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:09,959 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256429015d0f9598f932/lava-logs.yaml 2026-03-28 00:50:09,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:10,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:10,173 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256429015d0f9598f932/lava_callback.json.gz 2026-03-28 00:50:10,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,197 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:10,199 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257729015d0f9598f974/lava-logs.yaml 2026-03-28 00:50:10,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:10,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256429015d0f9598f932 HTTP/1.1" 200 None 2026-03-28 00:50:10,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:10,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256429015d0f9598f932 HTTP/1.1" 200 None 2026-03-28 00:50:10,255 - lava_callback - INFO - Completed processing callback for node 69c7256429015d0f9598f932 2026-03-28 00:50:10,379 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:10,381 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257729015d0f9598f974/lava_callback.json.gz 2026-03-28 00:50:10,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257729015d0f9598f974 HTTP/1.1" 200 None 2026-03-28 00:50:10,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:10,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257729015d0f9598f974 HTTP/1.1" 200 None 2026-03-28 00:50:10,504 - lava_callback - INFO - Completed processing callback for node 69c7257729015d0f9598f974 2026-03-28 00:50:10,825 - lava_callback - INFO - Received callback for node 69c7257829015d0f9598f977 INFO: 172.21.0.1:39902 - "POST /node/69c7257829015d0f9598f977 HTTP/1.0" 202 Accepted 2026-03-28 00:50:10,831 - lava_callback - INFO - Starting async job submit for node 69c7257829015d0f9598f977 2026-03-28 00:50:10,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:10,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257829015d0f9598f977 HTTP/1.1" 200 None 2026-03-28 00:50:10,874 - lava_callback - DEBUG - Processing node 69c7257829015d0f9598f977: baseline-arm-broonie 2026-03-28 00:50:11,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:11,244 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:11,246 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257829015d0f9598f977/log.txt.gz 2026-03-28 00:50:11,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:11,577 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:11,579 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257829015d0f9598f977/lava-logs.yaml 2026-03-28 00:50:11,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:11,736 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:11,738 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257829015d0f9598f977/lava_callback.json.gz 2026-03-28 00:50:11,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:11,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257829015d0f9598f977 HTTP/1.1" 200 None 2026-03-28 00:50:11,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:11,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:11,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:11,832 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257829015d0f9598f977 HTTP/1.1" 200 None 2026-03-28 00:50:11,833 - lava_callback - INFO - Completed processing callback for node 69c7257829015d0f9598f977 INFO: 172.21.0.1:39912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38086 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:50:28,918 - lava_callback - INFO - Received callback for node 69c7258829015d0f9598f9d8 INFO: 172.21.0.1:38094 - "POST /node/69c7258829015d0f9598f9d8 HTTP/1.0" 202 Accepted 2026-03-28 00:50:28,924 - lava_callback - INFO - Starting async job submit for node 69c7258829015d0f9598f9d8 2026-03-28 00:50:28,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:28,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258829015d0f9598f9d8 HTTP/1.1" 200 None 2026-03-28 00:50:28,962 - lava_callback - DEBUG - Processing node 69c7258829015d0f9598f9d8: baseline-arm-pengutronix 2026-03-28 00:50:29,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:29,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:29,221 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258829015d0f9598f9d8/log.txt.gz 2026-03-28 00:50:29,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:29,475 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:29,477 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258829015d0f9598f9d8/lava-logs.yaml 2026-03-28 00:50:29,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:29,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:29,620 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258829015d0f9598f9d8/lava_callback.json.gz 2026-03-28 00:50:29,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:29,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258829015d0f9598f9d8 HTTP/1.1" 200 None 2026-03-28 00:50:29,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:29,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:29,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:29,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258829015d0f9598f9d8 HTTP/1.1" 200 None 2026-03-28 00:50:29,712 - lava_callback - INFO - Completed processing callback for node 69c7258829015d0f9598f9d8 2026-03-28 00:50:31,590 - lava_callback - INFO - Received callback for node 69c7258929015d0f9598f9dc INFO: 172.21.0.1:41070 - "POST /node/69c7258929015d0f9598f9dc HTTP/1.0" 202 Accepted 2026-03-28 00:50:31,599 - lava_callback - INFO - Starting async job submit for node 69c7258929015d0f9598f9dc 2026-03-28 00:50:31,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:31,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258929015d0f9598f9dc HTTP/1.1" 200 None 2026-03-28 00:50:31,652 - lava_callback - DEBUG - Processing node 69c7258929015d0f9598f9dc: baseline-arm-pengutronix 2026-03-28 00:50:31,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:31,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:31,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258929015d0f9598f9dc/log.txt.gz 2026-03-28 00:50:32,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:32,184 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:32,185 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258929015d0f9598f9dc/lava-logs.yaml 2026-03-28 00:50:32,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:32,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:32,318 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258929015d0f9598f9dc/lava_callback.json.gz 2026-03-28 00:50:32,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:32,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258929015d0f9598f9dc HTTP/1.1" 200 None 2026-03-28 00:50:32,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:32,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:32,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:32,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258929015d0f9598f9dc HTTP/1.1" 200 None 2026-03-28 00:50:32,399 - lava_callback - INFO - Completed processing callback for node 69c7258929015d0f9598f9dc 2026-03-28 00:50:35,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:35,100 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:41076 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:50:35,457 - lava_callback - INFO - Received callback for node 69c7258c29015d0f9598f9e4 INFO: 172.21.0.1:41092 - "POST /node/69c7258c29015d0f9598f9e4 HTTP/1.0" 202 Accepted 2026-03-28 00:50:35,508 - lava_callback - INFO - Starting async job submit for node 69c7258c29015d0f9598f9e4 2026-03-28 00:50:35,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:35,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258c29015d0f9598f9e4 HTTP/1.1" 200 None 2026-03-28 00:50:35,579 - lava_callback - DEBUG - Processing node 69c7258c29015d0f9598f9e4: baseline-arm-pengutronix 2026-03-28 00:50:35,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:36,066 - lava_callback - INFO - Received callback for node 69c7257e29015d0f9598f98b INFO: 172.21.0.1:41104 - "POST /node/69c7257e29015d0f9598f98b HTTP/1.0" 202 Accepted 2026-03-28 00:50:36,077 - lava_callback - INFO - Starting async job submit for node 69c7257e29015d0f9598f98b 2026-03-28 00:50:36,116 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:36,118 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258c29015d0f9598f9e4/log.txt.gz 2026-03-28 00:50:36,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:36,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257e29015d0f9598f98b HTTP/1.1" 200 None 2026-03-28 00:50:36,208 - lava_callback - DEBUG - Processing node 69c7257e29015d0f9598f98b: baseline-arm-clabbe 2026-03-28 00:50:36,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:41118 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:50:36,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:36,837 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:36,839 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258c29015d0f9598f9e4/lava-logs.yaml 2026-03-28 00:50:36,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:37,047 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:37,049 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257e29015d0f9598f98b/log.txt.gz 2026-03-28 00:50:37,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:37,184 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258c29015d0f9598f9e4/lava_callback.json.gz 2026-03-28 00:50:37,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:37,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258c29015d0f9598f9e4 HTTP/1.1" 200 None 2026-03-28 00:50:37,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:37,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:37,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:37,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:37,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258c29015d0f9598f9e4 HTTP/1.1" 200 None 2026-03-28 00:50:37,408 - lava_callback - INFO - Completed processing callback for node 69c7258c29015d0f9598f9e4 2026-03-28 00:50:37,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:37,945 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257e29015d0f9598f98b/lava-logs.yaml 2026-03-28 00:50:37,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:50:38,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:50:38,383 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257e29015d0f9598f98b/lava_callback.json.gz 2026-03-28 00:50:38,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:38,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257e29015d0f9598f98b HTTP/1.1" 200 None 2026-03-28 00:50:38,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:38,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:50:38,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:50:38,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257e29015d0f9598f98b HTTP/1.1" 200 None 2026-03-28 00:50:38,529 - lava_callback - INFO - Completed processing callback for node 69c7257e29015d0f9598f98b INFO: 172.21.0.1:53372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36052 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:51:05,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:05,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41286 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41298 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:51:25,047 - lava_callback - INFO - Received callback for node 69c725a429015d0f9598fa03 INFO: 172.21.0.1:40478 - "POST /node/69c725a429015d0f9598fa03 HTTP/1.0" 202 Accepted 2026-03-28 00:51:25,055 - lava_callback - INFO - Starting async job submit for node 69c725a429015d0f9598fa03 2026-03-28 00:51:25,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:25,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a429015d0f9598fa03 HTTP/1.1" 200 None 2026-03-28 00:51:25,104 - lava_callback - DEBUG - Processing node 69c725a429015d0f9598fa03: ltp-dio 2026-03-28 00:51:25,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:40494 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:51:26,611 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:51:26,613 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69c725a429015d0f9598fa03/log.txt.gz 2026-03-28 00:51:26,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:51:27,741 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:51:27,743 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69c725a429015d0f9598fa03/lava-logs.yaml 2026-03-28 00:51:27,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:51:28,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:51:28,661 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69c725a429015d0f9598fa03/lava_callback.json.gz 2026-03-28 00:51:28,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:28,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a429015d0f9598fa03 HTTP/1.1" 200 None 2026-03-28 00:51:28,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:28,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:51:28,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:28,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725a429015d0f9598fa03 HTTP/1.1" 200 None 2026-03-28 00:51:28,962 - lava_callback - INFO - Completed processing callback for node 69c725a429015d0f9598fa03 2026-03-28 00:51:35,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:51:35,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33476 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:52:07,144 - lava_callback - INFO - Received callback for node 69c7257f29015d0f9598f990 INFO: 172.21.0.1:33482 - "POST /node/69c7257f29015d0f9598f990 HTTP/1.0" 202 Accepted 2026-03-28 00:52:07,154 - lava_callback - INFO - Starting async job submit for node 69c7257f29015d0f9598f990 2026-03-28 00:52:07,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:07,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257f29015d0f9598f990 HTTP/1.1" 200 None 2026-03-28 00:52:07,214 - lava_callback - DEBUG - Processing node 69c7257f29015d0f9598f990: baseline-arm-clabbe 2026-03-28 00:52:07,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:08,913 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:08,915 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257f29015d0f9598f990/log.txt.gz 2026-03-28 00:52:09,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:10,574 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:10,576 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257f29015d0f9598f990/lava-logs.yaml 2026-03-28 00:52:10,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:11,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:11,937 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7257f29015d0f9598f990/lava_callback.json.gz 2026-03-28 00:52:11,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:11,995 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257f29015d0f9598f990 HTTP/1.1" 200 None 2026-03-28 00:52:11,997 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:12,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:52:12,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:12,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257f29015d0f9598f990 HTTP/1.1" 200 None 2026-03-28 00:52:12,054 - lava_callback - INFO - Completed processing callback for node 69c7257f29015d0f9598f990 INFO: 172.21.0.1:42368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33158 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:52:35,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:35,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:40294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40302 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:52:43,458 - lava_callback - INFO - Received callback for node 69c722aa29015d0f9598f834 INFO: 172.21.0.1:60074 - "POST /node/69c722aa29015d0f9598f834 HTTP/1.0" 202 Accepted 2026-03-28 00:52:43,473 - lava_callback - INFO - Starting async job submit for node 69c722aa29015d0f9598f834 2026-03-28 00:52:43,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:43,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722aa29015d0f9598f834 HTTP/1.1" 200 None 2026-03-28 00:52:43,558 - lava_callback - DEBUG - Processing node 69c722aa29015d0f9598f834: baseline-x86 2026-03-28 00:52:44,655 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:46,085 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:46,088 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c722aa29015d0f9598f834/log.txt.gz INFO: 172.21.0.1:60076 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:52:46,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:48,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:48,141 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c722aa29015d0f9598f834/lava-logs.yaml 2026-03-28 00:52:48,164 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:52:49,063 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:52:49,066 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c722aa29015d0f9598f834/lava_callback.json.gz DEBUG: Setup failed for 69c722aa29015d0f9598f834. Transitting job node result: incomplete -> fail 2026-03-28 00:52:49,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:49,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c722aa29015d0f9598f834 HTTP/1.1" 200 None 2026-03-28 00:52:49,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:49,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 00:52:49,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:52:49,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c722aa29015d0f9598f834 HTTP/1.1" 200 None 2026-03-28 00:52:49,435 - lava_callback - INFO - Completed processing callback for node 69c722aa29015d0f9598f834 INFO: 172.21.0.1:60276 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:53:05,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:53:05,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54102 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:53:54,307 - lava_callback - INFO - Received callback for node 69c7258229015d0f9598f99d INFO: 172.21.0.1:46830 - "POST /node/69c7258229015d0f9598f99d HTTP/1.0" 202 Accepted 2026-03-28 00:53:54,317 - lava_callback - INFO - Starting async job submit for node 69c7258229015d0f9598f99d 2026-03-28 00:53:54,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:53:54,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258229015d0f9598f99d HTTP/1.1" 200 None 2026-03-28 00:53:54,366 - lava_callback - DEBUG - Processing node 69c7258229015d0f9598f99d: baseline-arm-clabbe 2026-03-28 00:53:54,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:53:55,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:53:55,599 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258229015d0f9598f99d/log.txt.gz 2026-03-28 00:53:55,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46844 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:53:56,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:53:56,793 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258229015d0f9598f99d/lava-logs.yaml 2026-03-28 00:53:56,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:53:57,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:53:57,797 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258229015d0f9598f99d/lava_callback.json.gz 2026-03-28 00:53:57,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:53:57,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258229015d0f9598f99d HTTP/1.1" 200 None 2026-03-28 00:53:57,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:53:57,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:53:57,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:53:57,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258229015d0f9598f99d HTTP/1.1" 200 None 2026-03-28 00:53:57,921 - lava_callback - INFO - Completed processing callback for node 69c7258229015d0f9598f99d 2026-03-28 00:54:05,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:05,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60656 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:54:23,131 - lava_callback - INFO - Received callback for node 69c483b6590270ad7b6d9383 INFO: 172.21.0.1:37266 - "POST /node/69c483b6590270ad7b6d9383 HTTP/1.0" 202 Accepted 2026-03-28 00:54:23,139 - lava_callback - INFO - Starting async job submit for node 69c483b6590270ad7b6d9383 2026-03-28 00:54:23,140 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:23,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c483b6590270ad7b6d9383 HTTP/1.1" 200 None 2026-03-28 00:54:23,158 - lava_callback - DEBUG - Processing node 69c483b6590270ad7b6d9383: baseline-arm-clabbe 2026-03-28 00:54:23,158 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c483b6590270ad7b6d9383 2026-03-28 00:54:23,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:23,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c483b6590270ad7b6d9383 HTTP/1.1" 200 None 2026-03-28 00:54:23,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:23,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 00:54:23,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:23,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c483b6590270ad7b6d9383 HTTP/1.1" 200 None 2026-03-28 00:54:23,239 - lava_callback - ERROR - Error processing callback for node 69c483b6590270ad7b6d9383: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37270 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60196 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:54:38,167 - lava_callback - INFO - Received callback for node 69c7257a29015d0f9598f97a INFO: 172.21.0.1:60206 - "POST /node/69c7257a29015d0f9598f97a HTTP/1.0" 202 Accepted 2026-03-28 00:54:38,173 - lava_callback - INFO - Starting async job submit for node 69c7257a29015d0f9598f97a 2026-03-28 00:54:38,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:38,210 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257a29015d0f9598f97a HTTP/1.1" 200 None 2026-03-28 00:54:38,210 - lava_callback - DEBUG - Processing node 69c7257a29015d0f9598f97a: baseline-arm-broonie 2026-03-28 00:54:38,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:54:38,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:54:38,562 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257a29015d0f9598f97a/log.txt.gz 2026-03-28 00:54:38,700 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:54:38,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:54:38,865 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257a29015d0f9598f97a/lava-logs.yaml 2026-03-28 00:54:38,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:54:39,007 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:54:39,009 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257a29015d0f9598f97a/lava_callback.json.gz 2026-03-28 00:54:39,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:39,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257a29015d0f9598f97a HTTP/1.1" 200 None 2026-03-28 00:54:39,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:39,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:54:39,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:54:39,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257a29015d0f9598f97a HTTP/1.1" 200 None 2026-03-28 00:54:39,094 - lava_callback - INFO - Completed processing callback for node 69c7257a29015d0f9598f97a INFO: 172.21.0.1:36372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39706 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:55:05,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:05,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45184 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42890 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:55:29,360 - lava_callback - INFO - Received callback for node 69c7258329015d0f9598f9a4 INFO: 172.21.0.1:42898 - "POST /node/69c7258329015d0f9598f9a4 HTTP/1.0" 202 Accepted 2026-03-28 00:55:29,366 - lava_callback - INFO - Starting async job submit for node 69c7258329015d0f9598f9a4 2026-03-28 00:55:29,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:29,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258329015d0f9598f9a4 HTTP/1.1" 200 None 2026-03-28 00:55:29,411 - lava_callback - DEBUG - Processing node 69c7258329015d0f9598f9a4: baseline-arm-clabbe 2026-03-28 00:55:29,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:55:29,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:55:29,754 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258329015d0f9598f9a4/log.txt.gz 2026-03-28 00:55:29,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:55:30,110 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:55:30,112 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258329015d0f9598f9a4/lava-logs.yaml 2026-03-28 00:55:30,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:55:30,284 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:55:30,286 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69c7258329015d0f9598f9a4/lava_callback.json.gz 2026-03-28 00:55:30,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:30,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258329015d0f9598f9a4 HTTP/1.1" 200 None 2026-03-28 00:55:30,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:30,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:55:30,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:30,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258329015d0f9598f9a4 HTTP/1.1" 200 None 2026-03-28 00:55:30,375 - lava_callback - INFO - Completed processing callback for node 69c7258329015d0f9598f9a4 2026-03-28 00:55:35,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:35,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48994 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:55:43,381 - lava_callback - INFO - Received callback for node 69c483fe590270ad7b6d93ac INFO: 172.21.0.1:33102 - "POST /node/69c483fe590270ad7b6d93ac HTTP/1.0" 202 Accepted 2026-03-28 00:55:43,390 - lava_callback - INFO - Starting async job submit for node 69c483fe590270ad7b6d93ac 2026-03-28 00:55:43,391 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:43,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c483fe590270ad7b6d93ac HTTP/1.1" 200 None 2026-03-28 00:55:43,410 - lava_callback - DEBUG - Processing node 69c483fe590270ad7b6d93ac: baseline-arm-clabbe 2026-03-28 00:55:43,411 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c483fe590270ad7b6d93ac 2026-03-28 00:55:43,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:43,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c483fe590270ad7b6d93ac HTTP/1.1" 200 None 2026-03-28 00:55:43,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:43,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 00:55:43,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:55:43,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c483fe590270ad7b6d93ac HTTP/1.1" 200 None 2026-03-28 00:55:43,495 - lava_callback - ERROR - Error processing callback for node 69c483fe590270ad7b6d93ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39722 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39740 - "GET /node/6999066ad17b46e99fad2aa5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:34754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49042 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:56:43,224 - lava_callback - INFO - Received callback for node 69c48444590270ad7b6d93cb INFO: 172.21.0.1:52704 - "POST /node/69c48444590270ad7b6d93cb HTTP/1.0" 202 Accepted 2026-03-28 00:56:43,230 - lava_callback - INFO - Starting async job submit for node 69c48444590270ad7b6d93cb 2026-03-28 00:56:43,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:56:43,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48444590270ad7b6d93cb HTTP/1.1" 200 None 2026-03-28 00:56:43,256 - lava_callback - DEBUG - Processing node 69c48444590270ad7b6d93cb: baseline-arm-clabbe 2026-03-28 00:56:43,256 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48444590270ad7b6d93cb 2026-03-28 00:56:43,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:56:43,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48444590270ad7b6d93cb HTTP/1.1" 200 None 2026-03-28 00:56:43,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:56:43,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 00:56:43,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:56:43,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48444590270ad7b6d93cb HTTP/1.1" 200 None 2026-03-28 00:56:43,329 - lava_callback - ERROR - Error processing callback for node 69c48444590270ad7b6d93cb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51328 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:57:23,190 - lava_callback - INFO - Received callback for node 69c4846b590270ad7b6d93dd INFO: 172.21.0.1:37102 - "POST /node/69c4846b590270ad7b6d93dd HTTP/1.0" 202 Accepted 2026-03-28 00:57:23,197 - lava_callback - INFO - Starting async job submit for node 69c4846b590270ad7b6d93dd 2026-03-28 00:57:23,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:57:23,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4846b590270ad7b6d93dd HTTP/1.1" 200 None 2026-03-28 00:57:23,220 - lava_callback - DEBUG - Processing node 69c4846b590270ad7b6d93dd: baseline-arm-clabbe 2026-03-28 00:57:23,221 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4846b590270ad7b6d93dd 2026-03-28 00:57:23,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:57:23,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4846b590270ad7b6d93dd HTTP/1.1" 200 None 2026-03-28 00:57:23,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:57:23,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 00:57:23,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:57:23,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4846b590270ad7b6d93dd HTTP/1.1" 200 None 2026-03-28 00:57:23,287 - lava_callback - ERROR - Error processing callback for node 69c4846b590270ad7b6d93dd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54530 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:58:09,792 - lava_callback - INFO - Received callback for node 69c7231129015d0f9598f877 INFO: 172.21.0.1:58442 - "POST /node/69c7231129015d0f9598f877 HTTP/1.0" 202 Accepted 2026-03-28 00:58:09,799 - lava_callback - INFO - Starting async job submit for node 69c7231129015d0f9598f877 2026-03-28 00:58:09,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:09,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7231129015d0f9598f877 HTTP/1.1" 200 None 2026-03-28 00:58:09,893 - lava_callback - DEBUG - Processing node 69c7231129015d0f9598f877: ltp-smoke 2026-03-28 00:58:10,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:10,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:10,726 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7231129015d0f9598f877/log.txt.gz 2026-03-28 00:58:11,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:11,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:11,328 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7231129015d0f9598f877/lava-logs.yaml 2026-03-28 00:58:11,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:11,539 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:11,541 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7231129015d0f9598f877/lava_callback.json.gz 2026-03-28 00:58:11,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:11,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7231129015d0f9598f877 HTTP/1.1" 200 None 2026-03-28 00:58:11,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:11,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4229015d0f9598f742 HTTP/1.1" 200 None 2026-03-28 00:58:11,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:11,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7231129015d0f9598f877 HTTP/1.1" 200 None 2026-03-28 00:58:11,895 - lava_callback - INFO - Completed processing callback for node 69c7231129015d0f9598f877 INFO: 172.21.0.1:58458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58468 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:58:19,835 - lava_callback - INFO - Received callback for node 69c725b329015d0f9598fa2d INFO: 172.21.0.1:50454 - "POST /node/69c725b329015d0f9598fa2d HTTP/1.0" 202 Accepted 2026-03-28 00:58:19,843 - lava_callback - INFO - Starting async job submit for node 69c725b329015d0f9598fa2d 2026-03-28 00:58:19,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:19,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b329015d0f9598fa2d HTTP/1.1" 200 None 2026-03-28 00:58:19,898 - lava_callback - DEBUG - Processing node 69c725b329015d0f9598fa2d: ltp-syscalls 2026-03-28 00:58:20,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:21,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:21,003 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c725b329015d0f9598fa2d/log.txt.gz 2026-03-28 00:58:21,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:21,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:21,827 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c725b329015d0f9598fa2d/lava-logs.yaml 2026-03-28 00:58:21,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 00:58:22,024 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 00:58:22,026 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69c725b329015d0f9598fa2d/lava_callback.json.gz 2026-03-28 00:58:22,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:22,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b329015d0f9598fa2d HTTP/1.1" 200 None 2026-03-28 00:58:22,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:22,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 00:58:22,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:22,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725b329015d0f9598fa2d HTTP/1.1" 200 None 2026-03-28 00:58:22,128 - lava_callback - INFO - Completed processing callback for node 69c725b329015d0f9598fa2d INFO: 172.21.0.1:50470 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:58:35,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:35,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47206 - "GET / HTTP/1.0" 200 OK 2026-03-28 00:58:43,275 - lava_callback - INFO - Received callback for node 69c484b2590270ad7b6d93f5 INFO: 172.21.0.1:47858 - "POST /node/69c484b2590270ad7b6d93f5 HTTP/1.0" 202 Accepted 2026-03-28 00:58:43,283 - lava_callback - INFO - Starting async job submit for node 69c484b2590270ad7b6d93f5 2026-03-28 00:58:43,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:43,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c484b2590270ad7b6d93f5 HTTP/1.1" 200 None 2026-03-28 00:58:43,302 - lava_callback - DEBUG - Processing node 69c484b2590270ad7b6d93f5: baseline-arm-clabbe 2026-03-28 00:58:43,302 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c484b2590270ad7b6d93f5 2026-03-28 00:58:43,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:43,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c484b2590270ad7b6d93f5 HTTP/1.1" 200 None 2026-03-28 00:58:43,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:43,341 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 00:58:43,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 00:58:43,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c484b2590270ad7b6d93f5 HTTP/1.1" 200 None 2026-03-28 00:58:43,366 - lava_callback - ERROR - Error processing callback for node 69c484b2590270ad7b6d93f5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57946 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33546 - "GET /node/69876ee4e31b58f834f1a37b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:39424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42132 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:01:43,447 - lava_callback - INFO - Received callback for node 69c4856a590270ad7b6d9437 INFO: 172.21.0.1:55074 - "POST /node/69c4856a590270ad7b6d9437 HTTP/1.0" 202 Accepted 2026-03-28 01:01:43,452 - lava_callback - INFO - Starting async job submit for node 69c4856a590270ad7b6d9437 2026-03-28 01:01:43,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:01:43,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4856a590270ad7b6d9437 HTTP/1.1" 200 None 2026-03-28 01:01:43,468 - lava_callback - DEBUG - Processing node 69c4856a590270ad7b6d9437: baseline-arm-clabbe 2026-03-28 01:01:43,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4856a590270ad7b6d9437 2026-03-28 01:01:43,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:01:43,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4856a590270ad7b6d9437 HTTP/1.1" 200 None 2026-03-28 01:01:43,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:01:43,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-28 01:01:43,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:01:43,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4856a590270ad7b6d9437 HTTP/1.1" 200 None 2026-03-28 01:01:43,526 - lava_callback - ERROR - Error processing callback for node 69c4856a590270ad7b6d9437: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60496 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:02:23,430 - lava_callback - INFO - Received callback for node 69c48591590270ad7b6d943e INFO: 172.21.0.1:55668 - "POST /node/69c48591590270ad7b6d943e HTTP/1.0" 202 Accepted 2026-03-28 01:02:23,435 - lava_callback - INFO - Starting async job submit for node 69c48591590270ad7b6d943e 2026-03-28 01:02:23,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:02:23,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48591590270ad7b6d943e HTTP/1.1" 200 None 2026-03-28 01:02:23,451 - lava_callback - DEBUG - Processing node 69c48591590270ad7b6d943e: baseline-arm-clabbe 2026-03-28 01:02:23,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48591590270ad7b6d943e 2026-03-28 01:02:23,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:02:23,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48591590270ad7b6d943e HTTP/1.1" 200 None 2026-03-28 01:02:23,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:02:23,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-28 01:02:23,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:02:23,501 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48591590270ad7b6d943e HTTP/1.1" 200 None 2026-03-28 01:02:23,517 - lava_callback - ERROR - Error processing callback for node 69c48591590270ad7b6d943e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57886 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:03:03,501 - lava_callback - INFO - Received callback for node 69c485b8590270ad7b6d9448 INFO: 172.21.0.1:55200 - "POST /node/69c485b8590270ad7b6d9448 HTTP/1.0" 202 Accepted 2026-03-28 01:03:03,506 - lava_callback - INFO - Starting async job submit for node 69c485b8590270ad7b6d9448 2026-03-28 01:03:03,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:03,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c485b8590270ad7b6d9448 HTTP/1.1" 200 None 2026-03-28 01:03:03,525 - lava_callback - DEBUG - Processing node 69c485b8590270ad7b6d9448: baseline-arm-clabbe 2026-03-28 01:03:03,525 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c485b8590270ad7b6d9448 2026-03-28 01:03:03,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:03,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c485b8590270ad7b6d9448 HTTP/1.1" 200 None 2026-03-28 01:03:03,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:03,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-28 01:03:03,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:03,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c485b8590270ad7b6d9448 HTTP/1.1" 200 None 2026-03-28 01:03:03,620 - lava_callback - ERROR - Error processing callback for node 69c485b8590270ad7b6d9448: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58248 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:03:43,611 - lava_callback - INFO - Received callback for node 69c485dd590270ad7b6d9460 INFO: 172.21.0.1:38842 - "POST /node/69c485dd590270ad7b6d9460 HTTP/1.0" 202 Accepted 2026-03-28 01:03:43,616 - lava_callback - INFO - Starting async job submit for node 69c485dd590270ad7b6d9460 2026-03-28 01:03:43,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:43,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c485dd590270ad7b6d9460 HTTP/1.1" 200 None 2026-03-28 01:03:43,633 - lava_callback - DEBUG - Processing node 69c485dd590270ad7b6d9460: baseline-arm-clabbe 2026-03-28 01:03:43,633 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c485dd590270ad7b6d9460 2026-03-28 01:03:43,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:43,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c485dd590270ad7b6d9460 HTTP/1.1" 200 None 2026-03-28 01:03:43,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:43,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-28 01:03:43,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:03:43,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c485dd590270ad7b6d9460 HTTP/1.1" 200 None 2026-03-28 01:03:43,683 - lava_callback - ERROR - Error processing callback for node 69c485dd590270ad7b6d9460: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37408 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:04:23,610 - lava_callback - INFO - Received callback for node 69c48603590270ad7b6d946e INFO: 172.21.0.1:55432 - "POST /node/69c48603590270ad7b6d946e HTTP/1.0" 202 Accepted 2026-03-28 01:04:23,620 - lava_callback - INFO - Starting async job submit for node 69c48603590270ad7b6d946e 2026-03-28 01:04:23,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:04:23,654 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48603590270ad7b6d946e HTTP/1.1" 200 None 2026-03-28 01:04:23,655 - lava_callback - DEBUG - Processing node 69c48603590270ad7b6d946e: baseline-arm-clabbe 2026-03-28 01:04:23,656 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48603590270ad7b6d946e 2026-03-28 01:04:23,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:04:23,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48603590270ad7b6d946e HTTP/1.1" 200 None 2026-03-28 01:04:23,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:04:23,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92d5b4744551b27f9fbe HTTP/1.1" 200 None 2026-03-28 01:04:23,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:04:23,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48603590270ad7b6d946e HTTP/1.1" 200 None 2026-03-28 01:04:23,737 - lava_callback - ERROR - Error processing callback for node 69c48603590270ad7b6d946e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33048 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:06:54,239 - lava_callback - INFO - Received callback for node 69c7256629015d0f9598f93c INFO: 172.21.0.1:47258 - "POST /node/69c7256629015d0f9598f93c HTTP/1.0" 202 Accepted 2026-03-28 01:06:54,246 - lava_callback - INFO - Starting async job submit for node 69c7256629015d0f9598f93c 2026-03-28 01:06:54,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:54,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256629015d0f9598f93c HTTP/1.1" 200 None 2026-03-28 01:06:54,322 - lava_callback - DEBUG - Processing node 69c7256629015d0f9598f93c: baseline-arm 2026-03-28 01:06:54,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:06:54,989 - lava_callback - INFO - Received callback for node 69c7298f29015d0f95992185 INFO: 172.21.0.1:47268 - "POST /node/69c7298f29015d0f95992185 HTTP/1.0" 202 Accepted 2026-03-28 01:06:54,995 - lava_callback - INFO - Starting async job submit for node 69c7298f29015d0f95992185 2026-03-28 01:06:54,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:55,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7298f29015d0f95992185 HTTP/1.1" 200 None 2026-03-28 01:06:55,032 - lava_callback - DEBUG - Processing node 69c7298f29015d0f95992185: baseline-arm 2026-03-28 01:06:55,038 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:06:55,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:06:55,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:06:55,095 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f93c/log.txt.gz 2026-03-28 01:06:55,201 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:06:55,234 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7298f29015d0f95992185/lava-logs.yaml 2026-03-28 01:06:55,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:06:55,478 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:06:55,500 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7298f29015d0f95992185/lava_callback.json.gz 2026-03-28 01:06:55,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:55,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:06:55,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7298f29015d0f95992185 HTTP/1.1" 200 None 2026-03-28 01:06:55,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:55,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:06:55,538 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:55,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7298f29015d0f95992185 HTTP/1.1" 200 None 2026-03-28 01:06:55,569 - lava_callback - INFO - Completed processing callback for node 69c7298f29015d0f95992185 2026-03-28 01:06:55,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:06:55,826 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f93c/lava-logs.yaml 2026-03-28 01:06:55,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:06:56,004 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:06:56,006 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7256629015d0f9598f93c/lava_callback.json.gz DEBUG: Setup failed for 69c7256629015d0f9598f93c. Transitting job node result: incomplete -> fail 2026-03-28 01:06:56,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:56,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7256629015d0f9598f93c HTTP/1.1" 200 None 2026-03-28 01:06:56,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:56,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:06:56,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:06:56,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7256629015d0f9598f93c HTTP/1.1" 200 None 2026-03-28 01:06:56,302 - lava_callback - INFO - Completed processing callback for node 69c7256629015d0f9598f93c INFO: 172.21.0.1:47270 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:07:05,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:07:05,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45906 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:07:07,202 - lava_callback - INFO - Received callback for node 69c486b7590270ad7b6d9495 INFO: 172.21.0.1:45918 - "POST /node/69c486b7590270ad7b6d9495 HTTP/1.0" 202 Accepted 2026-03-28 01:07:07,208 - lava_callback - INFO - Starting async job submit for node 69c486b7590270ad7b6d9495 2026-03-28 01:07:07,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:07:07,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c486b7590270ad7b6d9495 HTTP/1.1" 200 None 2026-03-28 01:07:07,225 - lava_callback - DEBUG - Processing node 69c486b7590270ad7b6d9495: baseline-arm64-baylibre 2026-03-28 01:07:07,225 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c486b7590270ad7b6d9495 2026-03-28 01:07:07,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:07:07,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c486b7590270ad7b6d9495 HTTP/1.1" 200 None 2026-03-28 01:07:07,240 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:07:07,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:07:07,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:07:07,274 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c486b7590270ad7b6d9495 HTTP/1.1" 200 None 2026-03-28 01:07:07,276 - lava_callback - ERROR - Error processing callback for node 69c486b7590270ad7b6d9495: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38620 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49284 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:08:47,153 - lava_callback - INFO - Received callback for node 69c4870a590270ad7b6d94b7 INFO: 172.21.0.1:49296 - "POST /node/69c4870a590270ad7b6d94b7 HTTP/1.0" 202 Accepted 2026-03-28 01:08:47,159 - lava_callback - INFO - Starting async job submit for node 69c4870a590270ad7b6d94b7 2026-03-28 01:08:47,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:08:47,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4870a590270ad7b6d94b7 HTTP/1.1" 200 None 2026-03-28 01:08:47,330 - lava_callback - DEBUG - Processing node 69c4870a590270ad7b6d94b7: baseline-arm64-baylibre 2026-03-28 01:08:47,330 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4870a590270ad7b6d94b7 2026-03-28 01:08:47,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:08:47,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4870a590270ad7b6d94b7 HTTP/1.1" 200 None 2026-03-28 01:08:47,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:08:47,365 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:08:47,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:08:47,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4870a590270ad7b6d94b7 HTTP/1.1" 200 None 2026-03-28 01:08:47,392 - lava_callback - ERROR - Error processing callback for node 69c4870a590270ad7b6d94b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41230 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35620 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:09:38,556 - lava_callback - INFO - Received callback for node 69c7257529015d0f9598f971 INFO: 172.21.0.1:35634 - "POST /node/69c7257529015d0f9598f971 HTTP/1.0" 202 Accepted 2026-03-28 01:09:38,565 - lava_callback - INFO - Starting async job submit for node 69c7257529015d0f9598f971 2026-03-28 01:09:38,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:09:38,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257529015d0f9598f971 HTTP/1.1" 200 None 2026-03-28 01:09:38,611 - lava_callback - DEBUG - Processing node 69c7257529015d0f9598f971: baseline-arm-broonie 2026-03-28 01:09:38,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:09:39,100 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:09:39,102 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257529015d0f9598f971/log.txt.gz 2026-03-28 01:09:39,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:09:39,551 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:09:39,554 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257529015d0f9598f971/lava-logs.yaml 2026-03-28 01:09:39,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:09:39,727 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:09:39,730 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69c7257529015d0f9598f971/lava_callback.json.gz 2026-03-28 01:09:39,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:09:39,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7257529015d0f9598f971 HTTP/1.1" 200 None 2026-03-28 01:09:39,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:09:39,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:09:39,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:09:39,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7257529015d0f9598f971 HTTP/1.1" 200 None 2026-03-28 01:09:39,844 - lava_callback - INFO - Completed processing callback for node 69c7257529015d0f9598f971 INFO: 172.21.0.1:33918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53222 - "GET /node/698b677b0a16aea5a0dc75a5%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:53232 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:10:05,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:10:05,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33964 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:10:07,264 - lava_callback - INFO - Received callback for node 69c4875a590270ad7b6d94c4 INFO: 172.21.0.1:33974 - "POST /node/69c4875a590270ad7b6d94c4 HTTP/1.0" 202 Accepted 2026-03-28 01:10:07,272 - lava_callback - INFO - Starting async job submit for node 69c4875a590270ad7b6d94c4 2026-03-28 01:10:07,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:10:07,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4875a590270ad7b6d94c4 HTTP/1.1" 200 None 2026-03-28 01:10:07,290 - lava_callback - DEBUG - Processing node 69c4875a590270ad7b6d94c4: baseline-arm64-baylibre 2026-03-28 01:10:07,291 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4875a590270ad7b6d94c4 2026-03-28 01:10:07,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:10:07,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4875a590270ad7b6d94c4 HTTP/1.1" 200 None 2026-03-28 01:10:07,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:10:07,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:10:07,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:10:07,359 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4875a590270ad7b6d94c4 HTTP/1.1" 200 None 2026-03-28 01:10:07,361 - lava_callback - ERROR - Error processing callback for node 69c4875a590270ad7b6d94c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40722 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:11:27,298 - lava_callback - INFO - Received callback for node 69c487b1590270ad7b6d94cf INFO: 172.21.0.1:40734 - "POST /node/69c487b1590270ad7b6d94cf HTTP/1.0" 202 Accepted 2026-03-28 01:11:27,307 - lava_callback - INFO - Starting async job submit for node 69c487b1590270ad7b6d94cf 2026-03-28 01:11:27,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:11:27,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c487b1590270ad7b6d94cf HTTP/1.1" 200 None 2026-03-28 01:11:27,337 - lava_callback - DEBUG - Processing node 69c487b1590270ad7b6d94cf: baseline-arm64-baylibre 2026-03-28 01:11:27,337 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c487b1590270ad7b6d94cf 2026-03-28 01:11:27,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:11:27,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c487b1590270ad7b6d94cf HTTP/1.1" 200 None 2026-03-28 01:11:27,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:11:27,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:11:27,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:11:27,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c487b1590270ad7b6d94cf HTTP/1.1" 200 None 2026-03-28 01:11:27,453 - lava_callback - ERROR - Error processing callback for node 69c487b1590270ad7b6d94cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45240 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:12:47,390 - lava_callback - INFO - Received callback for node 69c48804590270ad7b6d94db INFO: 172.21.0.1:45256 - "POST /node/69c48804590270ad7b6d94db HTTP/1.0" 202 Accepted 2026-03-28 01:12:47,396 - lava_callback - INFO - Starting async job submit for node 69c48804590270ad7b6d94db 2026-03-28 01:12:47,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:12:47,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48804590270ad7b6d94db HTTP/1.1" 200 None 2026-03-28 01:12:47,415 - lava_callback - DEBUG - Processing node 69c48804590270ad7b6d94db: baseline-arm64-baylibre 2026-03-28 01:12:47,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48804590270ad7b6d94db 2026-03-28 01:12:47,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:12:47,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48804590270ad7b6d94db HTTP/1.1" 200 None 2026-03-28 01:12:47,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:12:47,444 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:12:47,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:12:47,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48804590270ad7b6d94db HTTP/1.1" 200 None 2026-03-28 01:12:47,478 - lava_callback - ERROR - Error processing callback for node 69c48804590270ad7b6d94db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49546 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:14:07,479 - lava_callback - INFO - Received callback for node 69c48855590270ad7b6d9523 INFO: 172.21.0.1:49560 - "POST /node/69c48855590270ad7b6d9523 HTTP/1.0" 202 Accepted 2026-03-28 01:14:07,488 - lava_callback - INFO - Starting async job submit for node 69c48855590270ad7b6d9523 2026-03-28 01:14:07,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:07,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48855590270ad7b6d9523 HTTP/1.1" 200 None 2026-03-28 01:14:07,514 - lava_callback - DEBUG - Processing node 69c48855590270ad7b6d9523: baseline-arm64-baylibre 2026-03-28 01:14:07,514 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48855590270ad7b6d9523 2026-03-28 01:14:07,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:07,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48855590270ad7b6d9523 HTTP/1.1" 200 None 2026-03-28 01:14:07,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:07,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:14:07,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:07,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48855590270ad7b6d9523 HTTP/1.1" 200 None 2026-03-28 01:14:07,581 - lava_callback - ERROR - Error processing callback for node 69c48855590270ad7b6d9523: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42470 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35010 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:14:27,426 - lava_callback - INFO - Received callback for node 69c4886c590270ad7b6d9575 INFO: 172.21.0.1:35024 - "POST /node/69c4886c590270ad7b6d9575 HTTP/1.0" 202 Accepted 2026-03-28 01:14:27,431 - lava_callback - INFO - Starting async job submit for node 69c4886c590270ad7b6d9575 2026-03-28 01:14:27,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:27,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4886c590270ad7b6d9575 HTTP/1.1" 200 None 2026-03-28 01:14:27,462 - lava_callback - DEBUG - Processing node 69c4886c590270ad7b6d9575: baseline-arm64-baylibre 2026-03-28 01:14:27,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4886c590270ad7b6d9575 2026-03-28 01:14:27,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:27,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4886c590270ad7b6d9575 HTTP/1.1" 200 None 2026-03-28 01:14:27,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:27,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:14:27,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:27,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4886c590270ad7b6d9575 HTTP/1.1" 200 None 2026-03-28 01:14:27,516 - lava_callback - ERROR - Error processing callback for node 69c4886c590270ad7b6d9575: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39176 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:14:43,972 - lava_callback - INFO - Received callback for node 69c4887e590270ad7b6d95bd INFO: 172.21.0.1:52784 - "POST /node/69c4887e590270ad7b6d95bd HTTP/1.0" 202 Accepted 2026-03-28 01:14:43,978 - lava_callback - INFO - Starting async job submit for node 69c4887e590270ad7b6d95bd 2026-03-28 01:14:43,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4887e590270ad7b6d95bd HTTP/1.1" 200 None 2026-03-28 01:14:44,005 - lava_callback - DEBUG - Processing node 69c4887e590270ad7b6d95bd: baseline-arm64-clabbe 2026-03-28 01:14:44,006 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4887e590270ad7b6d95bd 2026-03-28 01:14:44,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4887e590270ad7b6d95bd HTTP/1.1" 200 None 2026-03-28 01:14:44,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:14:44,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4887e590270ad7b6d95bd HTTP/1.1" 200 None 2026-03-28 01:14:44,057 - lava_callback - ERROR - Error processing callback for node 69c4887e590270ad7b6d95bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:14:44,658 - lava_callback - INFO - Received callback for node 69c48880590270ad7b6d95c0 INFO: 172.21.0.1:52792 - "POST /node/69c48880590270ad7b6d95c0 HTTP/1.0" 202 Accepted 2026-03-28 01:14:44,665 - lava_callback - INFO - Starting async job submit for node 69c48880590270ad7b6d95c0 2026-03-28 01:14:44,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48880590270ad7b6d95c0 HTTP/1.1" 200 None 2026-03-28 01:14:44,686 - lava_callback - DEBUG - Processing node 69c48880590270ad7b6d95c0: baseline-arm64-clabbe 2026-03-28 01:14:44,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48880590270ad7b6d95c0 2026-03-28 01:14:44,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48880590270ad7b6d95c0 HTTP/1.1" 200 None 2026-03-28 01:14:44,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:14:44,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:44,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48880590270ad7b6d95c0 HTTP/1.1" 200 None 2026-03-28 01:14:44,744 - lava_callback - ERROR - Error processing callback for node 69c48880590270ad7b6d95c0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:14:45,334 - lava_callback - INFO - Received callback for node 69c48881590270ad7b6d95c3 INFO: 172.21.0.1:52804 - "POST /node/69c48881590270ad7b6d95c3 HTTP/1.0" 202 Accepted 2026-03-28 01:14:45,339 - lava_callback - INFO - Starting async job submit for node 69c48881590270ad7b6d95c3 2026-03-28 01:14:45,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:45,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48881590270ad7b6d95c3 HTTP/1.1" 200 None 2026-03-28 01:14:45,355 - lava_callback - DEBUG - Processing node 69c48881590270ad7b6d95c3: baseline-arm64-clabbe 2026-03-28 01:14:45,356 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48881590270ad7b6d95c3 2026-03-28 01:14:45,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:45,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48881590270ad7b6d95c3 HTTP/1.1" 200 None 2026-03-28 01:14:45,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:45,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:14:45,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:45,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48881590270ad7b6d95c3 HTTP/1.1" 200 None 2026-03-28 01:14:45,419 - lava_callback - ERROR - Error processing callback for node 69c48881590270ad7b6d95c3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:14:46,029 - lava_callback - INFO - Received callback for node 69c48882590270ad7b6d95c6 INFO: 172.21.0.1:52810 - "POST /node/69c48882590270ad7b6d95c6 HTTP/1.0" 202 Accepted 2026-03-28 01:14:46,034 - lava_callback - INFO - Starting async job submit for node 69c48882590270ad7b6d95c6 2026-03-28 01:14:46,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:46,056 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48882590270ad7b6d95c6 HTTP/1.1" 200 None 2026-03-28 01:14:46,057 - lava_callback - DEBUG - Processing node 69c48882590270ad7b6d95c6: baseline-arm64-clabbe 2026-03-28 01:14:46,057 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48882590270ad7b6d95c6 2026-03-28 01:14:46,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:46,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48882590270ad7b6d95c6 HTTP/1.1" 200 None 2026-03-28 01:14:46,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:46,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:14:46,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:46,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48882590270ad7b6d95c6 HTTP/1.1" 200 None 2026-03-28 01:14:46,114 - lava_callback - ERROR - Error processing callback for node 69c48882590270ad7b6d95c6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52826 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:14:50,553 - lava_callback - INFO - Received callback for node 69c7259d29015d0f9598f9ef INFO: 172.21.0.1:59312 - "POST /node/69c7259d29015d0f9598f9ef HTTP/1.0" 202 Accepted 2026-03-28 01:14:50,560 - lava_callback - INFO - Starting async job submit for node 69c7259d29015d0f9598f9ef 2026-03-28 01:14:50,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:50,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7259d29015d0f9598f9ef HTTP/1.1" 200 None 2026-03-28 01:14:50,600 - lava_callback - DEBUG - Processing node 69c7259d29015d0f9598f9ef: ltp-cap_bounds 2026-03-28 01:14:50,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:14:51,113 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:14:51,115 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c7259d29015d0f9598f9ef/log.txt.gz 2026-03-28 01:14:51,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:14:51,563 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:14:51,565 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c7259d29015d0f9598f9ef/lava-logs.yaml 2026-03-28 01:14:51,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:14:51,731 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:14:51,733 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69c7259d29015d0f9598f9ef/lava_callback.json.gz 2026-03-28 01:14:51,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:51,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7259d29015d0f9598f9ef HTTP/1.1" 200 None 2026-03-28 01:14:51,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:51,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:14:51,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:14:51,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7259d29015d0f9598f9ef HTTP/1.1" 200 None 2026-03-28 01:14:51,822 - lava_callback - INFO - Completed processing callback for node 69c7259d29015d0f9598f9ef INFO: 172.21.0.1:59328 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:15:03,958 - lava_callback - INFO - Received callback for node 69c48884590270ad7b6d95cc INFO: 172.21.0.1:54014 - "POST /node/69c48884590270ad7b6d95cc HTTP/1.0" 202 Accepted 2026-03-28 01:15:03,964 - lava_callback - INFO - Starting async job submit for node 69c48884590270ad7b6d95cc 2026-03-28 01:15:03,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:03,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48884590270ad7b6d95cc HTTP/1.1" 200 None 2026-03-28 01:15:03,980 - lava_callback - DEBUG - Processing node 69c48884590270ad7b6d95cc: baseline-arm64-clabbe 2026-03-28 01:15:03,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48884590270ad7b6d95cc 2026-03-28 01:15:03,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:03,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48884590270ad7b6d95cc HTTP/1.1" 200 None 2026-03-28 01:15:03,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:15:04,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48884590270ad7b6d95cc HTTP/1.1" 200 None 2026-03-28 01:15:04,043 - lava_callback - ERROR - Error processing callback for node 69c48884590270ad7b6d95cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:15:04,666 - lava_callback - INFO - Received callback for node 69c48885590270ad7b6d95cf INFO: 172.21.0.1:54026 - "POST /node/69c48885590270ad7b6d95cf HTTP/1.0" 202 Accepted 2026-03-28 01:15:04,672 - lava_callback - INFO - Starting async job submit for node 69c48885590270ad7b6d95cf 2026-03-28 01:15:04,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48885590270ad7b6d95cf HTTP/1.1" 200 None 2026-03-28 01:15:04,688 - lava_callback - DEBUG - Processing node 69c48885590270ad7b6d95cf: baseline-arm64-clabbe 2026-03-28 01:15:04,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48885590270ad7b6d95cf 2026-03-28 01:15:04,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48885590270ad7b6d95cf HTTP/1.1" 200 None 2026-03-28 01:15:04,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 01:15:04,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:04,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48885590270ad7b6d95cf HTTP/1.1" 200 None 2026-03-28 01:15:04,751 - lava_callback - ERROR - Error processing callback for node 69c48885590270ad7b6d95cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:15:05,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:05,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52584 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:15:39,495 - lava_callback - INFO - Received callback for node 69c7259f29015d0f9598f9f8 INFO: 172.21.0.1:52592 - "POST /node/69c7259f29015d0f9598f9f8 HTTP/1.0" 202 Accepted 2026-03-28 01:15:39,502 - lava_callback - INFO - Starting async job submit for node 69c7259f29015d0f9598f9f8 2026-03-28 01:15:39,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:39,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7259f29015d0f9598f9f8 HTTP/1.1" 200 None 2026-03-28 01:15:39,542 - lava_callback - DEBUG - Processing node 69c7259f29015d0f9598f9f8: ltp-containers 2026-03-28 01:15:39,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:15:40,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:15:40,113 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69c7259f29015d0f9598f9f8/log.txt.gz 2026-03-28 01:15:40,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:15:40,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:15:40,533 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69c7259f29015d0f9598f9f8/lava-logs.yaml 2026-03-28 01:15:40,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:15:40,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:15:40,714 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69c7259f29015d0f9598f9f8/lava_callback.json.gz 2026-03-28 01:15:40,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:40,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7259f29015d0f9598f9f8 HTTP/1.1" 200 None 2026-03-28 01:15:40,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:40,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:15:40,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:40,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7259f29015d0f9598f9f8 HTTP/1.1" 200 None 2026-03-28 01:15:40,809 - lava_callback - INFO - Completed processing callback for node 69c7259f29015d0f9598f9f8 2026-03-28 01:15:43,988 - lava_callback - INFO - Received callback for node 69c488ad590270ad7b6d9628 INFO: 172.21.0.1:45962 - "POST /node/69c488ad590270ad7b6d9628 HTTP/1.0" 202 Accepted 2026-03-28 01:15:43,993 - lava_callback - INFO - Starting async job submit for node 69c488ad590270ad7b6d9628 2026-03-28 01:15:43,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:44,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c488ad590270ad7b6d9628 HTTP/1.1" 200 None 2026-03-28 01:15:44,009 - lava_callback - DEBUG - Processing node 69c488ad590270ad7b6d9628: baseline-arm64-clabbe 2026-03-28 01:15:44,010 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c488ad590270ad7b6d9628 2026-03-28 01:15:44,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:44,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c488ad590270ad7b6d9628 HTTP/1.1" 200 None 2026-03-28 01:15:44,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:44,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:15:44,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:44,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c488ad590270ad7b6d9628 HTTP/1.1" 200 None 2026-03-28 01:15:44,068 - lava_callback - ERROR - Error processing callback for node 69c488ad590270ad7b6d9628: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55604 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:15:59,125 - lava_callback - INFO - Received callback for node 69c725a029015d0f9598f9fb INFO: 172.21.0.1:55612 - "POST /node/69c725a029015d0f9598f9fb HTTP/1.0" 202 Accepted 2026-03-28 01:15:59,132 - lava_callback - INFO - Starting async job submit for node 69c725a029015d0f9598f9fb 2026-03-28 01:15:59,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:15:59,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a029015d0f9598f9fb HTTP/1.1" 200 None 2026-03-28 01:15:59,172 - lava_callback - DEBUG - Processing node 69c725a029015d0f9598f9fb: ltp-controllers 2026-03-28 01:15:59,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:15:59,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:15:59,776 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69c725a029015d0f9598f9fb/log.txt.gz 2026-03-28 01:16:00,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:16:00,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:16:00,267 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69c725a029015d0f9598f9fb/lava-logs.yaml 2026-03-28 01:16:00,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:16:00,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:16:00,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69c725a029015d0f9598f9fb/lava_callback.json.gz 2026-03-28 01:16:00,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:16:00,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a029015d0f9598f9fb HTTP/1.1" 200 None 2026-03-28 01:16:00,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:16:00,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:16:00,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:16:00,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725a029015d0f9598f9fb HTTP/1.1" 200 None 2026-03-28 01:16:00,560 - lava_callback - INFO - Completed processing callback for node 69c725a029015d0f9598f9fb 2026-03-28 01:16:05,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:16:05,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45662 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:17:04,094 - lava_callback - INFO - Received callback for node 69c488ff590270ad7b6d96cd INFO: 172.21.0.1:40072 - "POST /node/69c488ff590270ad7b6d96cd HTTP/1.0" 202 Accepted 2026-03-28 01:17:04,099 - lava_callback - INFO - Starting async job submit for node 69c488ff590270ad7b6d96cd 2026-03-28 01:17:04,100 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:04,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c488ff590270ad7b6d96cd HTTP/1.1" 200 None 2026-03-28 01:17:04,118 - lava_callback - DEBUG - Processing node 69c488ff590270ad7b6d96cd: baseline-arm64-clabbe 2026-03-28 01:17:04,118 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c488ff590270ad7b6d96cd 2026-03-28 01:17:04,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:04,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c488ff590270ad7b6d96cd HTTP/1.1" 200 None 2026-03-28 01:17:04,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:04,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:17:04,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:04,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c488ff590270ad7b6d96cd HTTP/1.1" 200 None 2026-03-28 01:17:04,183 - lava_callback - ERROR - Error processing callback for node 69c488ff590270ad7b6d96cd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35518 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:17:42,568 - lava_callback - INFO - Received callback for node 69c72bde29015d0f95992257 INFO: 172.21.0.1:50438 - "POST /node/69c72bde29015d0f95992257 HTTP/1.0" 202 Accepted 2026-03-28 01:17:42,574 - lava_callback - INFO - Starting async job submit for node 69c72bde29015d0f95992257 2026-03-28 01:17:42,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:42,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72bde29015d0f95992257 HTTP/1.1" 200 None 2026-03-28 01:17:42,608 - lava_callback - DEBUG - Processing node 69c72bde29015d0f95992257: baseline-x86 2026-03-28 01:17:42,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:17:43,023 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:17:43,024 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69c72bde29015d0f95992257/log.txt.gz 2026-03-28 01:17:43,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:17:43,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:17:43,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69c72bde29015d0f95992257/lava-logs.yaml 2026-03-28 01:17:43,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:17:43,545 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:17:43,547 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69c72bde29015d0f95992257/lava_callback.json.gz 2026-03-28 01:17:43,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:43,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72bde29015d0f95992257 HTTP/1.1" 200 None 2026-03-28 01:17:43,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:43,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e6729015d0f9598f76e HTTP/1.1" 200 None 2026-03-28 01:17:43,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:17:43,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72bde29015d0f95992257 HTTP/1.1" 200 None 2026-03-28 01:17:43,639 - lava_callback - INFO - Completed processing callback for node 69c72bde29015d0f95992257 INFO: 172.21.0.1:50440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59962 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:18:05,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:18:05,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43678 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43684 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:18:24,159 - lava_callback - INFO - Received callback for node 69c48951590270ad7b6d96f1 INFO: 172.21.0.1:59398 - "POST /node/69c48951590270ad7b6d96f1 HTTP/1.0" 202 Accepted 2026-03-28 01:18:24,171 - lava_callback - INFO - Starting async job submit for node 69c48951590270ad7b6d96f1 2026-03-28 01:18:24,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:18:24,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48951590270ad7b6d96f1 HTTP/1.1" 200 None 2026-03-28 01:18:24,198 - lava_callback - DEBUG - Processing node 69c48951590270ad7b6d96f1: baseline-arm64-clabbe 2026-03-28 01:18:24,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48951590270ad7b6d96f1 2026-03-28 01:18:24,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:18:24,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48951590270ad7b6d96f1 HTTP/1.1" 200 None 2026-03-28 01:18:24,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:18:24,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:18:24,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:18:24,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48951590270ad7b6d96f1 HTTP/1.1" 200 None 2026-03-28 01:18:24,324 - lava_callback - ERROR - Error processing callback for node 69c48951590270ad7b6d96f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60168 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60182 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:19:44,282 - lava_callback - INFO - Received callback for node 69c489a1590270ad7b6d9711 INFO: 172.21.0.1:44874 - "POST /node/69c489a1590270ad7b6d9711 HTTP/1.0" 202 Accepted 2026-03-28 01:19:44,291 - lava_callback - INFO - Starting async job submit for node 69c489a1590270ad7b6d9711 2026-03-28 01:19:44,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:44,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c489a1590270ad7b6d9711 HTTP/1.1" 200 None 2026-03-28 01:19:44,319 - lava_callback - DEBUG - Processing node 69c489a1590270ad7b6d9711: baseline-arm64-clabbe 2026-03-28 01:19:44,319 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c489a1590270ad7b6d9711 2026-03-28 01:19:44,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:44,343 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c489a1590270ad7b6d9711 HTTP/1.1" 200 None 2026-03-28 01:19:44,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:44,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:19:44,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:44,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c489a1590270ad7b6d9711 HTTP/1.1" 200 None 2026-03-28 01:19:44,404 - lava_callback - ERROR - Error processing callback for node 69c489a1590270ad7b6d9711: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45722 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:19:58,832 - lava_callback - INFO - Received callback for node 69c72ca929015d0f959922b8 INFO: 172.21.0.1:45736 - "POST /node/69c72ca929015d0f959922b8 HTTP/1.0" 202 Accepted 2026-03-28 01:19:58,842 - lava_callback - INFO - Starting async job submit for node 69c72ca929015d0f959922b8 2026-03-28 01:19:58,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:58,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ca929015d0f959922b8 HTTP/1.1" 200 None 2026-03-28 01:19:58,887 - lava_callback - DEBUG - Processing node 69c72ca929015d0f959922b8: baseline-arm 2026-03-28 01:19:58,895 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:19:58,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:19:59,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:19:59,121 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c72ca929015d0f959922b8/lava-logs.yaml 2026-03-28 01:19:59,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:19:59,235 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:19:59,237 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c72ca929015d0f959922b8/lava_callback.json.gz 2026-03-28 01:19:59,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:59,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ca929015d0f959922b8 HTTP/1.1" 200 None 2026-03-28 01:19:59,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:59,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:19:59,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:19:59,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ca929015d0f959922b8 HTTP/1.1" 200 None 2026-03-28 01:19:59,339 - lava_callback - INFO - Completed processing callback for node 69c72ca929015d0f959922b8 2026-03-28 01:20:05,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:05,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54778 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:20:57,592 - lava_callback - INFO - Received callback for node 69c72ccf29015d0f959922d9 INFO: 172.21.0.1:54794 - "POST /node/69c72ccf29015d0f959922d9 HTTP/1.0" 202 Accepted 2026-03-28 01:20:57,599 - lava_callback - INFO - Starting async job submit for node 69c72ccf29015d0f959922d9 2026-03-28 01:20:57,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:57,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ccf29015d0f959922d9 HTTP/1.1" 200 None 2026-03-28 01:20:57,677 - lava_callback - DEBUG - Processing node 69c72ccf29015d0f959922d9: baseline-arm64 2026-03-28 01:20:57,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:20:58,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:20:58,011 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922d9/log.txt.gz 2026-03-28 01:20:58,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:20:58,286 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:20:58,288 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922d9/lava-logs.yaml 2026-03-28 01:20:58,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:20:58,480 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:20:58,482 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922d9/lava_callback.json.gz 2026-03-28 01:20:58,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:58,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ccf29015d0f959922d9 HTTP/1.1" 200 None 2026-03-28 01:20:58,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:58,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:20:58,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:58,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ccf29015d0f959922d9 HTTP/1.1" 200 None 2026-03-28 01:20:58,624 - lava_callback - INFO - Completed processing callback for node 69c72ccf29015d0f959922d9 2026-03-28 01:20:59,449 - lava_callback - INFO - Received callback for node 69c72cee29015d0f9599234a INFO: 172.21.0.1:54802 - "POST /node/69c72cee29015d0f9599234a HTTP/1.0" 202 Accepted 2026-03-28 01:20:59,455 - lava_callback - INFO - Starting async job submit for node 69c72cee29015d0f9599234a 2026-03-28 01:20:59,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:59,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cee29015d0f9599234a HTTP/1.1" 200 None 2026-03-28 01:20:59,476 - lava_callback - DEBUG - Processing node 69c72cee29015d0f9599234a: baseline-nfs-arm64 2026-03-28 01:20:59,478 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:20:59,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:20:59,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:20:59,604 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cee29015d0f9599234a/lava-logs.yaml 2026-03-28 01:20:59,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:20:59,729 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:20:59,731 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cee29015d0f9599234a/lava_callback.json.gz 2026-03-28 01:20:59,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:59,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cee29015d0f9599234a HTTP/1.1" 200 None 2026-03-28 01:20:59,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:59,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:20:59,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:20:59,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cee29015d0f9599234a HTTP/1.1" 200 None 2026-03-28 01:20:59,790 - lava_callback - INFO - Completed processing callback for node 69c72cee29015d0f9599234a 2026-03-28 01:21:04,265 - lava_callback - INFO - Received callback for node 69c489f1590270ad7b6d9727 INFO: 172.21.0.1:59454 - "POST /node/69c489f1590270ad7b6d9727 HTTP/1.0" 202 Accepted 2026-03-28 01:21:04,274 - lava_callback - INFO - Starting async job submit for node 69c489f1590270ad7b6d9727 2026-03-28 01:21:04,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:04,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c489f1590270ad7b6d9727 HTTP/1.1" 200 None 2026-03-28 01:21:04,298 - lava_callback - DEBUG - Processing node 69c489f1590270ad7b6d9727: baseline-arm64-clabbe 2026-03-28 01:21:04,298 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c489f1590270ad7b6d9727 2026-03-28 01:21:04,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:04,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c489f1590270ad7b6d9727 HTTP/1.1" 200 None 2026-03-28 01:21:04,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:04,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:21:04,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:04,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c489f1590270ad7b6d9727 HTTP/1.1" 200 None 2026-03-28 01:21:04,355 - lava_callback - ERROR - Error processing callback for node 69c489f1590270ad7b6d9727: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:21:05,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:05,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59468 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:21:06,633 - lava_callback - INFO - Received callback for node 69c72cdc29015d0f959922fd INFO: 172.21.0.1:59476 - "POST /node/69c72cdc29015d0f959922fd HTTP/1.0" 202 Accepted 2026-03-28 01:21:06,640 - lava_callback - INFO - Starting async job submit for node 69c72cdc29015d0f959922fd 2026-03-28 01:21:06,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:06,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdc29015d0f959922fd HTTP/1.1" 200 None 2026-03-28 01:21:06,674 - lava_callback - DEBUG - Processing node 69c72cdc29015d0f959922fd: baseline-arm64 2026-03-28 01:21:06,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:06,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:06,958 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdc29015d0f959922fd/log.txt.gz 2026-03-28 01:21:07,045 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:07,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:07,253 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdc29015d0f959922fd/lava-logs.yaml 2026-03-28 01:21:07,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:07,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:07,441 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdc29015d0f959922fd/lava_callback.json.gz 2026-03-28 01:21:07,472 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:07,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdc29015d0f959922fd HTTP/1.1" 200 None 2026-03-28 01:21:07,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:07,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:07,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:07,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cdc29015d0f959922fd HTTP/1.1" 200 None 2026-03-28 01:21:07,539 - lava_callback - INFO - Completed processing callback for node 69c72cdc29015d0f959922fd INFO: 172.21.0.1:46368 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46382 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:21:23,895 - lava_callback - INFO - Received callback for node 69c72cd329015d0f959922e8 INFO: 172.21.0.1:46250 - "POST /node/69c72cd329015d0f959922e8 HTTP/1.0" 202 Accepted 2026-03-28 01:21:23,907 - lava_callback - INFO - Starting async job submit for node 69c72cd329015d0f959922e8 2026-03-28 01:21:23,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:24,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd329015d0f959922e8 HTTP/1.1" 200 None 2026-03-28 01:21:24,012 - lava_callback - DEBUG - Processing node 69c72cd329015d0f959922e8: baseline-arm64 2026-03-28 01:21:24,155 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:21:24,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:46264 - "GET /node/69936dcdcebe99dd8e0680d4%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-28 01:21:24,394 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:24,395 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd329015d0f959922e8/lava-logs.yaml 2026-03-28 01:21:24,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:24,505 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:24,507 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd329015d0f959922e8/lava_callback.json.gz 2026-03-28 01:21:24,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:24,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd329015d0f959922e8 HTTP/1.1" 200 None 2026-03-28 01:21:24,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:24,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:24,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:24,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd329015d0f959922e8 HTTP/1.1" 200 None 2026-03-28 01:21:24,659 - lava_callback - INFO - Completed processing callback for node 69c72cd329015d0f959922e8 INFO: 172.21.0.1:46278 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:21:33,277 - lava_callback - INFO - Received callback for node 69c72ce229015d0f95992309 INFO: 172.21.0.1:48852 - "POST /node/69c72ce229015d0f95992309 HTTP/1.0" 202 Accepted 2026-03-28 01:21:33,287 - lava_callback - INFO - Starting async job submit for node 69c72ce229015d0f95992309 2026-03-28 01:21:33,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:33,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce229015d0f95992309 HTTP/1.1" 200 None 2026-03-28 01:21:33,332 - lava_callback - DEBUG - Processing node 69c72ce229015d0f95992309: baseline-arm64 2026-03-28 01:21:33,579 - lava_callback - INFO - Received callback for node 69c72cd429015d0f959922eb INFO: 172.21.0.1:48864 - "POST /node/69c72cd429015d0f959922eb HTTP/1.0" 202 Accepted 2026-03-28 01:21:33,611 - lava_callback - INFO - Starting async job submit for node 69c72cd429015d0f959922eb 2026-03-28 01:21:33,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:33,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:33,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd429015d0f959922eb HTTP/1.1" 200 None 2026-03-28 01:21:33,743 - lava_callback - DEBUG - Processing node 69c72cd429015d0f959922eb: baseline-arm64 2026-03-28 01:21:33,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:33,895 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce229015d0f95992309/log.txt.gz 2026-03-28 01:21:33,920 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:21:34,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:34,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:34,393 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:34,396 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce229015d0f95992309/lava-logs.yaml 2026-03-28 01:21:34,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:34,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:34,410 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd429015d0f959922eb/lava-logs.yaml 2026-03-28 01:21:34,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:34,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:34,611 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce229015d0f95992309/lava_callback.json.gz 2026-03-28 01:21:34,637 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:34,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,639 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd429015d0f959922eb/lava_callback.json.gz 2026-03-28 01:21:34,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce229015d0f95992309 HTTP/1.1" 200 None 2026-03-28 01:21:34,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd429015d0f959922eb HTTP/1.1" 200 None 2026-03-28 01:21:34,691 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:34,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:34,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:34,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd429015d0f959922eb HTTP/1.1" 200 None 2026-03-28 01:21:34,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce229015d0f95992309 HTTP/1.1" 200 None 2026-03-28 01:21:34,781 - lava_callback - INFO - Completed processing callback for node 69c72ce229015d0f95992309 2026-03-28 01:21:34,831 - lava_callback - INFO - Completed processing callback for node 69c72cd429015d0f959922eb 2026-03-28 01:21:35,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:35,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48876 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:21:40,622 - lava_callback - INFO - Received callback for node 69c72cd729015d0f959922f1 INFO: 172.21.0.1:38752 - "POST /node/69c72cd729015d0f959922f1 HTTP/1.0" 202 Accepted 2026-03-28 01:21:40,628 - lava_callback - INFO - Starting async job submit for node 69c72cd729015d0f959922f1 2026-03-28 01:21:40,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:40,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd729015d0f959922f1 HTTP/1.1" 200 None 2026-03-28 01:21:40,666 - lava_callback - DEBUG - Processing node 69c72cd729015d0f959922f1: baseline-arm64 2026-03-28 01:21:40,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:41,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:41,040 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd729015d0f959922f1/log.txt.gz 2026-03-28 01:21:41,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:41,398 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:41,400 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd729015d0f959922f1/lava-logs.yaml 2026-03-28 01:21:41,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:41,550 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:41,551 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd729015d0f959922f1/lava_callback.json.gz 2026-03-28 01:21:41,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:41,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd729015d0f959922f1 HTTP/1.1" 200 None 2026-03-28 01:21:41,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:41,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:41,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:41,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd729015d0f959922f1 HTTP/1.1" 200 None 2026-03-28 01:21:41,639 - lava_callback - INFO - Completed processing callback for node 69c72cd729015d0f959922f1 2026-03-28 01:21:41,666 - lava_callback - INFO - Received callback for node 69c72cd029015d0f959922e1 INFO: 172.21.0.1:38754 - "POST /node/69c72cd029015d0f959922e1 HTTP/1.0" 202 Accepted 2026-03-28 01:21:41,681 - lava_callback - INFO - Starting async job submit for node 69c72cd029015d0f959922e1 2026-03-28 01:21:41,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:41,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd029015d0f959922e1 HTTP/1.1" 200 None 2026-03-28 01:21:41,748 - lava_callback - DEBUG - Processing node 69c72cd029015d0f959922e1: baseline-arm64 2026-03-28 01:21:42,127 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:42,241 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:42,243 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd029015d0f959922e1/log.txt.gz 2026-03-28 01:21:42,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:42,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:42,605 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd029015d0f959922e1/lava-logs.yaml 2026-03-28 01:21:42,612 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:42,751 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:42,753 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd029015d0f959922e1/lava_callback.json.gz 2026-03-28 01:21:42,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:42,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd029015d0f959922e1 HTTP/1.1" 200 None 2026-03-28 01:21:42,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:42,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:21:42,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:42,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd029015d0f959922e1 HTTP/1.1" 200 None 2026-03-28 01:21:42,849 - lava_callback - INFO - Completed processing callback for node 69c72cd029015d0f959922e1 INFO: 172.21.0.1:38768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60546 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:21:58,400 - lava_callback - INFO - Received callback for node 69c72cf429015d0f95992356 INFO: 172.21.0.1:60554 - "POST /node/69c72cf429015d0f95992356 HTTP/1.0" 202 Accepted 2026-03-28 01:21:58,407 - lava_callback - INFO - Starting async job submit for node 69c72cf429015d0f95992356 2026-03-28 01:21:58,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:58,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf429015d0f95992356 HTTP/1.1" 200 None 2026-03-28 01:21:58,453 - lava_callback - DEBUG - Processing node 69c72cf429015d0f95992356: baseline-nfs-arm64 2026-03-28 01:21:58,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:58,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:58,947 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf429015d0f95992356/log.txt.gz 2026-03-28 01:21:59,100 - lava_callback - INFO - Received callback for node 69c725a329015d0f9598fa00 INFO: 172.21.0.1:60570 - "POST /node/69c725a329015d0f9598fa00 HTTP/1.0" 202 Accepted 2026-03-28 01:21:59,121 - lava_callback - INFO - Starting async job submit for node 69c725a329015d0f9598fa00 2026-03-28 01:21:59,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:59,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:59,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a329015d0f9598fa00 HTTP/1.1" 200 None 2026-03-28 01:21:59,226 - lava_callback - DEBUG - Processing node 69c725a329015d0f9598fa00: ltp-cve 2026-03-28 01:21:59,460 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:59,492 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf429015d0f95992356/lava-logs.yaml 2026-03-28 01:21:59,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:59,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:21:59,693 - lava_callback - INFO - Received callback for node 69c72cde29015d0f95992300 2026-03-28 01:21:59,702 - lava_callback - INFO - Starting async job submit for node 69c72cde29015d0f95992300 INFO: 172.21.0.1:60578 - "POST /node/69c72cde29015d0f95992300 HTTP/1.0" 202 Accepted 2026-03-28 01:21:59,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:21:59,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cde29015d0f95992300 HTTP/1.1" 200 None 2026-03-28 01:21:59,770 - lava_callback - DEBUG - Processing node 69c72cde29015d0f95992300: baseline-arm64 2026-03-28 01:21:59,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:21:59,927 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69c725a329015d0f9598fa00/log.txt.gz 2026-03-28 01:22:00,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:00,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:00,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:00,349 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cde29015d0f95992300/log.txt.gz 2026-03-28 01:22:00,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:00,501 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:00,504 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69c725a329015d0f9598fa00/lava-logs.yaml 2026-03-28 01:22:00,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:00,675 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:00,678 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cde29015d0f95992300/lava-logs.yaml 2026-03-28 01:22:00,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:00,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:00,726 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69c725a329015d0f9598fa00/lava_callback.json.gz 2026-03-28 01:22:00,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:00,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a329015d0f9598fa00 HTTP/1.1" 200 None 2026-03-28 01:22:00,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:00,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:22:00,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:00,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725a329015d0f9598fa00 HTTP/1.1" 200 None 2026-03-28 01:22:00,854 - lava_callback - INFO - Completed processing callback for node 69c725a329015d0f9598fa00 2026-03-28 01:22:00,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:00,891 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cde29015d0f95992300/lava_callback.json.gz 2026-03-28 01:22:00,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:00,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cde29015d0f95992300 HTTP/1.1" 200 None 2026-03-28 01:22:00,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:00,986 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:00,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:01,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cde29015d0f95992300 HTTP/1.1" 200 None 2026-03-28 01:22:01,056 - lava_callback - INFO - Completed processing callback for node 69c72cde29015d0f95992300 2026-03-28 01:22:04,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:04,228 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf429015d0f95992356/lava_callback.json.gz 2026-03-28 01:22:04,240 - lava_callback - INFO - Received callback for node 69c72ccf29015d0f959922dc INFO: 172.21.0.1:36676 - "POST /node/69c72ccf29015d0f959922dc HTTP/1.0" 202 Accepted 2026-03-28 01:22:04,257 - lava_callback - INFO - Starting async job submit for node 69c72ccf29015d0f959922dc 2026-03-28 01:22:04,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:04,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:04,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ccf29015d0f959922dc HTTP/1.1" 200 None 2026-03-28 01:22:04,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf429015d0f95992356 HTTP/1.1" 200 None 2026-03-28 01:22:04,316 - lava_callback - DEBUG - Processing node 69c72ccf29015d0f959922dc: baseline-arm64 2026-03-28 01:22:04,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:04,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:04,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:04,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:04,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cf429015d0f95992356 HTTP/1.1" 200 None 2026-03-28 01:22:04,571 - lava_callback - INFO - Completed processing callback for node 69c72cf429015d0f95992356 2026-03-28 01:22:04,718 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:04,721 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922dc/log.txt.gz 2026-03-28 01:22:04,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:05,089 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:05,091 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922dc/lava-logs.yaml 2026-03-28 01:22:05,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:05,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:05,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ccf29015d0f959922dc/lava_callback.json.gz 2026-03-28 01:22:05,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:05,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ccf29015d0f959922dc HTTP/1.1" 200 None 2026-03-28 01:22:05,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:05,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:05,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:05,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ccf29015d0f959922dc HTTP/1.1" 200 None 2026-03-28 01:22:05,361 - lava_callback - INFO - Completed processing callback for node 69c72ccf29015d0f959922dc 2026-03-28 01:22:05,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:05,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:36682 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:12,875 - lava_callback - INFO - Received callback for node 69c72ce929015d0f95992318 INFO: 172.21.0.1:39836 - "POST /node/69c72ce929015d0f95992318 HTTP/1.0" 202 Accepted 2026-03-28 01:22:12,882 - lava_callback - INFO - Starting async job submit for node 69c72ce929015d0f95992318 2026-03-28 01:22:12,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:12,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce929015d0f95992318 HTTP/1.1" 200 None 2026-03-28 01:22:12,920 - lava_callback - DEBUG - Processing node 69c72ce929015d0f95992318: baseline-nfs-arm64 2026-03-28 01:22:13,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:13,429 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:13,431 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce929015d0f95992318/log.txt.gz 2026-03-28 01:22:13,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:13,912 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:13,915 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce929015d0f95992318/lava-logs.yaml 2026-03-28 01:22:13,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:14,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:14,113 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce929015d0f95992318/lava_callback.json.gz 2026-03-28 01:22:14,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:14,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce929015d0f95992318 HTTP/1.1" 200 None 2026-03-28 01:22:14,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:14,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:14,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:14,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce929015d0f95992318 HTTP/1.1" 200 None 2026-03-28 01:22:14,218 - lava_callback - INFO - Completed processing callback for node 69c72ce929015d0f95992318 INFO: 172.21.0.1:39840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39850 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:19,130 - lava_callback - INFO - Received callback for node 69c725a629015d0f9598fa15 INFO: 172.21.0.1:39862 - "POST /node/69c725a629015d0f9598fa15 HTTP/1.0" 202 Accepted 2026-03-28 01:22:19,136 - lava_callback - INFO - Starting async job submit for node 69c725a629015d0f9598fa15 2026-03-28 01:22:19,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:19,175 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a629015d0f9598fa15 HTTP/1.1" 200 None 2026-03-28 01:22:19,176 - lava_callback - DEBUG - Processing node 69c725a629015d0f9598fa15: ltp-fs_bind 2026-03-28 01:22:19,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:19,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:19,715 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c725a629015d0f9598fa15/log.txt.gz 2026-03-28 01:22:19,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:20,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:20,147 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c725a629015d0f9598fa15/lava-logs.yaml 2026-03-28 01:22:20,156 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:20,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:20,299 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69c725a629015d0f9598fa15/lava_callback.json.gz 2026-03-28 01:22:20,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:20,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a629015d0f9598fa15 HTTP/1.1" 200 None 2026-03-28 01:22:20,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:20,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:22:20,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:20,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725a629015d0f9598fa15 HTTP/1.1" 200 None 2026-03-28 01:22:20,395 - lava_callback - INFO - Completed processing callback for node 69c725a629015d0f9598fa15 2026-03-28 01:22:25,012 - lava_callback - INFO - Received callback for node 69c72d1929015d0f959923d9 INFO: 172.21.0.1:46326 - "POST /node/69c72d1929015d0f959923d9 HTTP/1.0" 202 Accepted 2026-03-28 01:22:25,062 - lava_callback - INFO - Starting async job submit for node 69c72d1929015d0f959923d9 2026-03-28 01:22:25,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:25,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1929015d0f959923d9 HTTP/1.1" 200 None 2026-03-28 01:22:25,099 - lava_callback - DEBUG - Processing node 69c72d1929015d0f959923d9: baseline-arm64-pengutronix 2026-03-28 01:22:25,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:25,395 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:25,397 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1929015d0f959923d9/log.txt.gz 2026-03-28 01:22:25,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:25,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:25,729 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1929015d0f959923d9/lava-logs.yaml 2026-03-28 01:22:25,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:25,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:25,869 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1929015d0f959923d9/lava_callback.json.gz 2026-03-28 01:22:25,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:25,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1929015d0f959923d9 HTTP/1.1" 200 None 2026-03-28 01:22:25,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:25,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:25,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:25,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1929015d0f959923d9 HTTP/1.1" 200 None 2026-03-28 01:22:25,955 - lava_callback - INFO - Completed processing callback for node 69c72d1929015d0f959923d9 INFO: 172.21.0.1:46336 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:26,718 - lava_callback - INFO - Received callback for node 69c72ce329015d0f9599230c INFO: 172.21.0.1:46352 - "POST /node/69c72ce329015d0f9599230c HTTP/1.0" 202 Accepted 2026-03-28 01:22:26,725 - lava_callback - INFO - Starting async job submit for node 69c72ce329015d0f9599230c 2026-03-28 01:22:26,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:26,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce329015d0f9599230c HTTP/1.1" 200 None 2026-03-28 01:22:26,767 - lava_callback - DEBUG - Processing node 69c72ce329015d0f9599230c: baseline-arm64 2026-03-28 01:22:27,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:27,388 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:27,390 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce329015d0f9599230c/log.txt.gz 2026-03-28 01:22:27,552 - lava_callback - INFO - Received callback for node 69c7258a29015d0f9598f9e1 INFO: 172.21.0.1:46368 - "POST /node/69c7258a29015d0f9598f9e1 HTTP/1.0" 202 Accepted 2026-03-28 01:22:27,593 - lava_callback - INFO - Starting async job submit for node 69c7258a29015d0f9598f9e1 2026-03-28 01:22:27,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:27,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:27,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258a29015d0f9598f9e1 HTTP/1.1" 200 None 2026-03-28 01:22:27,635 - lava_callback - DEBUG - Processing node 69c7258a29015d0f9598f9e1: baseline-arm-pengutronix 2026-03-28 01:22:27,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:27,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:27,856 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce329015d0f9599230c/lava-logs.yaml 2026-03-28 01:22:27,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:27,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:27,887 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258a29015d0f9598f9e1/log.txt.gz 2026-03-28 01:22:27,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:28,056 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:28,057 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce329015d0f9599230c/lava_callback.json.gz 2026-03-28 01:22:28,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,106 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:28,108 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258a29015d0f9598f9e1/lava-logs.yaml 2026-03-28 01:22:28,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:28,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce329015d0f9599230c HTTP/1.1" 200 None 2026-03-28 01:22:28,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,135 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:28,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce329015d0f9599230c HTTP/1.1" 200 None 2026-03-28 01:22:28,236 - lava_callback - INFO - Completed processing callback for node 69c72ce329015d0f9599230c 2026-03-28 01:22:28,311 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:28,313 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7258a29015d0f9598f9e1/lava_callback.json.gz DEBUG: Setup failed for 69c7258a29015d0f9598f9e1. Transitting job node result: incomplete -> fail 2026-03-28 01:22:28,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7258a29015d0f9598f9e1 HTTP/1.1" 200 None 2026-03-28 01:22:28,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:22:28,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:28,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7258a29015d0f9598f9e1 HTTP/1.1" 200 None 2026-03-28 01:22:28,511 - lava_callback - INFO - Completed processing callback for node 69c7258a29015d0f9598f9e1 2026-03-28 01:22:35,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:35,521 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:40206 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:36,060 - lava_callback - INFO - Received callback for node 69c72d1d29015d0f959923e2 INFO: 172.21.0.1:40208 - "POST /node/69c72d1d29015d0f959923e2 HTTP/1.0" 202 Accepted 2026-03-28 01:22:36,066 - lava_callback - INFO - Starting async job submit for node 69c72d1d29015d0f959923e2 2026-03-28 01:22:36,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:36,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1d29015d0f959923e2 HTTP/1.1" 200 None 2026-03-28 01:22:36,104 - lava_callback - DEBUG - Processing node 69c72d1d29015d0f959923e2: baseline-arm64-foundriesio 2026-03-28 01:22:36,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:36,482 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:36,483 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e2/log.txt.gz INFO: 172.21.0.1:40220 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:36,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:36,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:36,770 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e2/lava-logs.yaml 2026-03-28 01:22:36,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:36,904 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:36,905 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e2/lava_callback.json.gz 2026-03-28 01:22:36,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:36,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1d29015d0f959923e2 HTTP/1.1" 200 None 2026-03-28 01:22:36,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:36,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:36,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:37,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1d29015d0f959923e2 HTTP/1.1" 200 None 2026-03-28 01:22:37,071 - lava_callback - INFO - Completed processing callback for node 69c72d1d29015d0f959923e2 2026-03-28 01:22:43,270 - lava_callback - INFO - Received callback for node 69c72cef29015d0f9599234d INFO: 172.21.0.1:55958 - "POST /node/69c72cef29015d0f9599234d HTTP/1.0" 202 Accepted 2026-03-28 01:22:43,277 - lava_callback - INFO - Starting async job submit for node 69c72cef29015d0f9599234d 2026-03-28 01:22:43,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:43,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cef29015d0f9599234d HTTP/1.1" 200 None 2026-03-28 01:22:43,316 - lava_callback - DEBUG - Processing node 69c72cef29015d0f9599234d: baseline-nfs-arm64 2026-03-28 01:22:43,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:43,740 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:43,741 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cef29015d0f9599234d/log.txt.gz 2026-03-28 01:22:43,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:44,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:44,121 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cef29015d0f9599234d/lava-logs.yaml 2026-03-28 01:22:44,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:44,267 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:44,269 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cef29015d0f9599234d/lava_callback.json.gz 2026-03-28 01:22:44,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:44,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cef29015d0f9599234d HTTP/1.1" 200 None 2026-03-28 01:22:44,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:44,324 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:44,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:44,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cef29015d0f9599234d HTTP/1.1" 200 None 2026-03-28 01:22:44,361 - lava_callback - INFO - Completed processing callback for node 69c72cef29015d0f9599234d INFO: 172.21.0.1:55966 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:47,695 - lava_callback - INFO - Received callback for node 69c72d1a29015d0f959923dc INFO: 172.21.0.1:55968 - "POST /node/69c72d1a29015d0f959923dc HTTP/1.0" 202 Accepted 2026-03-28 01:22:47,701 - lava_callback - INFO - Starting async job submit for node 69c72d1a29015d0f959923dc 2026-03-28 01:22:47,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:47,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1a29015d0f959923dc HTTP/1.1" 200 None 2026-03-28 01:22:47,737 - lava_callback - DEBUG - Processing node 69c72d1a29015d0f959923dc: baseline-arm64-pengutronix 2026-03-28 01:22:47,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:48,082 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:48,083 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1a29015d0f959923dc/log.txt.gz 2026-03-28 01:22:48,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:48,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:48,382 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1a29015d0f959923dc/lava-logs.yaml 2026-03-28 01:22:48,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:48,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:48,526 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1a29015d0f959923dc/lava_callback.json.gz 2026-03-28 01:22:48,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:48,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1a29015d0f959923dc HTTP/1.1" 200 None 2026-03-28 01:22:48,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:48,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:48,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:48,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1a29015d0f959923dc HTTP/1.1" 200 None 2026-03-28 01:22:48,621 - lava_callback - INFO - Completed processing callback for node 69c72d1a29015d0f959923dc 2026-03-28 01:22:53,289 - lava_callback - INFO - Received callback for node 69c72ce029015d0f95992306 INFO: 172.21.0.1:39156 - "POST /node/69c72ce029015d0f95992306 HTTP/1.0" 202 Accepted 2026-03-28 01:22:53,295 - lava_callback - INFO - Starting async job submit for node 69c72ce029015d0f95992306 2026-03-28 01:22:53,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:53,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce029015d0f95992306 HTTP/1.1" 200 None 2026-03-28 01:22:53,332 - lava_callback - DEBUG - Processing node 69c72ce029015d0f95992306: baseline-arm64 2026-03-28 01:22:53,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:53,712 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:53,714 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce029015d0f95992306/log.txt.gz 2026-03-28 01:22:53,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:54,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:54,043 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce029015d0f95992306/lava-logs.yaml 2026-03-28 01:22:54,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:54,183 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:54,185 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce029015d0f95992306/lava_callback.json.gz 2026-03-28 01:22:54,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:54,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce029015d0f95992306 HTTP/1.1" 200 None 2026-03-28 01:22:54,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:54,230 - lava_callback - INFO - Received callback for node 69c72d1d29015d0f959923e5 INFO: 172.21.0.1:39166 - "POST /node/69c72d1d29015d0f959923e5 HTTP/1.0" 202 Accepted 2026-03-28 01:22:54,238 - lava_callback - INFO - Starting async job submit for node 69c72d1d29015d0f959923e5 2026-03-28 01:22:54,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:54,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:54,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:54,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1d29015d0f959923e5 HTTP/1.1" 200 None 2026-03-28 01:22:54,284 - lava_callback - DEBUG - Processing node 69c72d1d29015d0f959923e5: baseline-arm64-foundriesio 2026-03-28 01:22:54,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce029015d0f95992306 HTTP/1.1" 200 None 2026-03-28 01:22:54,328 - lava_callback - INFO - Completed processing callback for node 69c72ce029015d0f95992306 2026-03-28 01:22:54,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:54,982 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:54,984 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e5/log.txt.gz 2026-03-28 01:22:55,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:55,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:55,414 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e5/lava-logs.yaml 2026-03-28 01:22:55,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:55,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:55,572 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c72d1d29015d0f959923e5/lava_callback.json.gz 2026-03-28 01:22:55,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:55,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1d29015d0f959923e5 HTTP/1.1" 200 None 2026-03-28 01:22:55,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:55,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:55,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:55,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1d29015d0f959923e5 HTTP/1.1" 200 None 2026-03-28 01:22:55,662 - lava_callback - INFO - Completed processing callback for node 69c72d1d29015d0f959923e5 2026-03-28 01:22:56,021 - lava_callback - INFO - Received callback for node 69c72d1b29015d0f959923df INFO: 172.21.0.1:39168 - "POST /node/69c72d1b29015d0f959923df HTTP/1.0" 202 Accepted 2026-03-28 01:22:56,028 - lava_callback - INFO - Starting async job submit for node 69c72d1b29015d0f959923df 2026-03-28 01:22:56,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:56,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1b29015d0f959923df HTTP/1.1" 200 None 2026-03-28 01:22:56,067 - lava_callback - DEBUG - Processing node 69c72d1b29015d0f959923df: baseline-arm64-pengutronix 2026-03-28 01:22:56,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:56,412 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:56,413 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1b29015d0f959923df/log.txt.gz INFO: 172.21.0.1:39172 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:22:56,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:56,723 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:56,725 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1b29015d0f959923df/lava-logs.yaml 2026-03-28 01:22:56,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:22:56,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:22:56,858 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69c72d1b29015d0f959923df/lava_callback.json.gz 2026-03-28 01:22:56,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:56,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1b29015d0f959923df HTTP/1.1" 200 None 2026-03-28 01:22:56,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:56,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:22:56,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:22:56,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1b29015d0f959923df HTTP/1.1" 200 None 2026-03-28 01:22:56,937 - lava_callback - INFO - Completed processing callback for node 69c72d1b29015d0f959923df 2026-03-28 01:23:00,687 - lava_callback - INFO - Received callback for node 69c72cf529015d0f95992359 INFO: 172.21.0.1:58156 - "POST /node/69c72cf529015d0f95992359 HTTP/1.0" 202 Accepted 2026-03-28 01:23:00,693 - lava_callback - INFO - Starting async job submit for node 69c72cf529015d0f95992359 2026-03-28 01:23:00,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:00,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf529015d0f95992359 HTTP/1.1" 200 None 2026-03-28 01:23:00,734 - lava_callback - DEBUG - Processing node 69c72cf529015d0f95992359: baseline-nfs-arm64 2026-03-28 01:23:01,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:01,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:01,300 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf529015d0f95992359/log.txt.gz 2026-03-28 01:23:01,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:01,734 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:01,736 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf529015d0f95992359/lava-logs.yaml 2026-03-28 01:23:01,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:01,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:01,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf529015d0f95992359/lava_callback.json.gz 2026-03-28 01:23:01,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:01,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf529015d0f95992359 HTTP/1.1" 200 None 2026-03-28 01:23:01,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:01,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:23:01,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:01,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cf529015d0f95992359 HTTP/1.1" 200 None 2026-03-28 01:23:01,993 - lava_callback - INFO - Completed processing callback for node 69c72cf529015d0f95992359 2026-03-28 01:23:04,333 - lava_callback - INFO - Received callback for node 69c48a67590270ad7b6d974d INFO: 172.21.0.1:58160 - "POST /node/69c48a67590270ad7b6d974d HTTP/1.0" 202 Accepted 2026-03-28 01:23:04,339 - lava_callback - INFO - Starting async job submit for node 69c48a67590270ad7b6d974d 2026-03-28 01:23:04,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:04,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48a67590270ad7b6d974d HTTP/1.1" 200 None 2026-03-28 01:23:04,361 - lava_callback - DEBUG - Processing node 69c48a67590270ad7b6d974d: baseline-arm64-clabbe 2026-03-28 01:23:04,361 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48a67590270ad7b6d974d 2026-03-28 01:23:04,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:04,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48a67590270ad7b6d974d HTTP/1.1" 200 None 2026-03-28 01:23:04,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:04,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 01:23:04,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:04,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48a67590270ad7b6d974d HTTP/1.1" 200 None 2026-03-28 01:23:04,414 - lava_callback - ERROR - Error processing callback for node 69c48a67590270ad7b6d974d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:23:05,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:05,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.21.0.1:58176 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:23:14,025 - lava_callback - INFO - Received callback for node 69c72d1429015d0f959923d0 INFO: 172.21.0.1:41206 - "POST /node/69c72d1429015d0f959923d0 HTTP/1.0" 202 Accepted 2026-03-28 01:23:14,032 - lava_callback - INFO - Starting async job submit for node 69c72d1429015d0f959923d0 2026-03-28 01:23:14,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:14,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1429015d0f959923d0 HTTP/1.1" 200 None 2026-03-28 01:23:14,072 - lava_callback - DEBUG - Processing node 69c72d1429015d0f959923d0: baseline-arm64-clabbe 2026-03-28 01:23:14,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:14,427 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:14,428 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c72d1429015d0f959923d0/log.txt.gz 2026-03-28 01:23:14,563 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:14,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:14,737 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c72d1429015d0f959923d0/lava-logs.yaml 2026-03-28 01:23:14,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:14,887 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:14,889 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69c72d1429015d0f959923d0/lava_callback.json.gz 2026-03-28 01:23:14,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:14,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d1429015d0f959923d0 HTTP/1.1" 200 None 2026-03-28 01:23:14,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:14,940 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:23:14,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:14,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d1429015d0f959923d0 HTTP/1.1" 200 None 2026-03-28 01:23:14,991 - lava_callback - INFO - Completed processing callback for node 69c72d1429015d0f959923d0 INFO: 172.21.0.1:41210 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 01:23:15,319 - lava_callback - INFO - Received callback for node 69c72cf229015d0f95992353 INFO: 172.21.0.1:41216 - "POST /node/69c72cf229015d0f95992353 HTTP/1.0" 202 Accepted 2026-03-28 01:23:15,327 - lava_callback - INFO - Starting async job submit for node 69c72cf229015d0f95992353 2026-03-28 01:23:15,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:15,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf229015d0f95992353 HTTP/1.1" 200 None 2026-03-28 01:23:15,370 - lava_callback - DEBUG - Processing node 69c72cf229015d0f95992353: baseline-nfs-arm64 2026-03-28 01:23:15,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:15,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:15,908 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf229015d0f95992353/log.txt.gz 2026-03-28 01:23:16,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:16,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:16,408 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf229015d0f95992353/lava-logs.yaml 2026-03-28 01:23:16,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:41222 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:23:16,579 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:16,581 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf229015d0f95992353/lava_callback.json.gz 2026-03-28 01:23:16,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:16,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf229015d0f95992353 HTTP/1.1" 200 None 2026-03-28 01:23:16,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:16,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:23:16,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:16,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cf229015d0f95992353 HTTP/1.1" 200 None 2026-03-28 01:23:16,670 - lava_callback - INFO - Completed processing callback for node 69c72cf229015d0f95992353 INFO: 172.21.0.1:34860 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:23:35,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:35,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:50714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39614 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:23:54,027 - lava_callback - INFO - Received callback for node 69c72cd229015d0f959922e5 INFO: 172.21.0.1:53370 - "POST /node/69c72cd229015d0f959922e5 HTTP/1.0" 202 Accepted 2026-03-28 01:23:54,033 - lava_callback - INFO - Starting async job submit for node 69c72cd229015d0f959922e5 2026-03-28 01:23:54,055 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:54,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd229015d0f959922e5 HTTP/1.1" 200 None 2026-03-28 01:23:54,070 - lava_callback - DEBUG - Processing node 69c72cd229015d0f959922e5: baseline-arm64 2026-03-28 01:23:54,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:54,473 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:54,475 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd229015d0f959922e5/log.txt.gz 2026-03-28 01:23:54,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:54,802 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:54,804 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd229015d0f959922e5/lava-logs.yaml 2026-03-28 01:23:54,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:23:54,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:23:54,983 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd229015d0f959922e5/lava_callback.json.gz 2026-03-28 01:23:55,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:55,067 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd229015d0f959922e5 HTTP/1.1" 200 None 2026-03-28 01:23:55,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:55,085 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:23:55,086 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:23:55,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd229015d0f959922e5 HTTP/1.1" 200 None 2026-03-28 01:23:55,119 - lava_callback - INFO - Completed processing callback for node 69c72cd229015d0f959922e5 INFO: 172.21.0.1:53378 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:24:05,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:24:05,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53942 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41578 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:24:51,405 - lava_callback - INFO - Received callback for node 69c72ce629015d0f95992312 INFO: 172.21.0.1:51136 - "POST /node/69c72ce629015d0f95992312 HTTP/1.0" 202 Accepted 2026-03-28 01:24:51,411 - lava_callback - INFO - Starting async job submit for node 69c72ce629015d0f95992312 2026-03-28 01:24:51,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:24:51,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce629015d0f95992312 HTTP/1.1" 200 None 2026-03-28 01:24:51,450 - lava_callback - DEBUG - Processing node 69c72ce629015d0f95992312: baseline-nfs-arm64 2026-03-28 01:24:51,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:24:51,925 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:24:51,927 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce629015d0f95992312/log.txt.gz 2026-03-28 01:24:52,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:24:52,407 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:24:52,409 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce629015d0f95992312/lava-logs.yaml 2026-03-28 01:24:52,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:24:52,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:24:52,549 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce629015d0f95992312/lava_callback.json.gz 2026-03-28 01:24:52,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:24:52,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce629015d0f95992312 HTTP/1.1" 200 None 2026-03-28 01:24:52,588 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:24:52,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:24:52,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:24:52,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce629015d0f95992312 HTTP/1.1" 200 None 2026-03-28 01:24:52,645 - lava_callback - INFO - Completed processing callback for node 69c72ce629015d0f95992312 INFO: 172.21.0.1:51140 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:25:05,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:05,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:38666 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:25:09,598 - lava_callback - INFO - Received callback for node 69c72cdb29015d0f959922fa INFO: 172.21.0.1:38680 - "POST /node/69c72cdb29015d0f959922fa HTTP/1.0" 202 Accepted 2026-03-28 01:25:09,605 - lava_callback - INFO - Starting async job submit for node 69c72cdb29015d0f959922fa 2026-03-28 01:25:09,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:09,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdb29015d0f959922fa HTTP/1.1" 200 None 2026-03-28 01:25:09,642 - lava_callback - DEBUG - Processing node 69c72cdb29015d0f959922fa: baseline-arm64 2026-03-28 01:25:10,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:10,202 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:10,204 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdb29015d0f959922fa/log.txt.gz 2026-03-28 01:25:10,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:10,926 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:10,928 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdb29015d0f959922fa/lava-logs.yaml 2026-03-28 01:25:10,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:11,114 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:11,116 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdb29015d0f959922fa/lava_callback.json.gz 2026-03-28 01:25:11,162 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:11,177 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdb29015d0f959922fa HTTP/1.1" 200 None 2026-03-28 01:25:11,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:11,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:25:11,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:11,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cdb29015d0f959922fa HTTP/1.1" 200 None 2026-03-28 01:25:11,299 - lava_callback - INFO - Completed processing callback for node 69c72cdb29015d0f959922fa INFO: 172.21.0.1:52610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44790 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:25:28,562 - lava_callback - INFO - Received callback for node 69c72ced29015d0f95992347 INFO: 172.21.0.1:44796 - "POST /node/69c72ced29015d0f95992347 HTTP/1.0" 202 Accepted 2026-03-28 01:25:28,569 - lava_callback - INFO - Starting async job submit for node 69c72ced29015d0f95992347 2026-03-28 01:25:28,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:28,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ced29015d0f95992347 HTTP/1.1" 200 None 2026-03-28 01:25:28,630 - lava_callback - DEBUG - Processing node 69c72ced29015d0f95992347: baseline-nfs-arm64 2026-03-28 01:25:29,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:29,269 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:29,270 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ced29015d0f95992347/log.txt.gz 2026-03-28 01:25:29,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:29,765 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:29,767 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ced29015d0f95992347/lava-logs.yaml 2026-03-28 01:25:29,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:25:29,943 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:25:29,945 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ced29015d0f95992347/lava_callback.json.gz 2026-03-28 01:25:29,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:29,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ced29015d0f95992347 HTTP/1.1" 200 None 2026-03-28 01:25:29,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:29,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:25:29,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:30,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ced29015d0f95992347 HTTP/1.1" 200 None 2026-03-28 01:25:30,025 - lava_callback - INFO - Completed processing callback for node 69c72ced29015d0f95992347 2026-03-28 01:25:35,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:25:35,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40910 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58164 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:26:33,371 - lava_callback - INFO - Received callback for node 69c72d0f29015d0f959923bf INFO: 172.21.0.1:35982 - "POST /node/69c72d0f29015d0f959923bf HTTP/1.0" 202 Accepted 2026-03-28 01:26:33,377 - lava_callback - INFO - Starting async job submit for node 69c72d0f29015d0f959923bf 2026-03-28 01:26:33,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:33,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0f29015d0f959923bf HTTP/1.1" 200 None 2026-03-28 01:26:33,419 - lava_callback - DEBUG - Processing node 69c72d0f29015d0f959923bf: baseline-arm64-broonie 2026-03-28 01:26:33,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:33,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:33,980 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0f29015d0f959923bf/log.txt.gz 2026-03-28 01:26:34,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:34,327 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:34,329 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0f29015d0f959923bf/lava-logs.yaml 2026-03-28 01:26:34,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:34,463 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:34,465 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0f29015d0f959923bf/lava_callback.json.gz 2026-03-28 01:26:34,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:34,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0f29015d0f959923bf HTTP/1.1" 200 None 2026-03-28 01:26:34,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:34,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:26:34,519 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:34,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0f29015d0f959923bf HTTP/1.1" 200 None 2026-03-28 01:26:34,545 - lava_callback - INFO - Completed processing callback for node 69c72d0f29015d0f959923bf 2026-03-28 01:26:35,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:35,639 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36002 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:26:39,343 - lava_callback - INFO - Received callback for node 69c725a829015d0f9598fa18 INFO: 172.21.0.1:36010 - "POST /node/69c725a829015d0f9598fa18 HTTP/1.0" 202 Accepted 2026-03-28 01:26:39,349 - lava_callback - INFO - Starting async job submit for node 69c725a829015d0f9598fa18 2026-03-28 01:26:39,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:39,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a829015d0f9598fa18 HTTP/1.1" 200 None 2026-03-28 01:26:39,391 - lava_callback - DEBUG - Processing node 69c725a829015d0f9598fa18: ltp-fsx 2026-03-28 01:26:39,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:39,929 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:39,931 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69c725a829015d0f9598fa18/log.txt.gz 2026-03-28 01:26:40,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:40,376 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:40,378 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69c725a829015d0f9598fa18/lava-logs.yaml 2026-03-28 01:26:40,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:40,553 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:40,555 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69c725a829015d0f9598fa18/lava_callback.json.gz 2026-03-28 01:26:40,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:40,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725a829015d0f9598fa18 HTTP/1.1" 200 None 2026-03-28 01:26:40,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:40,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:26:40,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:40,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725a829015d0f9598fa18 HTTP/1.1" 200 None 2026-03-28 01:26:40,653 - lava_callback - INFO - Completed processing callback for node 69c725a829015d0f9598fa18 2026-03-28 01:26:41,020 - lava_callback - INFO - Received callback for node 69c725aa29015d0f9598fa1b INFO: 172.21.0.1:48218 - "POST /node/69c725aa29015d0f9598fa1b HTTP/1.0" 202 Accepted 2026-03-28 01:26:41,027 - lava_callback - INFO - Starting async job submit for node 69c725aa29015d0f9598fa1b 2026-03-28 01:26:41,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:41,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725aa29015d0f9598fa1b HTTP/1.1" 200 None 2026-03-28 01:26:41,074 - lava_callback - DEBUG - Processing node 69c725aa29015d0f9598fa1b: ltp-input 2026-03-28 01:26:41,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:41,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:41,567 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69c725aa29015d0f9598fa1b/log.txt.gz 2026-03-28 01:26:41,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:41,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:41,976 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69c725aa29015d0f9598fa1b/lava-logs.yaml 2026-03-28 01:26:41,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:26:42,144 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:26:42,145 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69c725aa29015d0f9598fa1b/lava_callback.json.gz 2026-03-28 01:26:42,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:42,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725aa29015d0f9598fa1b HTTP/1.1" 200 None 2026-03-28 01:26:42,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:42,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:26:42,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:26:42,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725aa29015d0f9598fa1b HTTP/1.1" 200 None 2026-03-28 01:26:42,238 - lava_callback - INFO - Completed processing callback for node 69c725aa29015d0f9598fa1b INFO: 172.21.0.1:48222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35876 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:27:05,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:05,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38146 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:27:16,864 - lava_callback - INFO - Received callback for node 69c72cda29015d0f959922f7 INFO: 172.21.0.1:38152 - "POST /node/69c72cda29015d0f959922f7 HTTP/1.0" 202 Accepted 2026-03-28 01:27:16,871 - lava_callback - INFO - Starting async job submit for node 69c72cda29015d0f959922f7 2026-03-28 01:27:16,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:16,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cda29015d0f959922f7 HTTP/1.1" 200 None 2026-03-28 01:27:16,917 - lava_callback - DEBUG - Processing node 69c72cda29015d0f959922f7: baseline-arm64 2026-03-28 01:27:17,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:17,477 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:17,479 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cda29015d0f959922f7/log.txt.gz 2026-03-28 01:27:17,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:17,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:17,957 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cda29015d0f959922f7/lava-logs.yaml 2026-03-28 01:27:17,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:18,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:18,123 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cda29015d0f959922f7/lava_callback.json.gz 2026-03-28 01:27:18,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:18,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cda29015d0f959922f7 HTTP/1.1" 200 None 2026-03-28 01:27:18,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:18,183 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:27:18,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:18,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cda29015d0f959922f7 HTTP/1.1" 200 None 2026-03-28 01:27:18,214 - lava_callback - INFO - Completed processing callback for node 69c72cda29015d0f959922f7 2026-03-28 01:27:20,141 - lava_callback - INFO - Received callback for node 69c725ac29015d0f9598fa21 INFO: 172.21.0.1:45444 - "POST /node/69c725ac29015d0f9598fa21 HTTP/1.0" 202 Accepted 2026-03-28 01:27:20,151 - lava_callback - INFO - Starting async job submit for node 69c725ac29015d0f9598fa21 2026-03-28 01:27:20,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:20,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725ac29015d0f9598fa21 HTTP/1.1" 200 None 2026-03-28 01:27:20,196 - lava_callback - DEBUG - Processing node 69c725ac29015d0f9598fa21: ltp-ipc 2026-03-28 01:27:20,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:20,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:20,744 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69c725ac29015d0f9598fa21/log.txt.gz 2026-03-28 01:27:20,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:21,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:21,239 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69c725ac29015d0f9598fa21/lava-logs.yaml 2026-03-28 01:27:21,248 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:27:21,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:27:21,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69c725ac29015d0f9598fa21/lava_callback.json.gz 2026-03-28 01:27:21,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:21,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725ac29015d0f9598fa21 HTTP/1.1" 200 None 2026-03-28 01:27:21,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:21,468 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:27:21,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:21,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725ac29015d0f9598fa21 HTTP/1.1" 200 None 2026-03-28 01:27:21,495 - lava_callback - INFO - Completed processing callback for node 69c725ac29015d0f9598fa21 INFO: 172.21.0.1:45448 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:27:35,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:27:35,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39458 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:28:10,679 - lava_callback - INFO - Received callback for node 69c725ae29015d0f9598fa24 INFO: 172.21.0.1:52048 - "POST /node/69c725ae29015d0f9598fa24 HTTP/1.0" 202 Accepted 2026-03-28 01:28:10,686 - lava_callback - INFO - Starting async job submit for node 69c725ae29015d0f9598fa24 2026-03-28 01:28:10,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:10,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725ae29015d0f9598fa24 HTTP/1.1" 200 None 2026-03-28 01:28:10,743 - lava_callback - DEBUG - Processing node 69c725ae29015d0f9598fa24: ltp-pty 2026-03-28 01:28:10,819 - lava_callback - INFO - Received callback for node 69c72ceb29015d0f95992344 INFO: 172.21.0.1:52054 - "POST /node/69c72ceb29015d0f95992344 HTTP/1.0" 202 Accepted 2026-03-28 01:28:10,836 - lava_callback - INFO - Starting async job submit for node 69c72ceb29015d0f95992344 2026-03-28 01:28:10,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:10,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ceb29015d0f95992344 HTTP/1.1" 200 None 2026-03-28 01:28:11,009 - lava_callback - DEBUG - Processing node 69c72ceb29015d0f95992344: baseline-nfs-arm64 2026-03-28 01:28:11,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:11,609 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:11,625 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69c725ae29015d0f9598fa24/log.txt.gz 2026-03-28 01:28:11,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:11,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:11,884 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:11,886 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ceb29015d0f95992344/log.txt.gz 2026-03-28 01:28:12,119 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:12,152 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69c725ae29015d0f9598fa24/lava-logs.yaml 2026-03-28 01:28:12,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:12,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:12,423 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:12,425 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ceb29015d0f95992344/lava-logs.yaml 2026-03-28 01:28:12,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:12,436 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69c725ae29015d0f9598fa24/lava_callback.json.gz 2026-03-28 01:28:12,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,463 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:12,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725ae29015d0f9598fa24 HTTP/1.1" 200 None 2026-03-28 01:28:12,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:28:12,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725ae29015d0f9598fa24 HTTP/1.1" 200 None 2026-03-28 01:28:12,549 - lava_callback - INFO - Completed processing callback for node 69c725ae29015d0f9598fa24 2026-03-28 01:28:12,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:12,685 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ceb29015d0f95992344/lava_callback.json.gz 2026-03-28 01:28:12,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ceb29015d0f95992344 HTTP/1.1" 200 None 2026-03-28 01:28:12,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:28:12,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:12,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ceb29015d0f95992344 HTTP/1.1" 200 None 2026-03-28 01:28:12,787 - lava_callback - INFO - Completed processing callback for node 69c72ceb29015d0f95992344 INFO: 172.21.0.1:52058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43022 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:28:34,284 - lava_callback - INFO - Received callback for node 69c72ce429015d0f9599230f INFO: 172.21.0.1:39358 - "POST /node/69c72ce429015d0f9599230f HTTP/1.0" 202 Accepted 2026-03-28 01:28:34,291 - lava_callback - INFO - Starting async job submit for node 69c72ce429015d0f9599230f 2026-03-28 01:28:34,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:34,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce429015d0f9599230f HTTP/1.1" 200 None 2026-03-28 01:28:34,349 - lava_callback - DEBUG - Processing node 69c72ce429015d0f9599230f: baseline-arm64 2026-03-28 01:28:34,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:34,845 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:34,847 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce429015d0f9599230f/log.txt.gz 2026-03-28 01:28:35,058 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:35,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:35,279 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce429015d0f9599230f/lava-logs.yaml 2026-03-28 01:28:35,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:28:35,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:28:35,433 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72ce429015d0f9599230f/lava_callback.json.gz DEBUG: Setup failed for 69c72ce429015d0f9599230f. Transitting job node result: incomplete -> fail 2026-03-28 01:28:35,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:35,522 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce429015d0f9599230f HTTP/1.1" 200 None 2026-03-28 01:28:35,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:35,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:28:35,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:35,563 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce429015d0f9599230f HTTP/1.1" 200 None 2026-03-28 01:28:35,647 - lava_callback - INFO - Completed processing callback for node 69c72ce429015d0f9599230f 2026-03-28 01:28:35,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:28:35,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45374 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:30:59,456 - lava_callback - INFO - Received callback for node 69c725af29015d0f9598fa27 INFO: 172.21.0.1:45376 - "POST /node/69c725af29015d0f9598fa27 HTTP/1.0" 202 Accepted 2026-03-28 01:30:59,463 - lava_callback - INFO - Starting async job submit for node 69c725af29015d0f9598fa27 2026-03-28 01:30:59,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:30:59,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725af29015d0f9598fa27 HTTP/1.1" 200 None 2026-03-28 01:30:59,503 - lava_callback - DEBUG - Processing node 69c725af29015d0f9598fa27: ltp-sched 2026-03-28 01:30:59,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:00,009 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:00,011 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69c725af29015d0f9598fa27/log.txt.gz 2026-03-28 01:31:00,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:00,481 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:00,483 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69c725af29015d0f9598fa27/lava-logs.yaml 2026-03-28 01:31:00,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:00,824 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:00,825 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69c725af29015d0f9598fa27/lava_callback.json.gz 2026-03-28 01:31:00,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:00,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725af29015d0f9598fa27 HTTP/1.1" 200 None 2026-03-28 01:31:00,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:00,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:31:00,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:00,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725af29015d0f9598fa27 HTTP/1.1" 200 None 2026-03-28 01:31:00,928 - lava_callback - INFO - Completed processing callback for node 69c725af29015d0f9598fa27 2026-03-28 01:31:05,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:05,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37834 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45114 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:31:39,691 - lava_callback - INFO - Received callback for node 69c725b129015d0f9598fa2a INFO: 172.21.0.1:45128 - "POST /node/69c725b129015d0f9598fa2a HTTP/1.0" 202 Accepted 2026-03-28 01:31:39,698 - lava_callback - INFO - Starting async job submit for node 69c725b129015d0f9598fa2a 2026-03-28 01:31:39,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:39,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b129015d0f9598fa2a HTTP/1.1" 200 None 2026-03-28 01:31:39,743 - lava_callback - DEBUG - Processing node 69c725b129015d0f9598fa2a: ltp-smoke 2026-03-28 01:31:40,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:40,917 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:40,920 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c725b129015d0f9598fa2a/log.txt.gz 2026-03-28 01:31:41,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:41,277 - lava_callback - INFO - Received callback for node 69c725b529015d0f9598fa30 INFO: 172.21.0.1:33056 - "POST /node/69c725b529015d0f9598fa30 HTTP/1.0" 202 Accepted 2026-03-28 01:31:41,288 - lava_callback - INFO - Starting async job submit for node 69c725b529015d0f9598fa30 2026-03-28 01:31:41,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:41,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b529015d0f9598fa30 HTTP/1.1" 200 None 2026-03-28 01:31:41,363 - lava_callback - DEBUG - Processing node 69c725b529015d0f9598fa30: ltp-syscalls-ipc 2026-03-28 01:31:41,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:41,786 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c725b129015d0f9598fa2a/lava-logs.yaml 2026-03-28 01:31:41,877 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:41,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:42,046 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:42,048 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c725b129015d0f9598fa2a/lava_callback.json.gz 2026-03-28 01:31:42,081 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:42,083 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c725b529015d0f9598fa30/log.txt.gz 2026-03-28 01:31:42,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:42,161 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b129015d0f9598fa2a HTTP/1.1" 200 None 2026-03-28 01:31:42,163 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:42,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:31:42,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:42,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725b129015d0f9598fa2a HTTP/1.1" 200 None 2026-03-28 01:31:42,335 - lava_callback - INFO - Completed processing callback for node 69c725b129015d0f9598fa2a 2026-03-28 01:31:42,633 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:42,844 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:42,846 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c725b529015d0f9598fa30/lava-logs.yaml 2026-03-28 01:31:42,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:31:43,329 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:31:43,331 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69c725b529015d0f9598fa30/lava_callback.json.gz 2026-03-28 01:31:43,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:43,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b529015d0f9598fa30 HTTP/1.1" 200 None 2026-03-28 01:31:43,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:43,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:31:43,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:31:43,417 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725b529015d0f9598fa30 HTTP/1.1" 200 None 2026-03-28 01:31:43,423 - lava_callback - INFO - Completed processing callback for node 69c725b529015d0f9598fa30 INFO: 172.21.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50680 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:32:05,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:05,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57336 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:32:10,567 - lava_callback - INFO - Received callback for node 69c725b629015d0f9598fa33 INFO: 172.21.0.1:38510 - "POST /node/69c725b629015d0f9598fa33 HTTP/1.0" 202 Accepted 2026-03-28 01:32:10,577 - lava_callback - INFO - Starting async job submit for node 69c725b629015d0f9598fa33 2026-03-28 01:32:10,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:10,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b629015d0f9598fa33 HTTP/1.1" 200 None 2026-03-28 01:32:10,663 - lava_callback - DEBUG - Processing node 69c725b629015d0f9598fa33: ltp-timers 2026-03-28 01:32:11,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:32:11,608 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:32:11,611 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69c725b629015d0f9598fa33/log.txt.gz 2026-03-28 01:32:11,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:32:12,130 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:32:12,133 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69c725b629015d0f9598fa33/lava-logs.yaml 2026-03-28 01:32:12,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:32:12,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:32:12,469 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69c725b629015d0f9598fa33/lava_callback.json.gz 2026-03-28 01:32:12,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:12,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b629015d0f9598fa33 HTTP/1.1" 200 None 2026-03-28 01:32:12,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:12,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:32:12,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:12,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725b629015d0f9598fa33 HTTP/1.1" 200 None 2026-03-28 01:32:12,571 - lava_callback - INFO - Completed processing callback for node 69c725b629015d0f9598fa33 INFO: 172.21.0.1:38514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50118 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:32:35,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:32:35,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37372 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:33:28,515 - lava_callback - INFO - Received callback for node 69c48cd3590270ad7b6d97ef INFO: 172.21.0.1:37380 - "POST /node/69c48cd3590270ad7b6d97ef HTTP/1.0" 202 Accepted 2026-03-28 01:33:28,520 - lava_callback - INFO - Starting async job submit for node 69c48cd3590270ad7b6d97ef 2026-03-28 01:33:28,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:28,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48cd3590270ad7b6d97ef HTTP/1.1" 200 None 2026-03-28 01:33:28,540 - lava_callback - DEBUG - Processing node 69c48cd3590270ad7b6d97ef: baseline-arm64-baylibre 2026-03-28 01:33:28,540 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48cd3590270ad7b6d97ef 2026-03-28 01:33:28,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:28,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48cd3590270ad7b6d97ef HTTP/1.1" 200 None 2026-03-28 01:33:28,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:28,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:33:28,579 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:28,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48cd3590270ad7b6d97ef HTTP/1.1" 200 None 2026-03-28 01:33:28,600 - lava_callback - ERROR - Error processing callback for node 69c48cd3590270ad7b6d97ef: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 01:33:32,018 - lava_callback - INFO - Received callback for node 69c72d0829015d0f95992392 INFO: 172.21.0.1:43456 - "POST /node/69c72d0829015d0f95992392 HTTP/1.0" 202 Accepted 2026-03-28 01:33:32,025 - lava_callback - INFO - Starting async job submit for node 69c72d0829015d0f95992392 2026-03-28 01:33:32,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:32,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0829015d0f95992392 HTTP/1.1" 200 None 2026-03-28 01:33:32,062 - lava_callback - DEBUG - Processing node 69c72d0829015d0f95992392: baseline-arm64-broonie 2026-03-28 01:33:32,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:33:32,459 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:33:32,460 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0829015d0f95992392/log.txt.gz 2026-03-28 01:33:32,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:33:32,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:33:32,791 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0829015d0f95992392/lava-logs.yaml 2026-03-28 01:33:32,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:33:32,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:33:32,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0829015d0f95992392/lava_callback.json.gz 2026-03-28 01:33:33,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:33,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0829015d0f95992392 HTTP/1.1" 200 None 2026-03-28 01:33:33,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:33,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:33:33,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:33,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0829015d0f95992392 HTTP/1.1" 200 None 2026-03-28 01:33:33,086 - lava_callback - INFO - Completed processing callback for node 69c72d0829015d0f95992392 2026-03-28 01:33:35,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:33:35,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47110 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:35:08,621 - lava_callback - INFO - Received callback for node 69c48d49590270ad7b6d980e INFO: 172.21.0.1:47112 - "POST /node/69c48d49590270ad7b6d980e HTTP/1.0" 202 Accepted 2026-03-28 01:35:08,629 - lava_callback - INFO - Starting async job submit for node 69c48d49590270ad7b6d980e 2026-03-28 01:35:08,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:08,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48d49590270ad7b6d980e HTTP/1.1" 200 None 2026-03-28 01:35:08,644 - lava_callback - DEBUG - Processing node 69c48d49590270ad7b6d980e: baseline-arm64-baylibre 2026-03-28 01:35:08,645 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48d49590270ad7b6d980e 2026-03-28 01:35:08,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:08,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48d49590270ad7b6d980e HTTP/1.1" 200 None 2026-03-28 01:35:08,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:08,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:35:08,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:08,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48d49590270ad7b6d980e HTTP/1.1" 200 None 2026-03-28 01:35:08,708 - lava_callback - ERROR - Error processing callback for node 69c48d49590270ad7b6d980e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43698 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:35:19,780 - lava_callback - INFO - Received callback for node 69c725b829015d0f9598fa36 INFO: 172.21.0.1:53886 - "POST /node/69c725b829015d0f9598fa36 HTTP/1.0" 202 Accepted 2026-03-28 01:35:19,786 - lava_callback - INFO - Starting async job submit for node 69c725b829015d0f9598fa36 2026-03-28 01:35:19,812 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:19,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b829015d0f9598fa36 HTTP/1.1" 200 None 2026-03-28 01:35:19,832 - lava_callback - DEBUG - Processing node 69c725b829015d0f9598fa36: ltp-watchqueue 2026-03-28 01:35:20,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:35:20,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:35:20,372 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c725b829015d0f9598fa36/log.txt.gz 2026-03-28 01:35:20,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:35:20,795 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:35:20,797 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c725b829015d0f9598fa36/lava-logs.yaml 2026-03-28 01:35:20,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:35:20,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:35:20,982 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69c725b829015d0f9598fa36/lava_callback.json.gz 2026-03-28 01:35:21,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:21,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c725b829015d0f9598fa36 HTTP/1.1" 200 None 2026-03-28 01:35:21,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:21,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:35:21,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:21,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c725b829015d0f9598fa36 HTTP/1.1" 200 None 2026-03-28 01:35:21,076 - lava_callback - INFO - Completed processing callback for node 69c725b829015d0f9598fa36 INFO: 172.21.0.1:53890 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:35:35,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:35:35,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56732 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:36:33,457 - lava_callback - INFO - Received callback for node 69c72c8029015d0f959922a6 INFO: 172.21.0.1:43654 - "POST /node/69c72c8029015d0f959922a6 HTTP/1.0" 202 Accepted 2026-03-28 01:36:33,464 - lava_callback - INFO - Starting async job submit for node 69c72c8029015d0f959922a6 2026-03-28 01:36:33,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:36:33,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72c8029015d0f959922a6 HTTP/1.1" 200 None 2026-03-28 01:36:33,560 - lava_callback - DEBUG - Processing node 69c72c8029015d0f959922a6: ltp-smoke 2026-03-28 01:36:34,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:36:34,255 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:36:34,257 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c72c8029015d0f959922a6/log.txt.gz 2026-03-28 01:36:34,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:36:34,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:36:34,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c72c8029015d0f959922a6/lava-logs.yaml 2026-03-28 01:36:34,792 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:36:34,985 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:36:34,987 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c72c8029015d0f959922a6/lava_callback.json.gz 2026-03-28 01:36:35,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:36:35,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72c8029015d0f959922a6 HTTP/1.1" 200 None 2026-03-28 01:36:35,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:36:35,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4229015d0f9598f742 HTTP/1.1" 200 None 2026-03-28 01:36:35,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:36:35,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72c8029015d0f959922a6 HTTP/1.1" 200 None 2026-03-28 01:36:35,390 - lava_callback - INFO - Completed processing callback for node 69c72c8029015d0f959922a6 2026-03-28 01:36:35,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:36:35,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58446 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:37:39,515 - lava_callback - INFO - Received callback for node 69c72cd029015d0f959922de INFO: 172.21.0.1:58456 - "POST /node/69c72cd029015d0f959922de HTTP/1.0" 202 Accepted 2026-03-28 01:37:39,522 - lava_callback - INFO - Starting async job submit for node 69c72cd029015d0f959922de 2026-03-28 01:37:39,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:39,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd029015d0f959922de HTTP/1.1" 200 None 2026-03-28 01:37:39,592 - lava_callback - DEBUG - Processing node 69c72cd029015d0f959922de: baseline-arm 2026-03-28 01:37:40,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:37:40,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:37:40,382 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c72cd029015d0f959922de/log.txt.gz 2026-03-28 01:37:40,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:37:41,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:37:41,002 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c72cd029015d0f959922de/lava-logs.yaml 2026-03-28 01:37:41,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:37:41,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:37:41,192 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c72cd029015d0f959922de/lava_callback.json.gz DEBUG: Setup failed for 69c72cd029015d0f959922de. Transitting job node result: incomplete -> fail 2026-03-28 01:37:41,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:41,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd029015d0f959922de HTTP/1.1" 200 None 2026-03-28 01:37:41,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:41,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:37:41,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:41,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd029015d0f959922de HTTP/1.1" 200 None 2026-03-28 01:37:41,468 - lava_callback - INFO - Completed processing callback for node 69c72cd029015d0f959922de INFO: 172.21.0.1:42972 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:37:48,739 - lava_callback - INFO - Received callback for node 69c48de9590270ad7b6d9824 INFO: 172.21.0.1:42978 - "POST /node/69c48de9590270ad7b6d9824 HTTP/1.0" 202 Accepted 2026-03-28 01:37:48,744 - lava_callback - INFO - Starting async job submit for node 69c48de9590270ad7b6d9824 2026-03-28 01:37:48,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:48,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48de9590270ad7b6d9824 HTTP/1.1" 200 None 2026-03-28 01:37:48,779 - lava_callback - DEBUG - Processing node 69c48de9590270ad7b6d9824: baseline-arm64-baylibre 2026-03-28 01:37:48,779 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48de9590270ad7b6d9824 2026-03-28 01:37:48,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:48,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48de9590270ad7b6d9824 HTTP/1.1" 200 None 2026-03-28 01:37:48,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:48,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:37:48,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:37:48,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48de9590270ad7b6d9824 HTTP/1.1" 200 None 2026-03-28 01:37:48,837 - lava_callback - ERROR - Error processing callback for node 69c48de9590270ad7b6d9824: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53818 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:38:05,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:38:05,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43586 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:39:43,342 - lava_callback - INFO - Received callback for node 69c72cd829015d0f959922f4 INFO: 172.21.0.1:58184 - "POST /node/69c72cd829015d0f959922f4 HTTP/1.0" 202 Accepted 2026-03-28 01:39:43,349 - lava_callback - INFO - Starting async job submit for node 69c72cd829015d0f959922f4 2026-03-28 01:39:43,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:39:43,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd829015d0f959922f4 HTTP/1.1" 200 None 2026-03-28 01:39:43,386 - lava_callback - DEBUG - Processing node 69c72cd829015d0f959922f4: baseline-arm64 2026-03-28 01:39:43,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:39:43,919 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:39:43,921 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd829015d0f959922f4/log.txt.gz 2026-03-28 01:39:44,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:39:44,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:39:44,368 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd829015d0f959922f4/lava-logs.yaml 2026-03-28 01:39:44,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:39:44,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:39:44,514 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd829015d0f959922f4/lava_callback.json.gz 2026-03-28 01:39:44,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:39:44,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd829015d0f959922f4 HTTP/1.1" 200 None 2026-03-28 01:39:44,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:39:44,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:39:44,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:39:44,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd829015d0f959922f4 HTTP/1.1" 200 None 2026-03-28 01:39:44,605 - lava_callback - INFO - Completed processing callback for node 69c72cd829015d0f959922f4 INFO: 172.21.0.1:58198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39106 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:40:05,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:05,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47006 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:40:19,639 - lava_callback - INFO - Received callback for node 69c72d0729015d0f9599238f INFO: 172.21.0.1:47010 - "POST /node/69c72d0729015d0f9599238f HTTP/1.0" 202 Accepted 2026-03-28 01:40:19,645 - lava_callback - INFO - Starting async job submit for node 69c72d0729015d0f9599238f 2026-03-28 01:40:19,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:19,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0729015d0f9599238f HTTP/1.1" 200 None 2026-03-28 01:40:19,683 - lava_callback - DEBUG - Processing node 69c72d0729015d0f9599238f: baseline-arm64-broonie 2026-03-28 01:40:19,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:40:20,074 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:40:20,076 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0729015d0f9599238f/log.txt.gz 2026-03-28 01:40:20,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:40:20,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:40:20,500 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0729015d0f9599238f/lava-logs.yaml 2026-03-28 01:40:20,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:40:20,652 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:40:20,654 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0729015d0f9599238f/lava_callback.json.gz 2026-03-28 01:40:20,677 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:20,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0729015d0f9599238f HTTP/1.1" 200 None 2026-03-28 01:40:20,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:20,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:40:20,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:20,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0729015d0f9599238f HTTP/1.1" 200 None 2026-03-28 01:40:20,747 - lava_callback - INFO - Completed processing callback for node 69c72d0729015d0f9599238f INFO: 172.21.0.1:35116 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:40:35,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:35,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59242 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:40:45,203 - lava_callback - INFO - Received callback for node 69c48e95590270ad7b6d9851 INFO: 172.21.0.1:43708 - "POST /node/69c48e95590270ad7b6d9851 HTTP/1.0" 202 Accepted 2026-03-28 01:40:45,208 - lava_callback - INFO - Starting async job submit for node 69c48e95590270ad7b6d9851 2026-03-28 01:40:45,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:45,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48e95590270ad7b6d9851 HTTP/1.1" 200 None 2026-03-28 01:40:45,225 - lava_callback - DEBUG - Processing node 69c48e95590270ad7b6d9851: baseline-arm64-clabbe 2026-03-28 01:40:45,226 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48e95590270ad7b6d9851 2026-03-28 01:40:45,226 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:45,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48e95590270ad7b6d9851 HTTP/1.1" 200 None 2026-03-28 01:40:45,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:45,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:40:45,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:40:45,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48e95590270ad7b6d9851 HTTP/1.1" 200 None 2026-03-28 01:40:45,312 - lava_callback - ERROR - Error processing callback for node 69c48e95590270ad7b6d9851: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60330 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:45226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45518 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39806 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:44:05,358 - lava_callback - INFO - Received callback for node 69c48f5f590270ad7b6d9876 INFO: 172.21.0.1:34278 - "POST /node/69c48f5f590270ad7b6d9876 HTTP/1.0" 202 Accepted 2026-03-28 01:44:05,364 - lava_callback - INFO - Starting async job submit for node 69c48f5f590270ad7b6d9876 2026-03-28 01:44:05,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:44:05,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48f5f590270ad7b6d9876 HTTP/1.1" 200 None 2026-03-28 01:44:05,392 - lava_callback - DEBUG - Processing node 69c48f5f590270ad7b6d9876: baseline-arm64-clabbe 2026-03-28 01:44:05,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c48f5f590270ad7b6d9876 2026-03-28 01:44:05,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:44:05,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c48f5f590270ad7b6d9876 HTTP/1.1" 200 None 2026-03-28 01:44:05,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:44:05,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:44:05,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:44:05,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c48f5f590270ad7b6d9876 HTTP/1.1" 200 None 2026-03-28 01:44:05,452 - lava_callback - ERROR - Error processing callback for node 69c48f5f590270ad7b6d9876: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57438 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:45:19,854 - lava_callback - INFO - Received callback for node 69c72d0c29015d0f959923b8 INFO: 172.21.0.1:37038 - "POST /node/69c72d0c29015d0f959923b8 HTTP/1.0" 202 Accepted 2026-03-28 01:45:19,860 - lava_callback - INFO - Starting async job submit for node 69c72d0c29015d0f959923b8 2026-03-28 01:45:19,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:45:19,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0c29015d0f959923b8 HTTP/1.1" 200 None 2026-03-28 01:45:19,907 - lava_callback - DEBUG - Processing node 69c72d0c29015d0f959923b8: baseline-arm64-broonie 2026-03-28 01:45:20,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:45:20,339 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:45:20,341 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0c29015d0f959923b8/log.txt.gz 2026-03-28 01:45:20,503 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:45:20,701 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:45:20,703 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0c29015d0f959923b8/lava-logs.yaml 2026-03-28 01:45:20,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:45:20,853 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:45:20,855 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0c29015d0f959923b8/lava_callback.json.gz 2026-03-28 01:45:20,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:45:20,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0c29015d0f959923b8 HTTP/1.1" 200 None 2026-03-28 01:45:20,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:45:20,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:45:20,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:45:20,949 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0c29015d0f959923b8 HTTP/1.1" 200 None 2026-03-28 01:45:20,961 - lava_callback - INFO - Completed processing callback for node 69c72d0c29015d0f959923b8 INFO: 172.21.0.1:37040 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:45:35,946 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:45:35,966 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56698 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49596 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:47:05,524 - lava_callback - INFO - Received callback for node 69c49003590270ad7b6d989f INFO: 172.21.0.1:53588 - "POST /node/69c49003590270ad7b6d989f HTTP/1.0" 202 Accepted 2026-03-28 01:47:05,530 - lava_callback - INFO - Starting async job submit for node 69c49003590270ad7b6d989f 2026-03-28 01:47:05,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:47:05,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49003590270ad7b6d989f HTTP/1.1" 200 None 2026-03-28 01:47:05,546 - lava_callback - DEBUG - Processing node 69c49003590270ad7b6d989f: baseline-arm64-clabbe 2026-03-28 01:47:05,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49003590270ad7b6d989f 2026-03-28 01:47:05,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:47:05,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49003590270ad7b6d989f HTTP/1.1" 200 None 2026-03-28 01:47:05,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:47:05,624 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:47:05,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:47:05,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49003590270ad7b6d989f HTTP/1.1" 200 None 2026-03-28 01:47:05,647 - lava_callback - ERROR - Error processing callback for node 69c49003590270ad7b6d989f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50288 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60612 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:48:39,810 - lava_callback - INFO - Received callback for node 69c7337229015d0f9599264c INFO: 172.21.0.1:35900 - "POST /node/69c7337229015d0f9599264c HTTP/1.0" 202 Accepted 2026-03-28 01:48:39,817 - lava_callback - INFO - Starting async job submit for node 69c7337229015d0f9599264c 2026-03-28 01:48:39,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:48:39,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7337229015d0f9599264c HTTP/1.1" 200 None 2026-03-28 01:48:39,839 - lava_callback - DEBUG - Processing node 69c7337229015d0f9599264c: baseline-arm 2026-03-28 01:48:39,844 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:48:39,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:48:40,005 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:48:40,007 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7337229015d0f9599264c/lava-logs.yaml 2026-03-28 01:48:40,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:48:40,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:48:40,300 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7337229015d0f9599264c/lava_callback.json.gz 2026-03-28 01:48:40,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:48:40,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7337229015d0f9599264c HTTP/1.1" 200 None 2026-03-28 01:48:40,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:48:40,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 01:48:40,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:48:40,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7337229015d0f9599264c HTTP/1.1" 200 None 2026-03-28 01:48:40,362 - lava_callback - INFO - Completed processing callback for node 69c7337229015d0f9599264c INFO: 172.21.0.1:35916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45178 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:49:05,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:49:05,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33068 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:49:45,639 - lava_callback - INFO - Received callback for node 69c490a5590270ad7b6d98c5 INFO: 172.21.0.1:36802 - "POST /node/69c490a5590270ad7b6d98c5 HTTP/1.0" 202 Accepted 2026-03-28 01:49:45,645 - lava_callback - INFO - Starting async job submit for node 69c490a5590270ad7b6d98c5 2026-03-28 01:49:45,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:49:45,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c490a5590270ad7b6d98c5 HTTP/1.1" 200 None 2026-03-28 01:49:45,663 - lava_callback - DEBUG - Processing node 69c490a5590270ad7b6d98c5: baseline-arm64-clabbe 2026-03-28 01:49:45,663 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c490a5590270ad7b6d98c5 2026-03-28 01:49:45,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:49:45,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c490a5590270ad7b6d98c5 HTTP/1.1" 200 None 2026-03-28 01:49:45,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:49:45,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:49:45,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:49:45,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c490a5590270ad7b6d98c5 HTTP/1.1" 200 None 2026-03-28 01:49:45,717 - lava_callback - ERROR - Error processing callback for node 69c490a5590270ad7b6d98c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57662 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:50:06,783 - lava_callback - INFO - Received callback for node 69c733c129015d0f9599266d INFO: 172.21.0.1:57672 - "POST /node/69c733c129015d0f9599266d HTTP/1.0" 202 Accepted 2026-03-28 01:50:06,789 - lava_callback - INFO - Starting async job submit for node 69c733c129015d0f9599266d 2026-03-28 01:50:06,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:50:06,813 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c733c129015d0f9599266d HTTP/1.1" 200 None 2026-03-28 01:50:06,814 - lava_callback - DEBUG - Processing node 69c733c129015d0f9599266d: baseline-nfs-arm64 2026-03-28 01:50:06,815 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:50:06,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:50:06,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:50:06,974 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c733c129015d0f9599266d/lava-logs.yaml 2026-03-28 01:50:06,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:50:07,108 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:50:07,109 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c733c129015d0f9599266d/lava_callback.json.gz 2026-03-28 01:50:07,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:50:07,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c733c129015d0f9599266d HTTP/1.1" 200 None 2026-03-28 01:50:07,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:50:07,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:50:07,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:50:07,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c733c129015d0f9599266d HTTP/1.1" 200 None 2026-03-28 01:50:07,176 - lava_callback - INFO - Completed processing callback for node 69c733c129015d0f9599266d INFO: 172.21.0.1:56814 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38076 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:50:35,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:50:36,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:48972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35924 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:51:05,656 - lava_callback - INFO - Received callback for node 69c490f7590270ad7b6d98d5 INFO: 172.21.0.1:53810 - "POST /node/69c490f7590270ad7b6d98d5 HTTP/1.0" 202 Accepted 2026-03-28 01:51:05,662 - lava_callback - INFO - Starting async job submit for node 69c490f7590270ad7b6d98d5 2026-03-28 01:51:05,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:51:05,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c490f7590270ad7b6d98d5 HTTP/1.1" 200 None 2026-03-28 01:51:05,686 - lava_callback - DEBUG - Processing node 69c490f7590270ad7b6d98d5: baseline-arm64-clabbe 2026-03-28 01:51:05,686 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c490f7590270ad7b6d98d5 2026-03-28 01:51:05,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:51:05,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c490f7590270ad7b6d98d5 HTTP/1.1" 200 None 2026-03-28 01:51:05,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:51:05,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:51:05,719 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:51:05,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c490f7590270ad7b6d98d5 HTTP/1.1" 200 None 2026-03-28 01:51:05,740 - lava_callback - ERROR - Error processing callback for node 69c490f7590270ad7b6d98d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52594 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42996 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:52:25,701 - lava_callback - INFO - Received callback for node 69c49149590270ad7b6d98e6 INFO: 172.21.0.1:51928 - "POST /node/69c49149590270ad7b6d98e6 HTTP/1.0" 202 Accepted 2026-03-28 01:52:25,706 - lava_callback - INFO - Starting async job submit for node 69c49149590270ad7b6d98e6 2026-03-28 01:52:25,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:52:25,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49149590270ad7b6d98e6 HTTP/1.1" 200 None 2026-03-28 01:52:25,729 - lava_callback - DEBUG - Processing node 69c49149590270ad7b6d98e6: baseline-arm64-clabbe 2026-03-28 01:52:25,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49149590270ad7b6d98e6 2026-03-28 01:52:25,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:52:25,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49149590270ad7b6d98e6 HTTP/1.1" 200 None 2026-03-28 01:52:25,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:52:25,758 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 01:52:25,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:52:25,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49149590270ad7b6d98e6 HTTP/1.1" 200 None 2026-03-28 01:52:25,781 - lava_callback - ERROR - Error processing callback for node 69c49149590270ad7b6d98e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51716 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51826 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:54:33,462 - lava_callback - INFO - Received callback for node 69c734b029015d0f95992693 INFO: 172.21.0.1:55918 - "POST /node/69c734b029015d0f95992693 HTTP/1.0" 202 Accepted 2026-03-28 01:54:33,473 - lava_callback - INFO - Starting async job submit for node 69c734b029015d0f95992693 2026-03-28 01:54:33,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:54:33,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c734b029015d0f95992693 HTTP/1.1" 200 None 2026-03-28 01:54:33,521 - lava_callback - DEBUG - Processing node 69c734b029015d0f95992693: baseline-arm64 2026-03-28 01:54:33,658 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:54:33,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:54:34,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:54:34,134 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c734b029015d0f95992693/lava-logs.yaml 2026-03-28 01:54:34,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:54:34,447 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:54:34,449 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c734b029015d0f95992693/lava_callback.json.gz 2026-03-28 01:54:34,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:54:34,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c734b029015d0f95992693 HTTP/1.1" 200 None 2026-03-28 01:54:34,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:54:34,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:54:34,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:54:34,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c734b029015d0f95992693 HTTP/1.1" 200 None 2026-03-28 01:54:34,609 - lava_callback - INFO - Completed processing callback for node 69c734b029015d0f95992693 2026-03-28 01:54:36,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:54:36,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:55924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54432 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52318 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60518 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:57:23,019 - lava_callback - INFO - Received callback for node 69c7350429015d0f959926a0 INFO: 172.21.0.1:58182 - "POST /node/69c7350429015d0f959926a0 HTTP/1.0" 202 Accepted 2026-03-28 01:57:23,026 - lava_callback - INFO - Starting async job submit for node 69c7350429015d0f959926a0 2026-03-28 01:57:23,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:57:23,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7350429015d0f959926a0 HTTP/1.1" 200 None 2026-03-28 01:57:23,072 - lava_callback - DEBUG - Processing node 69c7350429015d0f959926a0: baseline-arm64 2026-03-28 01:57:23,281 - lava_callback - WARNING - Failed to upload log 2026-03-28 01:57:23,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:57:23,629 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:57:23,631 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7350429015d0f959926a0/lava-logs.yaml 2026-03-28 01:57:23,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:57:23,964 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:57:23,965 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7350429015d0f959926a0/lava_callback.json.gz 2026-03-28 01:57:24,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:57:24,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7350429015d0f959926a0 HTTP/1.1" 200 None 2026-03-28 01:57:24,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:57:24,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:57:24,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:57:24,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7350429015d0f959926a0 HTTP/1.1" 200 None 2026-03-28 01:57:24,153 - lava_callback - INFO - Completed processing callback for node 69c7350429015d0f959926a0 INFO: 172.21.0.1:58184 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:57:36,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:57:36,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34930 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:58:35,395 - lava_callback - INFO - Received callback for node 69c7355a29015d0f959926aa INFO: 172.21.0.1:44504 - "POST /node/69c7355a29015d0f959926aa HTTP/1.0" 202 Accepted 2026-03-28 01:58:35,401 - lava_callback - INFO - Starting async job submit for node 69c7355a29015d0f959926aa 2026-03-28 01:58:35,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:58:35,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7355a29015d0f959926aa HTTP/1.1" 200 None 2026-03-28 01:58:35,438 - lava_callback - DEBUG - Processing node 69c7355a29015d0f959926aa: baseline-arm64 2026-03-28 01:58:35,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:58:36,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:58:36,094 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7355a29015d0f959926aa/log.txt.gz 2026-03-28 01:58:36,295 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:44514 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:58:36,513 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 INFO: 172.21.0.1:44528 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:58:36,516 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7355a29015d0f959926aa/lava-logs.yaml 2026-03-28 01:58:36,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:58:36,699 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:58:36,701 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7355a29015d0f959926aa/lava_callback.json.gz 2026-03-28 01:58:36,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:58:36,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7355a29015d0f959926aa HTTP/1.1" 200 None 2026-03-28 01:58:36,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:58:36,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:58:36,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:58:36,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7355a29015d0f959926aa HTTP/1.1" 200 None 2026-03-28 01:58:36,862 - lava_callback - INFO - Completed processing callback for node 69c7355a29015d0f959926aa INFO: 172.21.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43572 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:59:06,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:06,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33306 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:59:46,278 - lava_callback - INFO - Received callback for node 69c72cea29015d0f9599231b INFO: 172.21.0.1:52460 - "POST /node/69c72cea29015d0f9599231b HTTP/1.0" 202 Accepted 2026-03-28 01:59:46,287 - lava_callback - INFO - Starting async job submit for node 69c72cea29015d0f9599231b 2026-03-28 01:59:46,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:46,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cea29015d0f9599231b HTTP/1.1" 200 None 2026-03-28 01:59:46,346 - lava_callback - DEBUG - Processing node 69c72cea29015d0f9599231b: baseline-nfs-arm64 INFO: 172.21.0.1:52470 - "GET / HTTP/1.0" 200 OK 2026-03-28 01:59:47,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:47,680 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:47,682 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cea29015d0f9599231b/log.txt.gz 2026-03-28 01:59:48,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:48,565 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:48,567 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cea29015d0f9599231b/lava-logs.yaml 2026-03-28 01:59:48,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:48,931 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:48,933 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cea29015d0f9599231b/lava_callback.json.gz DEBUG: Setup failed for 69c72cea29015d0f9599231b. Transitting job node result: incomplete -> fail 2026-03-28 01:59:49,026 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:49,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cea29015d0f9599231b HTTP/1.1" 200 None 2026-03-28 01:59:49,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:49,054 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:59:49,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:49,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cea29015d0f9599231b HTTP/1.1" 200 None 2026-03-28 01:59:49,175 - lava_callback - INFO - Completed processing callback for node 69c72cea29015d0f9599231b 2026-03-28 01:59:51,034 - lava_callback - INFO - Received callback for node 69c735e629015d0f959926cd INFO: 172.21.0.1:35502 - "POST /node/69c735e629015d0f959926cd HTTP/1.0" 202 Accepted 2026-03-28 01:59:51,040 - lava_callback - INFO - Starting async job submit for node 69c735e629015d0f959926cd 2026-03-28 01:59:51,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:51,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c735e629015d0f959926cd HTTP/1.1" 200 None 2026-03-28 01:59:51,079 - lava_callback - DEBUG - Processing node 69c735e629015d0f959926cd: baseline-arm64-foundriesio 2026-03-28 01:59:51,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:51,569 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:51,570 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c735e629015d0f959926cd/log.txt.gz 2026-03-28 01:59:51,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:51,867 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:51,869 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c735e629015d0f959926cd/lava-logs.yaml 2026-03-28 01:59:51,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 01:59:52,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 01:59:52,017 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69c735e629015d0f959926cd/lava_callback.json.gz 2026-03-28 01:59:52,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:52,088 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c735e629015d0f959926cd HTTP/1.1" 200 None 2026-03-28 01:59:52,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:52,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 01:59:52,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 01:59:52,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c735e629015d0f959926cd HTTP/1.1" 200 None 2026-03-28 01:59:52,134 - lava_callback - INFO - Completed processing callback for node 69c735e629015d0f959926cd INFO: 172.21.0.1:35514 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:00:06,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:00:06,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56378 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54380 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38904 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32826 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56498 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:03:47,129 - lava_callback - INFO - Received callback for node 69c736b929015d0f9599270b INFO: 172.21.0.1:56502 - "POST /node/69c736b929015d0f9599270b HTTP/1.0" 202 Accepted 2026-03-28 02:03:47,136 - lava_callback - INFO - Starting async job submit for node 69c736b929015d0f9599270b 2026-03-28 02:03:47,158 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:03:47,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c736b929015d0f9599270b HTTP/1.1" 200 None 2026-03-28 02:03:47,174 - lava_callback - DEBUG - Processing node 69c736b929015d0f9599270b: baseline-arm64 2026-03-28 02:03:47,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:03:47,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:03:47,795 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c736b929015d0f9599270b/log.txt.gz 2026-03-28 02:03:48,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:03:48,419 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:03:48,421 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c736b929015d0f9599270b/lava-logs.yaml 2026-03-28 02:03:48,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:03:48,603 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:03:48,605 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c736b929015d0f9599270b/lava_callback.json.gz 2026-03-28 02:03:48,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:03:48,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c736b929015d0f9599270b HTTP/1.1" 200 None 2026-03-28 02:03:48,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:03:48,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:03:48,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:03:48,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c736b929015d0f9599270b HTTP/1.1" 200 None 2026-03-28 02:03:48,776 - lava_callback - INFO - Completed processing callback for node 69c736b929015d0f9599270b INFO: 172.21.0.1:60338 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:04:06,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:04:06,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:46760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36384 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34258 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36126 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36882 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34866 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35824 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:12:33,714 - lava_callback - INFO - Received callback for node 69c7375629015d0f9599273a INFO: 172.21.0.1:45990 - "POST /node/69c7375629015d0f9599273a HTTP/1.0" 202 Accepted 2026-03-28 02:12:33,722 - lava_callback - INFO - Starting async job submit for node 69c7375629015d0f9599273a 2026-03-28 02:12:33,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:12:33,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7375629015d0f9599273a HTTP/1.1" 200 None 2026-03-28 02:12:33,774 - lava_callback - DEBUG - Processing node 69c7375629015d0f9599273a: baseline-arm64 2026-03-28 02:12:34,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:12:34,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:12:34,368 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7375629015d0f9599273a/log.txt.gz 2026-03-28 02:12:34,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:12:34,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:12:34,836 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7375629015d0f9599273a/lava-logs.yaml 2026-03-28 02:12:34,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:12:35,022 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:12:35,024 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c7375629015d0f9599273a/lava_callback.json.gz DEBUG: Setup failed for 69c7375629015d0f9599273a. Transitting job node result: incomplete -> fail 2026-03-28 02:12:35,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:12:35,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7375629015d0f9599273a HTTP/1.1" 200 None 2026-03-28 02:12:35,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:12:35,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:12:35,129 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:12:35,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7375629015d0f9599273a HTTP/1.1" 200 None 2026-03-28 02:12:35,235 - lava_callback - INFO - Completed processing callback for node 69c7375629015d0f9599273a 2026-03-28 02:12:36,116 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:12:36,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41544 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59886 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59628 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:18:02,468 - lava_callback - INFO - Received callback for node 69c73a4229015d0f959927b3 INFO: 172.21.0.1:56108 - "POST /node/69c73a4229015d0f959927b3 HTTP/1.0" 202 Accepted 2026-03-28 02:18:02,474 - lava_callback - INFO - Starting async job submit for node 69c73a4229015d0f959927b3 2026-03-28 02:18:02,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:18:02,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73a4229015d0f959927b3 HTTP/1.1" 200 None 2026-03-28 02:18:02,500 - lava_callback - DEBUG - Processing node 69c73a4229015d0f959927b3: baseline-nfs-arm64 2026-03-28 02:18:02,501 - lava_callback - WARNING - Failed to upload log 2026-03-28 02:18:02,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:18:02,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:18:02,803 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73a4229015d0f959927b3/lava-logs.yaml 2026-03-28 02:18:02,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:18:02,944 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:18:02,946 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73a4229015d0f959927b3/lava_callback.json.gz 2026-03-28 02:18:02,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:18:02,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73a4229015d0f959927b3 HTTP/1.1" 200 None 2026-03-28 02:18:02,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:18:02,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:18:02,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:18:03,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73a4229015d0f959927b3 HTTP/1.1" 200 None 2026-03-28 02:18:03,017 - lava_callback - INFO - Completed processing callback for node 69c73a4229015d0f959927b3 2026-03-28 02:18:06,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:18:06,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:56122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36224 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41862 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:23:54,625 - lava_callback - INFO - Received callback for node 69c73b8c29015d0f959927e3 INFO: 172.21.0.1:34752 - "POST /node/69c73b8c29015d0f959927e3 HTTP/1.0" 202 Accepted 2026-03-28 02:23:54,632 - lava_callback - INFO - Starting async job submit for node 69c73b8c29015d0f959927e3 2026-03-28 02:23:54,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:23:54,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73b8c29015d0f959927e3 HTTP/1.1" 200 None 2026-03-28 02:23:54,677 - lava_callback - DEBUG - Processing node 69c73b8c29015d0f959927e3: baseline-arm64 2026-03-28 02:23:54,815 - lava_callback - WARNING - Failed to upload log 2026-03-28 02:23:54,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:23:55,158 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:23:55,160 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73b8c29015d0f959927e3/lava-logs.yaml 2026-03-28 02:23:55,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:23:55,337 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:23:55,339 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73b8c29015d0f959927e3/lava_callback.json.gz 2026-03-28 02:23:55,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:23:55,386 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73b8c29015d0f959927e3 HTTP/1.1" 200 None 2026-03-28 02:23:55,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:23:55,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:23:55,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:23:55,423 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73b8c29015d0f959927e3 HTTP/1.1" 200 None 2026-03-28 02:23:55,489 - lava_callback - INFO - Completed processing callback for node 69c73b8c29015d0f959927e3 INFO: 172.21.0.1:34756 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:24:06,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:24:06,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37054 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:25:31,449 - lava_callback - INFO - Received callback for node 69c49906590270ad7b6d99db INFO: 172.21.0.1:40610 - "POST /node/69c49906590270ad7b6d99db HTTP/1.0" 202 Accepted 2026-03-28 02:25:31,455 - lava_callback - INFO - Starting async job submit for node 69c49906590270ad7b6d99db 2026-03-28 02:25:31,456 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:25:31,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49906590270ad7b6d99db HTTP/1.1" 200 None 2026-03-28 02:25:31,470 - lava_callback - DEBUG - Processing node 69c49906590270ad7b6d99db: baseline-arm64-baylibre 2026-03-28 02:25:31,471 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49906590270ad7b6d99db 2026-03-28 02:25:31,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:25:31,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49906590270ad7b6d99db HTTP/1.1" 200 None 2026-03-28 02:25:31,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:25:31,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:25:31,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:25:31,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49906590270ad7b6d99db HTTP/1.1" 200 None 2026-03-28 02:25:31,526 - lava_callback - ERROR - Error processing callback for node 69c49906590270ad7b6d99db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34802 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:26:07,336 - lava_callback - INFO - Received callback for node 69c73bff29015d0f959927f2 INFO: 172.21.0.1:34812 - "POST /node/69c73bff29015d0f959927f2 HTTP/1.0" 202 Accepted 2026-03-28 02:26:07,343 - lava_callback - INFO - Starting async job submit for node 69c73bff29015d0f959927f2 2026-03-28 02:26:07,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:07,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73bff29015d0f959927f2 HTTP/1.1" 200 None 2026-03-28 02:26:07,389 - lava_callback - DEBUG - Processing node 69c73bff29015d0f959927f2: baseline-arm64 2026-03-28 02:26:07,551 - lava_callback - WARNING - Failed to upload log 2026-03-28 02:26:07,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:26:07,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:26:07,811 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73bff29015d0f959927f2/lava-logs.yaml 2026-03-28 02:26:07,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:26:07,972 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:26:07,973 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73bff29015d0f959927f2/lava_callback.json.gz 2026-03-28 02:26:08,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:08,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73bff29015d0f959927f2 HTTP/1.1" 200 None 2026-03-28 02:26:08,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:08,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:26:08,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:08,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73bff29015d0f959927f2 HTTP/1.1" 200 None 2026-03-28 02:26:08,131 - lava_callback - INFO - Completed processing callback for node 69c73bff29015d0f959927f2 INFO: 172.21.0.1:44516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44522 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:26:21,400 - lava_callback - INFO - Received callback for node 69c7386e29015d0f95992769 INFO: 172.21.0.1:54332 - "POST /node/69c7386e29015d0f95992769 HTTP/1.0" 202 Accepted 2026-03-28 02:26:21,407 - lava_callback - INFO - Starting async job submit for node 69c7386e29015d0f95992769 2026-03-28 02:26:21,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:21,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7386e29015d0f95992769 HTTP/1.1" 200 None 2026-03-28 02:26:21,487 - lava_callback - DEBUG - Processing node 69c7386e29015d0f95992769: baseline-arm 2026-03-28 02:26:22,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:26:22,220 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:26:22,222 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c7386e29015d0f95992769/log.txt.gz 2026-03-28 02:26:22,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:26:22,836 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:26:22,838 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c7386e29015d0f95992769/lava-logs.yaml 2026-03-28 02:26:22,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:26:23,000 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:26:23,002 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c7386e29015d0f95992769/lava_callback.json.gz DEBUG: Setup failed for 69c7386e29015d0f95992769. Transitting job node result: incomplete -> fail 2026-03-28 02:26:23,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:23,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7386e29015d0f95992769 HTTP/1.1" 200 None 2026-03-28 02:26:23,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:23,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 02:26:23,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:23,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7386e29015d0f95992769 HTTP/1.1" 200 None 2026-03-28 02:26:23,271 - lava_callback - INFO - Completed processing callback for node 69c7386e29015d0f95992769 INFO: 172.21.0.1:54336 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:26:36,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:36,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:36688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:26:51,349 - lava_callback - INFO - Received callback for node 69c49969590270ad7b6d99e5 INFO: 172.21.0.1:37452 - "POST /node/69c49969590270ad7b6d99e5 HTTP/1.0" 202 Accepted 2026-03-28 02:26:51,355 - lava_callback - INFO - Starting async job submit for node 69c49969590270ad7b6d99e5 2026-03-28 02:26:51,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:51,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49969590270ad7b6d99e5 HTTP/1.1" 200 None 2026-03-28 02:26:51,371 - lava_callback - DEBUG - Processing node 69c49969590270ad7b6d99e5: baseline-arm64-baylibre 2026-03-28 02:26:51,371 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49969590270ad7b6d99e5 2026-03-28 02:26:51,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:51,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49969590270ad7b6d99e5 HTTP/1.1" 200 None 2026-03-28 02:26:51,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:51,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:26:51,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:26:51,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49969590270ad7b6d99e5 HTTP/1.1" 200 None 2026-03-28 02:26:51,427 - lava_callback - ERROR - Error processing callback for node 69c49969590270ad7b6d99e5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37464 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40174 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49148 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:27:32,939 - lava_callback - INFO - Received callback for node 69c7386c29015d0f95992766 INFO: 172.21.0.1:39088 - "POST /node/69c7386c29015d0f95992766 HTTP/1.0" 202 Accepted 2026-03-28 02:27:32,946 - lava_callback - INFO - Starting async job submit for node 69c7386c29015d0f95992766 2026-03-28 02:27:33,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:33,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7386c29015d0f95992766 HTTP/1.1" 200 None 2026-03-28 02:27:33,027 - lava_callback - DEBUG - Processing node 69c7386c29015d0f95992766: ltp-smoke 2026-03-28 02:27:33,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:33,832 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:33,833 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7386c29015d0f95992766/log.txt.gz 2026-03-28 02:27:34,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:34,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:34,324 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7386c29015d0f95992766/lava-logs.yaml 2026-03-28 02:27:34,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:34,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:34,505 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c7386c29015d0f95992766/lava_callback.json.gz 2026-03-28 02:27:34,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:34,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7386c29015d0f95992766 HTTP/1.1" 200 None 2026-03-28 02:27:34,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:34,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4229015d0f9598f742 HTTP/1.1" 200 None 2026-03-28 02:27:34,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:34,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7386c29015d0f95992766 HTTP/1.1" 200 None 2026-03-28 02:27:34,814 - lava_callback - INFO - Completed processing callback for node 69c7386c29015d0f95992766 2026-03-28 02:27:36,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:36,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:39100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39102 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:27:37,319 - lava_callback - INFO - Received callback for node 69c73c5c29015d0f95992804 INFO: 172.21.0.1:39104 - "POST /node/69c73c5c29015d0f95992804 HTTP/1.0" 202 Accepted 2026-03-28 02:27:37,326 - lava_callback - INFO - Starting async job submit for node 69c73c5c29015d0f95992804 2026-03-28 02:27:37,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:37,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73c5c29015d0f95992804 HTTP/1.1" 200 None 2026-03-28 02:27:37,363 - lava_callback - DEBUG - Processing node 69c73c5c29015d0f95992804: baseline-arm64 2026-03-28 02:27:37,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:37,769 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:37,771 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73c5c29015d0f95992804/log.txt.gz 2026-03-28 02:27:37,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:38,146 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:38,148 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73c5c29015d0f95992804/lava-logs.yaml 2026-03-28 02:27:38,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:27:38,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:27:38,463 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73c5c29015d0f95992804/lava_callback.json.gz 2026-03-28 02:27:38,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:38,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73c5c29015d0f95992804 HTTP/1.1" 200 None 2026-03-28 02:27:38,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:38,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:27:38,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:27:38,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73c5c29015d0f95992804 HTTP/1.1" 200 None 2026-03-28 02:27:38,545 - lava_callback - INFO - Completed processing callback for node 69c73c5c29015d0f95992804 INFO: 172.21.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56138 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:28:06,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:28:06,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:37182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45854 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:30:26,600 - lava_callback - INFO - Received callback for node 69c73cef29015d0f9599282b INFO: 172.21.0.1:45862 - "POST /node/69c73cef29015d0f9599282b HTTP/1.0" 202 Accepted 2026-03-28 02:30:26,606 - lava_callback - INFO - Starting async job submit for node 69c73cef29015d0f9599282b 2026-03-28 02:30:26,629 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:26,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73cef29015d0f9599282b HTTP/1.1" 200 None 2026-03-28 02:30:26,643 - lava_callback - DEBUG - Processing node 69c73cef29015d0f9599282b: baseline-arm64 2026-03-28 02:30:27,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:30:27,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:30:27,287 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73cef29015d0f9599282b/log.txt.gz 2026-03-28 02:30:27,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:30:27,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:30:27,789 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73cef29015d0f9599282b/lava-logs.yaml 2026-03-28 02:30:27,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:30:27,966 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:30:27,968 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73cef29015d0f9599282b/lava_callback.json.gz 2026-03-28 02:30:28,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:28,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73cef29015d0f9599282b HTTP/1.1" 200 None 2026-03-28 02:30:28,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:28,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:30:28,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:28,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73cef29015d0f9599282b HTTP/1.1" 200 None 2026-03-28 02:30:28,185 - lava_callback - INFO - Completed processing callback for node 69c73cef29015d0f9599282b 2026-03-28 02:30:31,553 - lava_callback - INFO - Received callback for node 69c49a34590270ad7b6d99f3 INFO: 172.21.0.1:43506 - "POST /node/69c49a34590270ad7b6d99f3 HTTP/1.0" 202 Accepted 2026-03-28 02:30:31,558 - lava_callback - INFO - Starting async job submit for node 69c49a34590270ad7b6d99f3 2026-03-28 02:30:31,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:31,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49a34590270ad7b6d99f3 HTTP/1.1" 200 None 2026-03-28 02:30:31,584 - lava_callback - DEBUG - Processing node 69c49a34590270ad7b6d99f3: baseline-arm64-baylibre 2026-03-28 02:30:31,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49a34590270ad7b6d99f3 2026-03-28 02:30:31,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:31,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49a34590270ad7b6d99f3 HTTP/1.1" 200 None 2026-03-28 02:30:31,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:31,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:30:31,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:31,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49a34590270ad7b6d99f3 HTTP/1.1" 200 None 2026-03-28 02:30:31,636 - lava_callback - ERROR - Error processing callback for node 69c49a34590270ad7b6d99f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 02:30:36,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:30:36,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39660 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:32:02,583 - lava_callback - INFO - Received callback for node 69c73d9729015d0f95992848 INFO: 172.21.0.1:49828 - "POST /node/69c73d9729015d0f95992848 HTTP/1.0" 202 Accepted 2026-03-28 02:32:02,589 - lava_callback - INFO - Starting async job submit for node 69c73d9729015d0f95992848 2026-03-28 02:32:02,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:02,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73d9729015d0f95992848 HTTP/1.1" 200 None 2026-03-28 02:32:02,606 - lava_callback - DEBUG - Processing node 69c73d9729015d0f95992848: baseline-nfs-arm64 2026-03-28 02:32:02,608 - lava_callback - WARNING - Failed to upload log 2026-03-28 02:32:02,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:32:02,760 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:32:02,762 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73d9729015d0f95992848/lava-logs.yaml 2026-03-28 02:32:02,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:32:02,920 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:32:02,922 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73d9729015d0f95992848/lava_callback.json.gz 2026-03-28 02:32:02,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:02,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73d9729015d0f95992848 HTTP/1.1" 200 None 2026-03-28 02:32:02,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:02,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:32:02,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:02,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73d9729015d0f95992848 HTTP/1.1" 200 None 2026-03-28 02:32:02,985 - lava_callback - INFO - Completed processing callback for node 69c73d9729015d0f95992848 2026-03-28 02:32:06,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:06,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49842 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:32:11,662 - lava_callback - INFO - Received callback for node 69c49a99590270ad7b6d99fe INFO: 172.21.0.1:41204 - "POST /node/69c49a99590270ad7b6d99fe HTTP/1.0" 202 Accepted 2026-03-28 02:32:11,668 - lava_callback - INFO - Starting async job submit for node 69c49a99590270ad7b6d99fe 2026-03-28 02:32:11,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:11,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49a99590270ad7b6d99fe HTTP/1.1" 200 None 2026-03-28 02:32:11,691 - lava_callback - DEBUG - Processing node 69c49a99590270ad7b6d99fe: baseline-arm64-baylibre 2026-03-28 02:32:11,691 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49a99590270ad7b6d99fe 2026-03-28 02:32:11,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:11,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49a99590270ad7b6d99fe HTTP/1.1" 200 None 2026-03-28 02:32:11,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:11,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:32:11,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:32:11,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49a99590270ad7b6d99fe HTTP/1.1" 200 None 2026-03-28 02:32:11,752 - lava_callback - ERROR - Error processing callback for node 69c49a99590270ad7b6d99fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41772 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:33:47,569 - lava_callback - INFO - Received callback for node 69c49af6590270ad7b6d9a0a INFO: 172.21.0.1:41774 - "POST /node/69c49af6590270ad7b6d9a0a HTTP/1.0" 202 Accepted 2026-03-28 02:33:47,575 - lava_callback - INFO - Starting async job submit for node 69c49af6590270ad7b6d9a0a 2026-03-28 02:33:47,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:33:47,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49af6590270ad7b6d9a0a HTTP/1.1" 200 None 2026-03-28 02:33:47,591 - lava_callback - DEBUG - Processing node 69c49af6590270ad7b6d9a0a: baseline-arm64-clabbe 2026-03-28 02:33:47,592 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49af6590270ad7b6d9a0a 2026-03-28 02:33:47,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:33:47,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49af6590270ad7b6d9a0a HTTP/1.1" 200 None 2026-03-28 02:33:47,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:33:47,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:33:47,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:33:47,651 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49af6590270ad7b6d9a0a HTTP/1.1" 200 None 2026-03-28 02:33:47,652 - lava_callback - ERROR - Error processing callback for node 69c49af6590270ad7b6d9a0a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52800 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:34:11,717 - lava_callback - INFO - Received callback for node 69c735b129015d0f959926b7 INFO: 172.21.0.1:52806 - "POST /node/69c735b129015d0f959926b7 HTTP/1.0" 202 Accepted 2026-03-28 02:34:11,765 - lava_callback - INFO - Starting async job submit for node 69c735b129015d0f959926b7 2026-03-28 02:34:11,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:34:11,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c735b129015d0f959926b7 HTTP/1.1" 200 None 2026-03-28 02:34:11,812 - lava_callback - DEBUG - Processing node 69c735b129015d0f959926b7: baseline-arm-pengutronix 2026-03-28 02:34:11,922 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:34:12,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:34:12,093 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c735b129015d0f959926b7/log.txt.gz 2026-03-28 02:34:12,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:34:12,483 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:34:12,485 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c735b129015d0f959926b7/lava-logs.yaml 2026-03-28 02:34:12,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:34:12,625 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:34:12,627 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c735b129015d0f959926b7/lava_callback.json.gz DEBUG: Setup failed for 69c735b129015d0f959926b7. Transitting job node result: incomplete -> fail 2026-03-28 02:34:12,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:34:12,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c735b129015d0f959926b7 HTTP/1.1" 200 None 2026-03-28 02:34:12,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:34:12,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 02:34:12,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:34:12,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c735b129015d0f959926b7 HTTP/1.1" 200 None 2026-03-28 02:34:12,815 - lava_callback - INFO - Completed processing callback for node 69c735b129015d0f959926b7 INFO: 172.21.0.1:52812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43276 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:34:36,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:34:36,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40328 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:35:30,814 - lava_callback - INFO - Received callback for node 69c73e3329015d0f95992863 INFO: 172.21.0.1:42910 - "POST /node/69c73e3329015d0f95992863 HTTP/1.0" 202 Accepted 2026-03-28 02:35:30,821 - lava_callback - INFO - Starting async job submit for node 69c73e3329015d0f95992863 2026-03-28 02:35:30,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:35:30,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73e3329015d0f95992863 HTTP/1.1" 200 None 2026-03-28 02:35:30,885 - lava_callback - DEBUG - Processing node 69c73e3329015d0f95992863: baseline-arm64 2026-03-28 02:35:31,030 - lava_callback - WARNING - Failed to upload log 2026-03-28 02:35:31,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:35:31,309 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:35:31,311 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73e3329015d0f95992863/lava-logs.yaml 2026-03-28 02:35:31,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:35:31,454 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:35:31,456 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73e3329015d0f95992863/lava_callback.json.gz 2026-03-28 02:35:31,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:35:31,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73e3329015d0f95992863 HTTP/1.1" 200 None 2026-03-28 02:35:31,502 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:35:31,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:35:31,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:35:31,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73e3329015d0f95992863 HTTP/1.1" 200 None 2026-03-28 02:35:31,616 - lava_callback - INFO - Completed processing callback for node 69c73e3329015d0f95992863 2026-03-28 02:35:36,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:35:36,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54992 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:36:07,635 - lava_callback - INFO - Received callback for node 69c49b94590270ad7b6d9a31 INFO: 172.21.0.1:55008 - "POST /node/69c49b94590270ad7b6d9a31 HTTP/1.0" 202 Accepted 2026-03-28 02:36:07,641 - lava_callback - INFO - Starting async job submit for node 69c49b94590270ad7b6d9a31 2026-03-28 02:36:07,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:07,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49b94590270ad7b6d9a31 HTTP/1.1" 200 None 2026-03-28 02:36:07,657 - lava_callback - DEBUG - Processing node 69c49b94590270ad7b6d9a31: baseline-arm64-clabbe 2026-03-28 02:36:07,657 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49b94590270ad7b6d9a31 2026-03-28 02:36:07,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:07,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49b94590270ad7b6d9a31 HTTP/1.1" 200 None 2026-03-28 02:36:07,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:07,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:36:07,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:07,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49b94590270ad7b6d9a31 HTTP/1.1" 200 None 2026-03-28 02:36:07,719 - lava_callback - ERROR - Error processing callback for node 69c49b94590270ad7b6d9a31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43590 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:36:53,429 - lava_callback - INFO - Received callback for node 69c73e8629015d0f95992871 INFO: 172.21.0.1:52124 - "POST /node/69c73e8629015d0f95992871 HTTP/1.0" 202 Accepted 2026-03-28 02:36:53,436 - lava_callback - INFO - Starting async job submit for node 69c73e8629015d0f95992871 2026-03-28 02:36:53,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:53,480 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73e8629015d0f95992871 HTTP/1.1" 200 None 2026-03-28 02:36:53,481 - lava_callback - DEBUG - Processing node 69c73e8629015d0f95992871: baseline-arm64 2026-03-28 02:36:53,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:36:53,822 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:36:53,824 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73e8629015d0f95992871/log.txt.gz 2026-03-28 02:36:53,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:36:54,087 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:36:54,089 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73e8629015d0f95992871/lava-logs.yaml 2026-03-28 02:36:54,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:36:54,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:36:54,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73e8629015d0f95992871/lava_callback.json.gz 2026-03-28 02:36:54,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:54,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73e8629015d0f95992871 HTTP/1.1" 200 None 2026-03-28 02:36:54,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:54,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:36:54,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:36:54,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73e8629015d0f95992871 HTTP/1.1" 200 None 2026-03-28 02:36:54,392 - lava_callback - INFO - Completed processing callback for node 69c73e8629015d0f95992871 INFO: 172.21.0.1:52134 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:37:06,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:37:06,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:58128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58458 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:37:47,705 - lava_callback - INFO - Received callback for node 69c49be5590270ad7b6d9a3e INFO: 172.21.0.1:58474 - "POST /node/69c49be5590270ad7b6d9a3e HTTP/1.0" 202 Accepted 2026-03-28 02:37:47,711 - lava_callback - INFO - Starting async job submit for node 69c49be5590270ad7b6d9a3e 2026-03-28 02:37:47,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:37:47,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49be5590270ad7b6d9a3e HTTP/1.1" 200 None 2026-03-28 02:37:47,727 - lava_callback - DEBUG - Processing node 69c49be5590270ad7b6d9a3e: baseline-arm64-clabbe 2026-03-28 02:37:47,727 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49be5590270ad7b6d9a3e 2026-03-28 02:37:47,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:37:47,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49be5590270ad7b6d9a3e HTTP/1.1" 200 None 2026-03-28 02:37:47,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:37:47,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:37:47,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:37:47,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49be5590270ad7b6d9a3e HTTP/1.1" 200 None 2026-03-28 02:37:47,793 - lava_callback - ERROR - Error processing callback for node 69c49be5590270ad7b6d9a3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49522 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:38:02,236 - lava_callback - INFO - Received callback for node 69c73d4429015d0f9599283a INFO: 172.21.0.1:49216 - "POST /node/69c73d4429015d0f9599283a HTTP/1.0" 202 Accepted 2026-03-28 02:38:02,244 - lava_callback - INFO - Starting async job submit for node 69c73d4429015d0f9599283a 2026-03-28 02:38:02,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:38:02,305 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73d4429015d0f9599283a HTTP/1.1" 200 None 2026-03-28 02:38:02,306 - lava_callback - DEBUG - Processing node 69c73d4429015d0f9599283a: baseline-arm64 2026-03-28 02:38:02,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:38:02,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:38:02,858 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73d4429015d0f9599283a/log.txt.gz 2026-03-28 02:38:03,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:38:03,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:38:03,309 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73d4429015d0f9599283a/lava-logs.yaml 2026-03-28 02:38:03,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:38:03,490 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:38:03,492 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73d4429015d0f9599283a/lava_callback.json.gz DEBUG: Setup failed for 69c73d4429015d0f9599283a. Transitting job node result: incomplete -> fail 2026-03-28 02:38:03,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:38:03,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73d4429015d0f9599283a HTTP/1.1" 200 None 2026-03-28 02:38:03,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:38:03,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:38:03,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:38:03,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73d4429015d0f9599283a HTTP/1.1" 200 None 2026-03-28 02:38:03,698 - lava_callback - INFO - Completed processing callback for node 69c73d4429015d0f9599283a 2026-03-28 02:38:06,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:38:06,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44394 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60898 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:39:07,769 - lava_callback - INFO - Received callback for node 69c49c37590270ad7b6d9a59 INFO: 172.21.0.1:60906 - "POST /node/69c49c37590270ad7b6d9a59 HTTP/1.0" 202 Accepted 2026-03-28 02:39:07,775 - lava_callback - INFO - Starting async job submit for node 69c49c37590270ad7b6d9a59 2026-03-28 02:39:07,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:07,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49c37590270ad7b6d9a59 HTTP/1.1" 200 None 2026-03-28 02:39:07,806 - lava_callback - DEBUG - Processing node 69c49c37590270ad7b6d9a59: baseline-arm64-clabbe 2026-03-28 02:39:07,807 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49c37590270ad7b6d9a59 2026-03-28 02:39:07,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:07,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49c37590270ad7b6d9a59 HTTP/1.1" 200 None 2026-03-28 02:39:07,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:07,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:39:07,850 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:07,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49c37590270ad7b6d9a59 HTTP/1.1" 200 None 2026-03-28 02:39:07,890 - lava_callback - ERROR - Error processing callback for node 69c49c37590270ad7b6d9a59: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59466 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:39:53,480 - lava_callback - INFO - Received callback for node 69c73eff29015d0f9599288f INFO: 172.21.0.1:59004 - "POST /node/69c73eff29015d0f9599288f HTTP/1.0" 202 Accepted 2026-03-28 02:39:53,487 - lava_callback - INFO - Starting async job submit for node 69c73eff29015d0f9599288f 2026-03-28 02:39:53,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:53,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73eff29015d0f9599288f HTTP/1.1" 200 None 2026-03-28 02:39:53,525 - lava_callback - DEBUG - Processing node 69c73eff29015d0f9599288f: baseline-arm64 2026-03-28 02:39:53,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:39:54,172 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:39:54,174 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73eff29015d0f9599288f/log.txt.gz 2026-03-28 02:39:54,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:39:54,637 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:39:54,639 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73eff29015d0f9599288f/lava-logs.yaml 2026-03-28 02:39:54,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:39:54,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:39:54,813 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73eff29015d0f9599288f/lava_callback.json.gz 2026-03-28 02:39:54,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:54,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73eff29015d0f9599288f HTTP/1.1" 200 None 2026-03-28 02:39:54,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:54,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:39:54,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:39:54,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73eff29015d0f9599288f HTTP/1.1" 200 None 2026-03-28 02:39:54,999 - lava_callback - INFO - Completed processing callback for node 69c73eff29015d0f9599288f INFO: 172.21.0.1:59018 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:40:06,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:40:06,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33900 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:41:27,821 - lava_callback - INFO - Received callback for node 69c49cd4590270ad7b6d9a87 INFO: 172.21.0.1:33914 - "POST /node/69c49cd4590270ad7b6d9a87 HTTP/1.0" 202 Accepted 2026-03-28 02:41:27,827 - lava_callback - INFO - Starting async job submit for node 69c49cd4590270ad7b6d9a87 2026-03-28 02:41:27,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:41:27,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49cd4590270ad7b6d9a87 HTTP/1.1" 200 None 2026-03-28 02:41:27,843 - lava_callback - DEBUG - Processing node 69c49cd4590270ad7b6d9a87: baseline-arm64-clabbe 2026-03-28 02:41:27,844 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49cd4590270ad7b6d9a87 2026-03-28 02:41:27,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:41:27,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49cd4590270ad7b6d9a87 HTTP/1.1" 200 None 2026-03-28 02:41:27,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:41:27,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:41:27,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:41:27,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49cd4590270ad7b6d9a87 HTTP/1.1" 200 None 2026-03-28 02:41:27,904 - lava_callback - ERROR - Error processing callback for node 69c49cd4590270ad7b6d9a87: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38240 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:42:32,559 - lava_callback - INFO - Received callback for node 69c49d11590270ad7b6d9a9a INFO: 172.21.0.1:46306 - "POST /node/69c49d11590270ad7b6d9a9a HTTP/1.0" 202 Accepted 2026-03-28 02:42:32,564 - lava_callback - INFO - Starting async job submit for node 69c49d11590270ad7b6d9a9a 2026-03-28 02:42:32,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:42:32,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d11590270ad7b6d9a9a HTTP/1.1" 200 None 2026-03-28 02:42:32,580 - lava_callback - DEBUG - Processing node 69c49d11590270ad7b6d9a9a: baseline-arm-baylibre 2026-03-28 02:42:32,580 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49d11590270ad7b6d9a9a 2026-03-28 02:42:32,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:42:32,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d11590270ad7b6d9a9a HTTP/1.1" 200 None 2026-03-28 02:42:32,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:42:32,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:42:32,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:42:32,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49d11590270ad7b6d9a9a HTTP/1.1" 200 None 2026-03-28 02:42:32,639 - lava_callback - ERROR - Error processing callback for node 69c49d11590270ad7b6d9a9a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45390 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:43:27,908 - lava_callback - INFO - Received callback for node 69c49d4d590270ad7b6d9aad INFO: 172.21.0.1:45400 - "POST /node/69c49d4d590270ad7b6d9aad HTTP/1.0" 202 Accepted 2026-03-28 02:43:27,913 - lava_callback - INFO - Starting async job submit for node 69c49d4d590270ad7b6d9aad 2026-03-28 02:43:27,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:43:27,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d4d590270ad7b6d9aad HTTP/1.1" 200 None 2026-03-28 02:43:27,930 - lava_callback - DEBUG - Processing node 69c49d4d590270ad7b6d9aad: baseline-arm64-clabbe 2026-03-28 02:43:27,930 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49d4d590270ad7b6d9aad 2026-03-28 02:43:27,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:43:27,944 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d4d590270ad7b6d9aad HTTP/1.1" 200 None 2026-03-28 02:43:27,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:43:27,961 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bc92e6b4744551b27f9fd3 HTTP/1.1" 200 None 2026-03-28 02:43:27,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:43:27,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49d4d590270ad7b6d9aad HTTP/1.1" 200 None 2026-03-28 02:43:27,995 - lava_callback - ERROR - Error processing callback for node 69c49d4d590270ad7b6d9aad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34928 - "GET /node/67ff8032c0534c95c39deb1e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:41398 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55940 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53326 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:44:32,356 - lava_callback - INFO - Received callback for node 69c49d8a590270ad7b6d9aba INFO: 172.21.0.1:52698 - "POST /node/69c49d8a590270ad7b6d9aba HTTP/1.0" 202 Accepted 2026-03-28 02:44:32,361 - lava_callback - INFO - Starting async job submit for node 69c49d8a590270ad7b6d9aba 2026-03-28 02:44:32,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:44:32,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d8a590270ad7b6d9aba HTTP/1.1" 200 None 2026-03-28 02:44:32,394 - lava_callback - DEBUG - Processing node 69c49d8a590270ad7b6d9aba: baseline-arm-baylibre 2026-03-28 02:44:32,394 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49d8a590270ad7b6d9aba 2026-03-28 02:44:32,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:44:32,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49d8a590270ad7b6d9aba HTTP/1.1" 200 None 2026-03-28 02:44:32,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:44:32,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:44:32,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:44:32,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49d8a590270ad7b6d9aba HTTP/1.1" 200 None 2026-03-28 02:44:32,459 - lava_callback - ERROR - Error processing callback for node 69c49d8a590270ad7b6d9aba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41064 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:45:12,375 - lava_callback - INFO - Received callback for node 69c49db1590270ad7b6d9acb INFO: 172.21.0.1:53936 - "POST /node/69c49db1590270ad7b6d9acb HTTP/1.0" 202 Accepted 2026-03-28 02:45:12,380 - lava_callback - INFO - Starting async job submit for node 69c49db1590270ad7b6d9acb 2026-03-28 02:45:12,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:12,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49db1590270ad7b6d9acb HTTP/1.1" 200 None 2026-03-28 02:45:12,396 - lava_callback - DEBUG - Processing node 69c49db1590270ad7b6d9acb: baseline-arm-baylibre 2026-03-28 02:45:12,396 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49db1590270ad7b6d9acb 2026-03-28 02:45:12,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:12,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49db1590270ad7b6d9acb HTTP/1.1" 200 None 2026-03-28 02:45:12,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:12,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:45:12,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:12,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49db1590270ad7b6d9acb HTTP/1.1" 200 None 2026-03-28 02:45:12,461 - lava_callback - ERROR - Error processing callback for node 69c49db1590270ad7b6d9acb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54048 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:45:52,407 - lava_callback - INFO - Received callback for node 69c49dd7590270ad7b6d9ad4 INFO: 172.21.0.1:57330 - "POST /node/69c49dd7590270ad7b6d9ad4 HTTP/1.0" 202 Accepted 2026-03-28 02:45:52,413 - lava_callback - INFO - Starting async job submit for node 69c49dd7590270ad7b6d9ad4 2026-03-28 02:45:52,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:52,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49dd7590270ad7b6d9ad4 HTTP/1.1" 200 None 2026-03-28 02:45:52,429 - lava_callback - DEBUG - Processing node 69c49dd7590270ad7b6d9ad4: baseline-arm-baylibre 2026-03-28 02:45:52,429 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49dd7590270ad7b6d9ad4 2026-03-28 02:45:52,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:52,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49dd7590270ad7b6d9ad4 HTTP/1.1" 200 None 2026-03-28 02:45:52,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:52,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:45:52,461 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:45:52,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49dd7590270ad7b6d9ad4 HTTP/1.1" 200 None 2026-03-28 02:45:52,494 - lava_callback - ERROR - Error processing callback for node 69c49dd7590270ad7b6d9ad4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:46:32,430 - lava_callback - INFO - Received callback for node 69c49dff590270ad7b6d9ae1 INFO: 172.21.0.1:57388 - "POST /node/69c49dff590270ad7b6d9ae1 HTTP/1.0" 202 Accepted 2026-03-28 02:46:32,436 - lava_callback - INFO - Starting async job submit for node 69c49dff590270ad7b6d9ae1 2026-03-28 02:46:32,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:32,452 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49dff590270ad7b6d9ae1 HTTP/1.1" 200 None 2026-03-28 02:46:32,453 - lava_callback - DEBUG - Processing node 69c49dff590270ad7b6d9ae1: baseline-arm-baylibre 2026-03-28 02:46:32,453 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49dff590270ad7b6d9ae1 2026-03-28 02:46:32,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:32,471 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49dff590270ad7b6d9ae1 HTTP/1.1" 200 None 2026-03-28 02:46:32,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:32,492 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:46:32,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:32,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49dff590270ad7b6d9ae1 HTTP/1.1" 200 None 2026-03-28 02:46:32,526 - lava_callback - ERROR - Error processing callback for node 69c49dff590270ad7b6d9ae1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57398 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:46:37,307 - lava_callback - INFO - Received callback for node 69c73fbd29015d0f959928c8 INFO: 172.21.0.1:57406 - "POST /node/69c73fbd29015d0f959928c8 HTTP/1.0" 202 Accepted 2026-03-28 02:46:37,315 - lava_callback - INFO - Starting async job submit for node 69c73fbd29015d0f959928c8 2026-03-28 02:46:37,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:37,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73fbd29015d0f959928c8 HTTP/1.1" 200 None 2026-03-28 02:46:37,372 - lava_callback - DEBUG - Processing node 69c73fbd29015d0f959928c8: baseline-arm64 2026-03-28 02:46:37,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:46:37,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:46:37,911 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73fbd29015d0f959928c8/log.txt.gz 2026-03-28 02:46:38,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:46:38,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:46:38,328 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73fbd29015d0f959928c8/lava-logs.yaml 2026-03-28 02:46:38,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:46:38,498 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:46:38,500 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c73fbd29015d0f959928c8/lava_callback.json.gz DEBUG: Setup failed for 69c73fbd29015d0f959928c8. Transitting job node result: incomplete -> fail 2026-03-28 02:46:38,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:38,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73fbd29015d0f959928c8 HTTP/1.1" 200 None 2026-03-28 02:46:38,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:38,604 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 02:46:38,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:46:38,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73fbd29015d0f959928c8 HTTP/1.1" 200 None 2026-03-28 02:46:38,724 - lava_callback - INFO - Completed processing callback for node 69c73fbd29015d0f959928c8 INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34934 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:47:06,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:47:06,422 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33178 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:47:12,560 - lava_callback - INFO - Received callback for node 69c49e26590270ad7b6d9af0 INFO: 172.21.0.1:36764 - "POST /node/69c49e26590270ad7b6d9af0 HTTP/1.0" 202 Accepted 2026-03-28 02:47:12,566 - lava_callback - INFO - Starting async job submit for node 69c49e26590270ad7b6d9af0 2026-03-28 02:47:12,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:47:12,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49e26590270ad7b6d9af0 HTTP/1.1" 200 None 2026-03-28 02:47:12,582 - lava_callback - DEBUG - Processing node 69c49e26590270ad7b6d9af0: baseline-arm-baylibre 2026-03-28 02:47:12,582 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49e26590270ad7b6d9af0 2026-03-28 02:47:12,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:47:12,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49e26590270ad7b6d9af0 HTTP/1.1" 200 None 2026-03-28 02:47:12,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:47:12,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:47:12,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:47:12,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49e26590270ad7b6d9af0 HTTP/1.1" 200 None 2026-03-28 02:47:12,662 - lava_callback - ERROR - Error processing callback for node 69c49e26590270ad7b6d9af0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:49:08,237 - lava_callback - INFO - Received callback for node 69c49e9a590270ad7b6d9b13 INFO: 172.21.0.1:33332 - "POST /node/69c49e9a590270ad7b6d9b13 HTTP/1.0" 202 Accepted 2026-03-28 02:49:08,243 - lava_callback - INFO - Starting async job submit for node 69c49e9a590270ad7b6d9b13 2026-03-28 02:49:08,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:08,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49e9a590270ad7b6d9b13 HTTP/1.1" 200 None 2026-03-28 02:49:08,263 - lava_callback - DEBUG - Processing node 69c49e9a590270ad7b6d9b13: baseline-arm-clabbe 2026-03-28 02:49:08,263 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49e9a590270ad7b6d9b13 2026-03-28 02:49:08,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:08,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49e9a590270ad7b6d9b13 HTTP/1.1" 200 None 2026-03-28 02:49:08,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:08,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:49:08,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:08,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49e9a590270ad7b6d9b13 HTTP/1.1" 200 None 2026-03-28 02:49:08,333 - lava_callback - ERROR - Error processing callback for node 69c49e9a590270ad7b6d9b13: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56872 - "GET /node/6992691098a8bdb31c392310%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:36808 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:49:48,227 - lava_callback - INFO - Received callback for node 69c49ec2590270ad7b6d9b1e INFO: 172.21.0.1:36816 - "POST /node/69c49ec2590270ad7b6d9b1e HTTP/1.0" 202 Accepted 2026-03-28 02:49:48,233 - lava_callback - INFO - Starting async job submit for node 69c49ec2590270ad7b6d9b1e 2026-03-28 02:49:48,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:48,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49ec2590270ad7b6d9b1e HTTP/1.1" 200 None 2026-03-28 02:49:48,249 - lava_callback - DEBUG - Processing node 69c49ec2590270ad7b6d9b1e: baseline-arm-clabbe 2026-03-28 02:49:48,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49ec2590270ad7b6d9b1e 2026-03-28 02:49:48,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:48,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49ec2590270ad7b6d9b1e HTTP/1.1" 200 None 2026-03-28 02:49:48,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:48,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:49:48,282 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:49:48,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49ec2590270ad7b6d9b1e HTTP/1.1" 200 None 2026-03-28 02:49:48,310 - lava_callback - ERROR - Error processing callback for node 69c49ec2590270ad7b6d9b1e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50088 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:50:28,278 - lava_callback - INFO - Received callback for node 69c49eea590270ad7b6d9b28 INFO: 172.21.0.1:50104 - "POST /node/69c49eea590270ad7b6d9b28 HTTP/1.0" 202 Accepted 2026-03-28 02:50:28,283 - lava_callback - INFO - Starting async job submit for node 69c49eea590270ad7b6d9b28 2026-03-28 02:50:28,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:50:28,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49eea590270ad7b6d9b28 HTTP/1.1" 200 None 2026-03-28 02:50:28,299 - lava_callback - DEBUG - Processing node 69c49eea590270ad7b6d9b28: baseline-arm-clabbe 2026-03-28 02:50:28,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49eea590270ad7b6d9b28 2026-03-28 02:50:28,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:50:28,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49eea590270ad7b6d9b28 HTTP/1.1" 200 None 2026-03-28 02:50:28,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:50:28,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:50:28,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:50:28,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49eea590270ad7b6d9b28 HTTP/1.1" 200 None 2026-03-28 02:50:28,361 - lava_callback - ERROR - Error processing callback for node 69c49eea590270ad7b6d9b28: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56142 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:51:08,291 - lava_callback - INFO - Received callback for node 69c49f12590270ad7b6d9b31 INFO: 172.21.0.1:56158 - "POST /node/69c49f12590270ad7b6d9b31 HTTP/1.0" 202 Accepted 2026-03-28 02:51:08,297 - lava_callback - INFO - Starting async job submit for node 69c49f12590270ad7b6d9b31 2026-03-28 02:51:08,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:08,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49f12590270ad7b6d9b31 HTTP/1.1" 200 None 2026-03-28 02:51:08,313 - lava_callback - DEBUG - Processing node 69c49f12590270ad7b6d9b31: baseline-arm-clabbe 2026-03-28 02:51:08,314 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49f12590270ad7b6d9b31 2026-03-28 02:51:08,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:08,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49f12590270ad7b6d9b31 HTTP/1.1" 200 None 2026-03-28 02:51:08,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:08,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:51:08,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:08,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49f12590270ad7b6d9b31 HTTP/1.1" 200 None 2026-03-28 02:51:08,375 - lava_callback - ERROR - Error processing callback for node 69c49f12590270ad7b6d9b31: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48326 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:51:48,351 - lava_callback - INFO - Received callback for node 69c49f3f590270ad7b6d9b3c INFO: 172.21.0.1:48328 - "POST /node/69c49f3f590270ad7b6d9b3c HTTP/1.0" 202 Accepted 2026-03-28 02:51:48,357 - lava_callback - INFO - Starting async job submit for node 69c49f3f590270ad7b6d9b3c 2026-03-28 02:51:48,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:48,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49f3f590270ad7b6d9b3c HTTP/1.1" 200 None 2026-03-28 02:51:48,392 - lava_callback - DEBUG - Processing node 69c49f3f590270ad7b6d9b3c: baseline-arm-clabbe 2026-03-28 02:51:48,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c49f3f590270ad7b6d9b3c 2026-03-28 02:51:48,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:48,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c49f3f590270ad7b6d9b3c HTTP/1.1" 200 None 2026-03-28 02:51:48,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:48,421 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399c1bbc022e792643e8d HTTP/1.1" 200 None 2026-03-28 02:51:48,423 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:51:48,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c49f3f590270ad7b6d9b3c HTTP/1.1" 200 None 2026-03-28 02:51:48,444 - lava_callback - ERROR - Error processing callback for node 69c49f3f590270ad7b6d9b3c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45884 - "GET /node/699268a098a8bdb31c392252%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:45894 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39110 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35324 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42824 - "GET /node/69bbb7b4ec3f435d20d51c72%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60584 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:55:25,292 - lava_callback - INFO - Received callback for node 69c73ed429015d0f9599287c INFO: 172.21.0.1:54484 - "POST /node/69c73ed429015d0f9599287c HTTP/1.0" 202 Accepted 2026-03-28 02:55:25,308 - lava_callback - INFO - Starting async job submit for node 69c73ed429015d0f9599287c 2026-03-28 02:55:25,383 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:55:25,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ed429015d0f9599287c HTTP/1.1" 200 None 2026-03-28 02:55:25,399 - lava_callback - DEBUG - Processing node 69c73ed429015d0f9599287c: baseline-arm 2026-03-28 02:55:25,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:55:26,145 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:55:26,146 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69c73ed429015d0f9599287c/log.txt.gz 2026-03-28 02:55:26,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:54496 - "GET / HTTP/1.0" 200 OK 2026-03-28 02:55:26,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:55:26,756 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69c73ed429015d0f9599287c/lava-logs.yaml 2026-03-28 02:55:26,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:55:27,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:55:27,113 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69c73ed429015d0f9599287c/lava_callback.json.gz DEBUG: Setup failed for 69c73ed429015d0f9599287c. Transitting job node result: incomplete -> fail 2026-03-28 02:55:27,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:55:27,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ed429015d0f9599287c HTTP/1.1" 200 None 2026-03-28 02:55:27,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:55:27,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 02:55:27,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:55:27,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73ed429015d0f9599287c HTTP/1.1" 200 None 2026-03-28 02:55:27,418 - lava_callback - INFO - Completed processing callback for node 69c73ed429015d0f9599287c 2026-03-28 02:55:36,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:55:36,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:35312 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53176 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59308 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33446 - "GET /node/68fad18ca011bb5928ee75a5%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-28 02:59:52,573 - lava_callback - INFO - Received callback for node 69c73f9d29015d0f959928b6 INFO: 172.21.0.1:54194 - "POST /node/69c73f9d29015d0f959928b6 HTTP/1.0" 202 Accepted 2026-03-28 02:59:52,621 - lava_callback - INFO - Starting async job submit for node 69c73f9d29015d0f959928b6 2026-03-28 02:59:52,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:59:52,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73f9d29015d0f959928b6 HTTP/1.1" 200 None 2026-03-28 02:59:52,672 - lava_callback - DEBUG - Processing node 69c73f9d29015d0f959928b6: baseline-arm-pengutronix 2026-03-28 02:59:52,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:59:53,073 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:59:53,074 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c73f9d29015d0f959928b6/log.txt.gz 2026-03-28 02:59:53,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:59:53,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:59:53,338 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c73f9d29015d0f959928b6/lava-logs.yaml 2026-03-28 02:59:53,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 02:59:53,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 02:59:53,628 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c73f9d29015d0f959928b6/lava_callback.json.gz DEBUG: Setup failed for 69c73f9d29015d0f959928b6. Transitting job node result: incomplete -> fail 2026-03-28 02:59:53,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:59:53,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73f9d29015d0f959928b6 HTTP/1.1" 200 None 2026-03-28 02:59:53,692 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:59:53,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 02:59:53,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 02:59:53,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73f9d29015d0f959928b6 HTTP/1.1" 200 None 2026-03-28 02:59:53,785 - lava_callback - INFO - Completed processing callback for node 69c73f9d29015d0f959928b6 INFO: 172.21.0.1:54198 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:00:06,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:00:06,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:57972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET /node/693e742a3c8f1be6d3cef710%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47898 - "GET /node/698e0aef952a14bf500e2400%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:33720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59858 - "GET /node/69876b22e31b58f834f197a8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:59870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59882 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41730 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52292 - "GET /node/69bbb7b8ec3f435d20d51c7f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:52302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37506 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:07:20,322 - lava_callback - INFO - Received callback for node 69c73ca929015d0f95992821 INFO: 172.21.0.1:46852 - "POST /node/69c73ca929015d0f95992821 HTTP/1.0" 202 Accepted 2026-03-28 03:07:20,331 - lava_callback - INFO - Starting async job submit for node 69c73ca929015d0f95992821 2026-03-28 03:07:20,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:07:20,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ca929015d0f95992821 HTTP/1.1" 200 None 2026-03-28 03:07:20,391 - lava_callback - DEBUG - Processing node 69c73ca929015d0f95992821: baseline-nfs-arm64 2026-03-28 03:07:21,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:07:21,706 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:07:21,707 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73ca929015d0f95992821/log.txt.gz 2026-03-28 03:07:22,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:07:22,621 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:07:22,623 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73ca929015d0f95992821/lava-logs.yaml 2026-03-28 03:07:22,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:07:22,886 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:07:22,888 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c73ca929015d0f95992821/lava_callback.json.gz DEBUG: Setup failed for 69c73ca929015d0f95992821. Transitting job node result: incomplete -> fail 2026-03-28 03:07:22,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:07:22,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ca929015d0f95992821 HTTP/1.1" 200 None 2026-03-28 03:07:22,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:07:23,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:07:23,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:07:23,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73ca929015d0f95992821 HTTP/1.1" 200 None 2026-03-28 03:07:23,144 - lava_callback - INFO - Completed processing callback for node 69c73ca929015d0f95992821 INFO: 172.21.0.1:46856 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:07:36,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:07:36,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:42638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43144 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:08:34,422 - lava_callback - INFO - Received callback for node 69c73ef929015d0f9599288b INFO: 172.21.0.1:47246 - "POST /node/69c73ef929015d0f9599288b HTTP/1.0" 202 Accepted 2026-03-28 03:08:34,429 - lava_callback - INFO - Starting async job submit for node 69c73ef929015d0f9599288b 2026-03-28 03:08:34,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:08:34,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ef929015d0f9599288b HTTP/1.1" 200 None 2026-03-28 03:08:34,508 - lava_callback - DEBUG - Processing node 69c73ef929015d0f9599288b: ltp-smoke 2026-03-28 03:08:35,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:08:35,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:08:35,348 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69c73ef929015d0f9599288b/log.txt.gz 2026-03-28 03:08:35,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:08:35,892 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:08:35,895 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69c73ef929015d0f9599288b/lava-logs.yaml 2026-03-28 03:08:35,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:08:36,058 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:08:36,060 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69c73ef929015d0f9599288b/lava_callback.json.gz 2026-03-28 03:08:36,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:08:36,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c73ef929015d0f9599288b HTTP/1.1" 200 None 2026-03-28 03:08:36,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:08:36,233 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4229015d0f9598f742 HTTP/1.1" 200 None 2026-03-28 03:08:36,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:08:36,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c73ef929015d0f9599288b HTTP/1.1" 200 None 2026-03-28 03:08:36,384 - lava_callback - INFO - Completed processing callback for node 69c73ef929015d0f9599288b 2026-03-28 03:08:36,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.21.0.1:47248 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:08:36,544 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:47260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38642 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59382 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:11:03,193 - lava_callback - INFO - Received callback for node 69c72d0e29015d0f959923bc INFO: 172.21.0.1:41980 - "POST /node/69c72d0e29015d0f959923bc HTTP/1.0" 202 Accepted 2026-03-28 03:11:03,199 - lava_callback - INFO - Starting async job submit for node 69c72d0e29015d0f959923bc 2026-03-28 03:11:03,222 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:11:03,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0e29015d0f959923bc HTTP/1.1" 200 None 2026-03-28 03:11:03,261 - lava_callback - DEBUG - Processing node 69c72d0e29015d0f959923bc: baseline-arm64-broonie 2026-03-28 03:11:03,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:11:03,660 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:11:03,662 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0e29015d0f959923bc/log.txt.gz 2026-03-28 03:11:03,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:11:04,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:11:04,012 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0e29015d0f959923bc/lava-logs.yaml 2026-03-28 03:11:04,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:11:04,205 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:11:04,206 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0e29015d0f959923bc/lava_callback.json.gz 2026-03-28 03:11:04,230 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:11:04,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0e29015d0f959923bc HTTP/1.1" 200 None 2026-03-28 03:11:04,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:11:04,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:11:04,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:11:04,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0e29015d0f959923bc HTTP/1.1" 200 None 2026-03-28 03:11:04,322 - lava_callback - INFO - Completed processing callback for node 69c72d0e29015d0f959923bc INFO: 172.21.0.1:41996 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:11:06,546 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:11:06,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:41718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41730 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51938 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46174 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60468 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54564 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:19:33,540 - lava_callback - INFO - Received callback for node 69c7443729015d0f959929d3 INFO: 172.21.0.1:51554 - "POST /node/69c7443729015d0f959929d3 HTTP/1.0" 202 Accepted 2026-03-28 03:19:33,589 - lava_callback - INFO - Starting async job submit for node 69c7443729015d0f959929d3 2026-03-28 03:19:33,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:19:33,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7443729015d0f959929d3 HTTP/1.1" 200 None 2026-03-28 03:19:33,650 - lava_callback - DEBUG - Processing node 69c7443729015d0f959929d3: baseline-arm-pengutronix 2026-03-28 03:19:33,775 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:19:33,934 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:19:33,936 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7443729015d0f959929d3/log.txt.gz 2026-03-28 03:19:33,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:19:34,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:19:34,182 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7443729015d0f959929d3/lava-logs.yaml 2026-03-28 03:19:34,185 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:19:34,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:19:34,326 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69c7443729015d0f959929d3/lava_callback.json.gz DEBUG: Setup failed for 69c7443729015d0f959929d3. Transitting job node result: incomplete -> fail 2026-03-28 03:19:34,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:19:34,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7443729015d0f959929d3 HTTP/1.1" 200 None 2026-03-28 03:19:34,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:19:34,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e3a29015d0f9598f73c HTTP/1.1" 200 None 2026-03-28 03:19:34,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:19:34,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7443729015d0f959929d3 HTTP/1.1" 200 None 2026-03-28 03:19:34,503 - lava_callback - INFO - Completed processing callback for node 69c7443729015d0f959929d3 INFO: 172.21.0.1:51558 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:19:36,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:19:36,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:51574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33132 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33186 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:21:14,442 - lava_callback - INFO - Received callback for node 69c4a623590270ad7b6d9bc6 INFO: 172.21.0.1:40270 - "POST /node/69c4a623590270ad7b6d9bc6 HTTP/1.0" 202 Accepted 2026-03-28 03:21:14,449 - lava_callback - INFO - Starting async job submit for node 69c4a623590270ad7b6d9bc6 2026-03-28 03:21:14,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:21:14,477 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a623590270ad7b6d9bc6 HTTP/1.1" 200 None 2026-03-28 03:21:14,478 - lava_callback - DEBUG - Processing node 69c4a623590270ad7b6d9bc6: baseline-arm64-baylibre 2026-03-28 03:21:14,478 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a623590270ad7b6d9bc6 2026-03-28 03:21:14,479 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:21:14,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a623590270ad7b6d9bc6 HTTP/1.1" 200 None 2026-03-28 03:21:14,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:21:14,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:21:14,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:21:14,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a623590270ad7b6d9bc6 HTTP/1.1" 200 None 2026-03-28 03:21:14,556 - lava_callback - ERROR - Error processing callback for node 69c4a623590270ad7b6d9bc6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38222 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:24:29,781 - lava_callback - INFO - Received callback for node 69c4a6db590270ad7b6d9bdd INFO: 172.21.0.1:44890 - "POST /node/69c4a6db590270ad7b6d9bdd HTTP/1.0" 202 Accepted 2026-03-28 03:24:29,786 - lava_callback - INFO - Starting async job submit for node 69c4a6db590270ad7b6d9bdd 2026-03-28 03:24:29,787 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:24:29,825 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a6db590270ad7b6d9bdd HTTP/1.1" 200 None 2026-03-28 03:24:29,825 - lava_callback - DEBUG - Processing node 69c4a6db590270ad7b6d9bdd: baseline-arm64-clabbe 2026-03-28 03:24:29,826 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a6db590270ad7b6d9bdd 2026-03-28 03:24:29,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:24:29,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a6db590270ad7b6d9bdd HTTP/1.1" 200 None 2026-03-28 03:24:29,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:24:29,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:24:29,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:24:29,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a6db590270ad7b6d9bdd HTTP/1.1" 200 None 2026-03-28 03:24:29,885 - lava_callback - ERROR - Error processing callback for node 69c4a6db590270ad7b6d9bdd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57854 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:25:48,648 - lava_callback - INFO - Received callback for node 69c72cd629015d0f959922ee INFO: 172.21.0.1:57856 - "POST /node/69c72cd629015d0f959922ee HTTP/1.0" 202 Accepted 2026-03-28 03:25:48,654 - lava_callback - INFO - Starting async job submit for node 69c72cd629015d0f959922ee 2026-03-28 03:25:48,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:25:48,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd629015d0f959922ee HTTP/1.1" 200 None 2026-03-28 03:25:48,688 - lava_callback - DEBUG - Processing node 69c72cd629015d0f959922ee: baseline-arm64 2026-03-28 03:25:49,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:25:49,209 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:25:49,211 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd629015d0f959922ee/log.txt.gz 2026-03-28 03:25:49,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:25:49,778 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:25:49,781 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd629015d0f959922ee/lava-logs.yaml 2026-03-28 03:25:49,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:25:49,955 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:25:49,957 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cd629015d0f959922ee/lava_callback.json.gz 2026-03-28 03:25:49,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:25:49,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cd629015d0f959922ee HTTP/1.1" 200 None 2026-03-28 03:25:49,991 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:25:50,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:25:50,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:25:50,030 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cd629015d0f959922ee HTTP/1.1" 200 None 2026-03-28 03:25:50,032 - lava_callback - INFO - Completed processing callback for node 69c72cd629015d0f959922ee INFO: 172.21.0.1:33716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34142 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:26:06,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:26:06,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42320 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:26:54,575 - lava_callback - INFO - Received callback for node 69c72ce729015d0f95992315 INFO: 172.21.0.1:50066 - "POST /node/69c72ce729015d0f95992315 HTTP/1.0" 202 Accepted 2026-03-28 03:26:54,581 - lava_callback - INFO - Starting async job submit for node 69c72ce729015d0f95992315 2026-03-28 03:26:54,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:26:54,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce729015d0f95992315 HTTP/1.1" 200 None 2026-03-28 03:26:54,617 - lava_callback - DEBUG - Processing node 69c72ce729015d0f95992315: baseline-nfs-arm64 2026-03-28 03:26:55,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:26:55,208 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:26:55,209 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce729015d0f95992315/log.txt.gz 2026-03-28 03:26:55,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:26:55,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:26:55,730 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce729015d0f95992315/lava-logs.yaml 2026-03-28 03:26:55,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:26:55,909 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:26:55,911 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72ce729015d0f95992315/lava_callback.json.gz 2026-03-28 03:26:55,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:26:55,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72ce729015d0f95992315 HTTP/1.1" 200 None 2026-03-28 03:26:55,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:26:55,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:26:55,966 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:26:56,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72ce729015d0f95992315 HTTP/1.1" 200 None 2026-03-28 03:26:56,006 - lava_callback - INFO - Completed processing callback for node 69c72ce729015d0f95992315 INFO: 172.21.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59196 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:27:06,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:06,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43272 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43282 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:27:18,569 - lava_callback - INFO - Received callback for node 69c745f429015d0f959929f3 INFO: 172.21.0.1:43286 - "POST /node/69c745f429015d0f959929f3 HTTP/1.0" 202 Accepted 2026-03-28 03:27:18,577 - lava_callback - INFO - Starting async job submit for node 69c745f429015d0f959929f3 2026-03-28 03:27:18,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:18,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c745f429015d0f959929f3 HTTP/1.1" 200 None 2026-03-28 03:27:18,637 - lava_callback - DEBUG - Processing node 69c745f429015d0f959929f3: baseline-nfs-arm64 2026-03-28 03:27:19,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:27:19,947 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:27:19,949 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c745f429015d0f959929f3/log.txt.gz 2026-03-28 03:27:20,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:27:20,827 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:27:20,829 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c745f429015d0f959929f3/lava-logs.yaml 2026-03-28 03:27:20,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:27:21,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:27:21,055 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c745f429015d0f959929f3/lava_callback.json.gz DEBUG: Setup failed for 69c745f429015d0f959929f3. Transitting job node result: incomplete -> fail 2026-03-28 03:27:21,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:21,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c745f429015d0f959929f3 HTTP/1.1" 200 None 2026-03-28 03:27:21,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:21,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:27:21,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:21,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c745f429015d0f959929f3 HTTP/1.1" 200 None 2026-03-28 03:27:21,301 - lava_callback - INFO - Completed processing callback for node 69c745f429015d0f959929f3 INFO: 172.21.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49088 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:27:36,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:27:36,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:49094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58852 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:28:29,900 - lava_callback - INFO - Received callback for node 69c4a7cb590270ad7b6d9bf1 INFO: 172.21.0.1:43742 - "POST /node/69c4a7cb590270ad7b6d9bf1 HTTP/1.0" 202 Accepted 2026-03-28 03:28:29,906 - lava_callback - INFO - Starting async job submit for node 69c4a7cb590270ad7b6d9bf1 2026-03-28 03:28:29,907 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:28:29,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a7cb590270ad7b6d9bf1 HTTP/1.1" 200 None 2026-03-28 03:28:29,922 - lava_callback - DEBUG - Processing node 69c4a7cb590270ad7b6d9bf1: baseline-arm64-clabbe 2026-03-28 03:28:29,922 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a7cb590270ad7b6d9bf1 2026-03-28 03:28:29,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:28:29,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a7cb590270ad7b6d9bf1 HTTP/1.1" 200 None 2026-03-28 03:28:29,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:28:29,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:28:29,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:28:29,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a7cb590270ad7b6d9bf1 HTTP/1.1" 200 None 2026-03-28 03:28:29,974 - lava_callback - ERROR - Error processing callback for node 69c4a7cb590270ad7b6d9bf1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43744 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40906 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46306 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:30:30,017 - lava_callback - INFO - Received callback for node 69c4a843590270ad7b6d9bff INFO: 172.21.0.1:37500 - "POST /node/69c4a843590270ad7b6d9bff HTTP/1.0" 202 Accepted 2026-03-28 03:30:30,023 - lava_callback - INFO - Starting async job submit for node 69c4a843590270ad7b6d9bff 2026-03-28 03:30:30,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:30:30,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a843590270ad7b6d9bff HTTP/1.1" 200 None 2026-03-28 03:30:30,039 - lava_callback - DEBUG - Processing node 69c4a843590270ad7b6d9bff: baseline-arm64-clabbe 2026-03-28 03:30:30,039 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a843590270ad7b6d9bff 2026-03-28 03:30:30,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:30:30,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a843590270ad7b6d9bff HTTP/1.1" 200 None 2026-03-28 03:30:30,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:30:30,076 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:30:30,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:30:30,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a843590270ad7b6d9bff HTTP/1.1" 200 None 2026-03-28 03:30:30,104 - lava_callback - ERROR - Error processing callback for node 69c4a843590270ad7b6d9bff: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34884 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:31:50,039 - lava_callback - INFO - Received callback for node 69c4a895590270ad7b6d9c0b INFO: 172.21.0.1:35138 - "POST /node/69c4a895590270ad7b6d9c0b HTTP/1.0" 202 Accepted 2026-03-28 03:31:50,045 - lava_callback - INFO - Starting async job submit for node 69c4a895590270ad7b6d9c0b 2026-03-28 03:31:50,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:31:50,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a895590270ad7b6d9c0b HTTP/1.1" 200 None 2026-03-28 03:31:50,080 - lava_callback - DEBUG - Processing node 69c4a895590270ad7b6d9c0b: baseline-arm64-clabbe 2026-03-28 03:31:50,080 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a895590270ad7b6d9c0b 2026-03-28 03:31:50,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:31:50,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a895590270ad7b6d9c0b HTTP/1.1" 200 None 2026-03-28 03:31:50,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:31:50,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:31:50,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:31:50,151 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a895590270ad7b6d9c0b HTTP/1.1" 200 None 2026-03-28 03:31:50,158 - lava_callback - ERROR - Error processing callback for node 69c4a895590270ad7b6d9c0b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38026 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47582 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:34:30,362 - lava_callback - INFO - Received callback for node 69c4a934590270ad7b6d9c1a INFO: 172.21.0.1:48792 - "POST /node/69c4a934590270ad7b6d9c1a HTTP/1.0" 202 Accepted 2026-03-28 03:34:30,367 - lava_callback - INFO - Starting async job submit for node 69c4a934590270ad7b6d9c1a 2026-03-28 03:34:30,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:34:30,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a934590270ad7b6d9c1a HTTP/1.1" 200 None 2026-03-28 03:34:30,383 - lava_callback - DEBUG - Processing node 69c4a934590270ad7b6d9c1a: baseline-arm64-clabbe 2026-03-28 03:34:30,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a934590270ad7b6d9c1a 2026-03-28 03:34:30,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:34:30,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a934590270ad7b6d9c1a HTTP/1.1" 200 None 2026-03-28 03:34:30,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:34:30,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:34:30,414 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:34:30,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a934590270ad7b6d9c1a HTTP/1.1" 200 None 2026-03-28 03:34:30,442 - lava_callback - ERROR - Error processing callback for node 69c4a934590270ad7b6d9c1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58646 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:37:30,300 - lava_callback - INFO - Received callback for node 69c4a9e9590270ad7b6d9c29 INFO: 172.21.0.1:55402 - "POST /node/69c4a9e9590270ad7b6d9c29 HTTP/1.0" 202 Accepted 2026-03-28 03:37:30,305 - lava_callback - INFO - Starting async job submit for node 69c4a9e9590270ad7b6d9c29 2026-03-28 03:37:30,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:37:30,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a9e9590270ad7b6d9c29 HTTP/1.1" 200 None 2026-03-28 03:37:30,321 - lava_callback - DEBUG - Processing node 69c4a9e9590270ad7b6d9c29: baseline-arm64-clabbe 2026-03-28 03:37:30,321 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4a9e9590270ad7b6d9c29 2026-03-28 03:37:30,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:37:30,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4a9e9590270ad7b6d9c29 HTTP/1.1" 200 None 2026-03-28 03:37:30,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:37:30,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c399cdbbc022e792643ea0 HTTP/1.1" 200 None 2026-03-28 03:37:30,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:37:30,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4a9e9590270ad7b6d9c29 HTTP/1.1" 200 None 2026-03-28 03:37:30,558 - lava_callback - ERROR - Error processing callback for node 69c4a9e9590270ad7b6d9c29: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55418 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42876 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36108 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57550 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55802 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55814 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:48:23,975 - lava_callback - INFO - Received callback for node 69c74ad529015d0f95992a84 INFO: 172.21.0.1:43396 - "POST /node/69c74ad529015d0f95992a84 HTTP/1.0" 202 Accepted 2026-03-28 03:48:23,983 - lava_callback - INFO - Starting async job submit for node 69c74ad529015d0f95992a84 2026-03-28 03:48:24,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:48:24,046 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c74ad529015d0f95992a84 HTTP/1.1" 200 None 2026-03-28 03:48:24,047 - lava_callback - DEBUG - Processing node 69c74ad529015d0f95992a84: baseline-nfs-arm64 2026-03-28 03:48:25,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:48:25,516 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:48:25,518 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c74ad529015d0f95992a84/log.txt.gz 2026-03-28 03:48:26,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:48:26,444 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:48:26,447 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c74ad529015d0f95992a84/lava-logs.yaml 2026-03-28 03:48:26,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:43408 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:48:26,648 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:48:26,650 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c74ad529015d0f95992a84/lava_callback.json.gz DEBUG: Setup failed for 69c74ad529015d0f95992a84. Transitting job node result: incomplete -> fail 2026-03-28 03:48:26,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:48:26,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c74ad529015d0f95992a84 HTTP/1.1" 200 None 2026-03-28 03:48:26,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:48:26,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:48:26,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:48:26,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c74ad529015d0f95992a84 HTTP/1.1" 200 None 2026-03-28 03:48:26,907 - lava_callback - INFO - Completed processing callback for node 69c74ad529015d0f95992a84 INFO: 172.21.0.1:53946 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:48:36,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:48:36,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39538 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48274 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43980 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46674 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:55:32,122 - lava_callback - INFO - Received callback for node 69c72cdf29015d0f95992303 INFO: 172.21.0.1:43814 - "POST /node/69c72cdf29015d0f95992303 HTTP/1.0" 202 Accepted 2026-03-28 03:55:32,128 - lava_callback - INFO - Starting async job submit for node 69c72cdf29015d0f95992303 2026-03-28 03:55:32,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:55:32,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdf29015d0f95992303 HTTP/1.1" 200 None 2026-03-28 03:55:32,168 - lava_callback - DEBUG - Processing node 69c72cdf29015d0f95992303: baseline-arm64 2026-03-28 03:55:32,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:55:32,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:55:32,859 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdf29015d0f95992303/log.txt.gz 2026-03-28 03:55:33,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:55:33,363 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:55:33,365 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdf29015d0f95992303/lava-logs.yaml 2026-03-28 03:55:33,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 03:55:33,528 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 03:55:33,530 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69c72cdf29015d0f95992303/lava_callback.json.gz 2026-03-28 03:55:33,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:55:33,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cdf29015d0f95992303 HTTP/1.1" 200 None 2026-03-28 03:55:33,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:55:33,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 03:55:33,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:55:33,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cdf29015d0f95992303 HTTP/1.1" 200 None 2026-03-28 03:55:33,691 - lava_callback - INFO - Completed processing callback for node 69c72cdf29015d0f95992303 INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK 2026-03-28 03:55:36,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 03:55:36,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50736 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57690 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58348 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56372 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56448 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60468 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53292 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48290 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39806 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58740 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46962 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42434 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42266 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40824 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32846 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:32860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50034 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60932 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51526 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50338 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34358 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42312 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44386 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50032 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53260 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49780 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60226 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40090 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:41:18,803 - lava_callback - INFO - Received callback for node 69c4b8dc590270ad7b6d9c7f INFO: 172.21.0.1:40098 - "POST /node/69c4b8dc590270ad7b6d9c7f HTTP/1.0" 202 Accepted 2026-03-28 04:41:18,809 - lava_callback - INFO - Starting async job submit for node 69c4b8dc590270ad7b6d9c7f 2026-03-28 04:41:18,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:41:18,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b8dc590270ad7b6d9c7f HTTP/1.1" 200 None 2026-03-28 04:41:18,825 - lava_callback - DEBUG - Processing node 69c4b8dc590270ad7b6d9c7f: baseline-arm-baylibre 2026-03-28 04:41:18,825 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b8dc590270ad7b6d9c7f 2026-03-28 04:41:18,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:41:18,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b8dc590270ad7b6d9c7f HTTP/1.1" 200 None 2026-03-28 04:41:18,845 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:41:18,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:41:18,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:41:18,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b8dc590270ad7b6d9c7f HTTP/1.1" 200 None 2026-03-28 04:41:18,891 - lava_callback - ERROR - Error processing callback for node 69c4b8dc590270ad7b6d9c7f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52830 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52848 - "GET /node/6992680698a8bdb31c3920ea%5Cn HTTP/1.0" 405 Method Not Allowed 2026-03-28 04:42:18,808 - lava_callback - INFO - Received callback for node 69c4b917590270ad7b6d9c8e INFO: 172.21.0.1:52862 - "POST /node/69c4b917590270ad7b6d9c8e HTTP/1.0" 202 Accepted 2026-03-28 04:42:18,814 - lava_callback - INFO - Starting async job submit for node 69c4b917590270ad7b6d9c8e 2026-03-28 04:42:18,815 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:18,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b917590270ad7b6d9c8e HTTP/1.1" 200 None 2026-03-28 04:42:18,831 - lava_callback - DEBUG - Processing node 69c4b917590270ad7b6d9c8e: baseline-arm-baylibre 2026-03-28 04:42:18,831 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b917590270ad7b6d9c8e 2026-03-28 04:42:18,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:19,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b917590270ad7b6d9c8e HTTP/1.1" 200 None 2026-03-28 04:42:19,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:19,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:42:19,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:19,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b917590270ad7b6d9c8e HTTP/1.1" 200 None 2026-03-28 04:42:19,068 - lava_callback - ERROR - Error processing callback for node 69c4b917590270ad7b6d9c8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38014 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38026 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:42:39,041 - lava_callback - INFO - Received callback for node 69c4b93b590270ad7b6d9c98 INFO: 172.21.0.1:38030 - "POST /node/69c4b93b590270ad7b6d9c98 HTTP/1.0" 202 Accepted 2026-03-28 04:42:39,046 - lava_callback - INFO - Starting async job submit for node 69c4b93b590270ad7b6d9c98 2026-03-28 04:42:39,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:39,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b93b590270ad7b6d9c98 HTTP/1.1" 200 None 2026-03-28 04:42:39,063 - lava_callback - DEBUG - Processing node 69c4b93b590270ad7b6d9c98: baseline-arm-baylibre 2026-03-28 04:42:39,064 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b93b590270ad7b6d9c98 2026-03-28 04:42:39,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:39,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b93b590270ad7b6d9c98 HTTP/1.1" 200 None 2026-03-28 04:42:39,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:39,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:42:39,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:42:39,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b93b590270ad7b6d9c98 HTTP/1.1" 200 None 2026-03-28 04:42:39,119 - lava_callback - ERROR - Error processing callback for node 69c4b93b590270ad7b6d9c98: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59794 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:43:18,867 - lava_callback - INFO - Received callback for node 69c4b960590270ad7b6d9ca3 INFO: 172.21.0.1:59804 - "POST /node/69c4b960590270ad7b6d9ca3 HTTP/1.0" 202 Accepted 2026-03-28 04:43:18,872 - lava_callback - INFO - Starting async job submit for node 69c4b960590270ad7b6d9ca3 2026-03-28 04:43:18,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:43:18,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b960590270ad7b6d9ca3 HTTP/1.1" 200 None 2026-03-28 04:43:18,896 - lava_callback - DEBUG - Processing node 69c4b960590270ad7b6d9ca3: baseline-arm-baylibre 2026-03-28 04:43:18,896 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b960590270ad7b6d9ca3 2026-03-28 04:43:18,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:43:18,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b960590270ad7b6d9ca3 HTTP/1.1" 200 None 2026-03-28 04:43:18,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:43:18,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:43:18,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:43:18,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b960590270ad7b6d9ca3 HTTP/1.1" 200 None 2026-03-28 04:43:18,980 - lava_callback - ERROR - Error processing callback for node 69c4b960590270ad7b6d9ca3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56224 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41744 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41746 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:44:18,973 - lava_callback - INFO - Received callback for node 69c4b990590270ad7b6d9cb3 INFO: 172.21.0.1:41760 - "POST /node/69c4b990590270ad7b6d9cb3 HTTP/1.0" 202 Accepted 2026-03-28 04:44:18,978 - lava_callback - INFO - Starting async job submit for node 69c4b990590270ad7b6d9cb3 2026-03-28 04:44:18,979 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:18,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b990590270ad7b6d9cb3 HTTP/1.1" 200 None 2026-03-28 04:44:18,995 - lava_callback - DEBUG - Processing node 69c4b990590270ad7b6d9cb3: baseline-arm-baylibre 2026-03-28 04:44:18,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b990590270ad7b6d9cb3 2026-03-28 04:44:18,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:19,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b990590270ad7b6d9cb3 HTTP/1.1" 200 None 2026-03-28 04:44:19,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:19,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:44:19,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:19,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b990590270ad7b6d9cb3 HTTP/1.1" 200 None 2026-03-28 04:44:19,075 - lava_callback - ERROR - Error processing callback for node 69c4b990590270ad7b6d9cb3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54048 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:44:58,947 - lava_callback - INFO - Received callback for node 69c4b9b5590270ad7b6d9cbf INFO: 172.21.0.1:54058 - "POST /node/69c4b9b5590270ad7b6d9cbf HTTP/1.0" 202 Accepted 2026-03-28 04:44:58,952 - lava_callback - INFO - Starting async job submit for node 69c4b9b5590270ad7b6d9cbf 2026-03-28 04:44:58,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:58,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b9b5590270ad7b6d9cbf HTTP/1.1" 200 None 2026-03-28 04:44:58,969 - lava_callback - DEBUG - Processing node 69c4b9b5590270ad7b6d9cbf: baseline-arm-baylibre 2026-03-28 04:44:58,969 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b9b5590270ad7b6d9cbf 2026-03-28 04:44:58,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:59,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b9b5590270ad7b6d9cbf HTTP/1.1" 200 None 2026-03-28 04:44:59,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:59,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:44:59,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:44:59,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b9b5590270ad7b6d9cbf HTTP/1.1" 200 None 2026-03-28 04:44:59,052 - lava_callback - ERROR - Error processing callback for node 69c4b9b5590270ad7b6d9cbf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45810 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:45:18,967 - lava_callback - INFO - Received callback for node 69c4b9da590270ad7b6d9cd0 INFO: 172.21.0.1:45814 - "POST /node/69c4b9da590270ad7b6d9cd0 HTTP/1.0" 202 Accepted 2026-03-28 04:45:18,973 - lava_callback - INFO - Starting async job submit for node 69c4b9da590270ad7b6d9cd0 2026-03-28 04:45:18,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:45:18,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b9da590270ad7b6d9cd0 HTTP/1.1" 200 None 2026-03-28 04:45:18,989 - lava_callback - DEBUG - Processing node 69c4b9da590270ad7b6d9cd0: baseline-arm-baylibre 2026-03-28 04:45:18,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4b9da590270ad7b6d9cd0 2026-03-28 04:45:18,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:45:19,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4b9da590270ad7b6d9cd0 HTTP/1.1" 200 None 2026-03-28 04:45:19,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:45:19,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:45:19,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:45:19,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4b9da590270ad7b6d9cd0 HTTP/1.1" 200 None 2026-03-28 04:45:19,072 - lava_callback - ERROR - Error processing callback for node 69c4b9da590270ad7b6d9cd0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42322 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34032 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:48:13,277 - lava_callback - INFO - Received callback for node 69c4ba7d590270ad7b6d9d07 INFO: 172.21.0.1:55932 - "POST /node/69c4ba7d590270ad7b6d9d07 HTTP/1.0" 202 Accepted 2026-03-28 04:48:13,284 - lava_callback - INFO - Starting async job submit for node 69c4ba7d590270ad7b6d9d07 2026-03-28 04:48:13,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:13,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ba7d590270ad7b6d9d07 HTTP/1.1" 200 None 2026-03-28 04:48:13,300 - lava_callback - DEBUG - Processing node 69c4ba7d590270ad7b6d9d07: baseline-arm-clabbe 2026-03-28 04:48:13,301 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4ba7d590270ad7b6d9d07 2026-03-28 04:48:13,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:13,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ba7d590270ad7b6d9d07 HTTP/1.1" 200 None 2026-03-28 04:48:13,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:13,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:48:13,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:13,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ba7d590270ad7b6d9d07 HTTP/1.1" 200 None 2026-03-28 04:48:13,354 - lava_callback - ERROR - Error processing callback for node 69c4ba7d590270ad7b6d9d07: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39322 - "GET /node/69b12a0da870a1dc3343bb74%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43840 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:48:53,347 - lava_callback - INFO - Received callback for node 69c4baa5590270ad7b6d9d1b INFO: 172.21.0.1:45498 - "POST /node/69c4baa5590270ad7b6d9d1b HTTP/1.0" 202 Accepted 2026-03-28 04:48:53,353 - lava_callback - INFO - Starting async job submit for node 69c4baa5590270ad7b6d9d1b 2026-03-28 04:48:53,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:53,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4baa5590270ad7b6d9d1b HTTP/1.1" 200 None 2026-03-28 04:48:53,370 - lava_callback - DEBUG - Processing node 69c4baa5590270ad7b6d9d1b: baseline-arm-clabbe 2026-03-28 04:48:53,370 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4baa5590270ad7b6d9d1b 2026-03-28 04:48:53,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:53,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4baa5590270ad7b6d9d1b HTTP/1.1" 200 None 2026-03-28 04:48:53,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:53,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:48:53,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:48:53,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4baa5590270ad7b6d9d1b HTTP/1.1" 200 None 2026-03-28 04:48:53,447 - lava_callback - ERROR - Error processing callback for node 69c4baa5590270ad7b6d9d1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35872 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:49:33,228 - lava_callback - INFO - Received callback for node 69c4bacd590270ad7b6d9d25 INFO: 172.21.0.1:50142 - "POST /node/69c4bacd590270ad7b6d9d25 HTTP/1.0" 202 Accepted 2026-03-28 04:49:33,233 - lava_callback - INFO - Starting async job submit for node 69c4bacd590270ad7b6d9d25 2026-03-28 04:49:33,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:49:33,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bacd590270ad7b6d9d25 HTTP/1.1" 200 None 2026-03-28 04:49:33,249 - lava_callback - DEBUG - Processing node 69c4bacd590270ad7b6d9d25: baseline-arm-clabbe 2026-03-28 04:49:33,250 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bacd590270ad7b6d9d25 2026-03-28 04:49:33,250 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:49:33,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bacd590270ad7b6d9d25 HTTP/1.1" 200 None 2026-03-28 04:49:33,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:49:33,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:49:33,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:49:33,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bacd590270ad7b6d9d25 HTTP/1.1" 200 None 2026-03-28 04:49:33,301 - lava_callback - ERROR - Error processing callback for node 69c4bacd590270ad7b6d9d25: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37758 - "GET /api/metrics HTTP/1.0" 200 OK 2026-03-28 04:50:15,871 - lava_callback - INFO - Received callback for node 69c4baf5590270ad7b6d9d38 INFO: 172.21.0.1:37764 - "POST /node/69c4baf5590270ad7b6d9d38 HTTP/1.0" 202 Accepted 2026-03-28 04:50:15,877 - lava_callback - INFO - Starting async job submit for node 69c4baf5590270ad7b6d9d38 2026-03-28 04:50:15,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:15,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4baf5590270ad7b6d9d38 HTTP/1.1" 200 None 2026-03-28 04:50:15,893 - lava_callback - DEBUG - Processing node 69c4baf5590270ad7b6d9d38: baseline-arm-clabbe 2026-03-28 04:50:15,893 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4baf5590270ad7b6d9d38 2026-03-28 04:50:15,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:15,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4baf5590270ad7b6d9d38 HTTP/1.1" 200 None 2026-03-28 04:50:15,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:15,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:50:15,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:15,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4baf5590270ad7b6d9d38 HTTP/1.1" 200 None 2026-03-28 04:50:15,957 - lava_callback - ERROR - Error processing callback for node 69c4baf5590270ad7b6d9d38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58832 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:50:39,291 - lava_callback - INFO - Received callback for node 69c4bb17590270ad7b6d9d49 INFO: 172.21.0.1:58848 - "POST /node/69c4bb17590270ad7b6d9d49 HTTP/1.0" 202 Accepted 2026-03-28 04:50:39,296 - lava_callback - INFO - Starting async job submit for node 69c4bb17590270ad7b6d9d49 2026-03-28 04:50:39,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:39,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb17590270ad7b6d9d49 HTTP/1.1" 200 None 2026-03-28 04:50:39,320 - lava_callback - DEBUG - Processing node 69c4bb17590270ad7b6d9d49: baseline-arm-baylibre 2026-03-28 04:50:39,320 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bb17590270ad7b6d9d49 2026-03-28 04:50:39,321 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:39,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb17590270ad7b6d9d49 HTTP/1.1" 200 None 2026-03-28 04:50:39,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:39,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:50:39,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:50:39,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bb17590270ad7b6d9d49 HTTP/1.1" 200 None 2026-03-28 04:50:39,388 - lava_callback - ERROR - Error processing callback for node 69c4bb17590270ad7b6d9d49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56466 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56474 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:51:19,302 - lava_callback - INFO - Received callback for node 69c4bb3c590270ad7b6d9d5a INFO: 172.21.0.1:56486 - "POST /node/69c4bb3c590270ad7b6d9d5a HTTP/1.0" 202 Accepted 2026-03-28 04:51:19,308 - lava_callback - INFO - Starting async job submit for node 69c4bb3c590270ad7b6d9d5a 2026-03-28 04:51:19,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:19,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb3c590270ad7b6d9d5a HTTP/1.1" 200 None 2026-03-28 04:51:19,324 - lava_callback - DEBUG - Processing node 69c4bb3c590270ad7b6d9d5a: baseline-arm-baylibre 2026-03-28 04:51:19,325 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bb3c590270ad7b6d9d5a 2026-03-28 04:51:19,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:19,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb3c590270ad7b6d9d5a HTTP/1.1" 200 None 2026-03-28 04:51:19,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:19,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:51:19,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:19,375 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bb3c590270ad7b6d9d5a HTTP/1.1" 200 None 2026-03-28 04:51:19,377 - lava_callback - ERROR - Error processing callback for node 69c4bb3c590270ad7b6d9d5a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36930 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36946 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60660 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:51:53,415 - lava_callback - INFO - Received callback for node 69c4bb63590270ad7b6d9d67 INFO: 172.21.0.1:52350 - "POST /node/69c4bb63590270ad7b6d9d67 HTTP/1.0" 202 Accepted 2026-03-28 04:51:53,421 - lava_callback - INFO - Starting async job submit for node 69c4bb63590270ad7b6d9d67 2026-03-28 04:51:53,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:53,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb63590270ad7b6d9d67 HTTP/1.1" 200 None 2026-03-28 04:51:53,437 - lava_callback - DEBUG - Processing node 69c4bb63590270ad7b6d9d67: baseline-arm-clabbe 2026-03-28 04:51:53,437 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bb63590270ad7b6d9d67 2026-03-28 04:51:53,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:53,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bb63590270ad7b6d9d67 HTTP/1.1" 200 None 2026-03-28 04:51:53,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:53,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:51:53,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:51:53,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bb63590270ad7b6d9d67 HTTP/1.1" 200 None 2026-03-28 04:51:53,501 - lava_callback - ERROR - Error processing callback for node 69c4bb63590270ad7b6d9d67: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58054 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58990 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:53:53,473 - lava_callback - INFO - Received callback for node 69c4bbd0590270ad7b6d9da6 INFO: 172.21.0.1:36528 - "POST /node/69c4bbd0590270ad7b6d9da6 HTTP/1.0" 202 Accepted 2026-03-28 04:53:53,479 - lava_callback - INFO - Starting async job submit for node 69c4bbd0590270ad7b6d9da6 2026-03-28 04:53:53,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:53:53,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bbd0590270ad7b6d9da6 HTTP/1.1" 200 None 2026-03-28 04:53:53,498 - lava_callback - DEBUG - Processing node 69c4bbd0590270ad7b6d9da6: baseline-arm-clabbe 2026-03-28 04:53:53,498 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bbd0590270ad7b6d9da6 2026-03-28 04:53:53,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:53:53,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bbd0590270ad7b6d9da6 HTTP/1.1" 200 None 2026-03-28 04:53:53,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:53:53,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:53:53,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:53:53,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bbd0590270ad7b6d9da6 HTTP/1.1" 200 None 2026-03-28 04:53:53,557 - lava_callback - ERROR - Error processing callback for node 69c4bbd0590270ad7b6d9da6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42822 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:54:33,473 - lava_callback - INFO - Received callback for node 69c4bbf6590270ad7b6d9dbd INFO: 172.21.0.1:53382 - "POST /node/69c4bbf6590270ad7b6d9dbd HTTP/1.0" 202 Accepted 2026-03-28 04:54:33,479 - lava_callback - INFO - Starting async job submit for node 69c4bbf6590270ad7b6d9dbd 2026-03-28 04:54:33,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:33,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bbf6590270ad7b6d9dbd HTTP/1.1" 200 None 2026-03-28 04:54:33,495 - lava_callback - DEBUG - Processing node 69c4bbf6590270ad7b6d9dbd: baseline-arm-clabbe 2026-03-28 04:54:33,495 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bbf6590270ad7b6d9dbd 2026-03-28 04:54:33,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:33,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bbf6590270ad7b6d9dbd HTTP/1.1" 200 None 2026-03-28 04:54:33,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:33,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:54:33,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:33,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bbf6590270ad7b6d9dbd HTTP/1.1" 200 None 2026-03-28 04:54:33,546 - lava_callback - ERROR - Error processing callback for node 69c4bbf6590270ad7b6d9dbd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57954 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:54:53,510 - lava_callback - INFO - Received callback for node 69c4bc1b590270ad7b6d9dcf INFO: 172.21.0.1:41414 - "POST /node/69c4bc1b590270ad7b6d9dcf HTTP/1.0" 202 Accepted 2026-03-28 04:54:53,515 - lava_callback - INFO - Starting async job submit for node 69c4bc1b590270ad7b6d9dcf 2026-03-28 04:54:53,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:53,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc1b590270ad7b6d9dcf HTTP/1.1" 200 None 2026-03-28 04:54:53,531 - lava_callback - DEBUG - Processing node 69c4bc1b590270ad7b6d9dcf: baseline-arm-clabbe 2026-03-28 04:54:53,531 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bc1b590270ad7b6d9dcf 2026-03-28 04:54:53,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:53,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc1b590270ad7b6d9dcf HTTP/1.1" 200 None 2026-03-28 04:54:53,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:53,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:54:53,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:54:53,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bc1b590270ad7b6d9dcf HTTP/1.1" 200 None 2026-03-28 04:54:53,596 - lava_callback - ERROR - Error processing callback for node 69c4bc1b590270ad7b6d9dcf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41426 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53880 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:55:33,500 - lava_callback - INFO - Received callback for node 69c4bc40590270ad7b6d9de4 INFO: 172.21.0.1:48558 - "POST /node/69c4bc40590270ad7b6d9de4 HTTP/1.0" 202 Accepted 2026-03-28 04:55:33,505 - lava_callback - INFO - Starting async job submit for node 69c4bc40590270ad7b6d9de4 2026-03-28 04:55:33,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:55:33,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc40590270ad7b6d9de4 HTTP/1.1" 200 None 2026-03-28 04:55:33,521 - lava_callback - DEBUG - Processing node 69c4bc40590270ad7b6d9de4: baseline-arm-clabbe 2026-03-28 04:55:33,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bc40590270ad7b6d9de4 2026-03-28 04:55:33,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:55:33,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc40590270ad7b6d9de4 HTTP/1.1" 200 None 2026-03-28 04:55:33,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:55:33,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:55:33,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:55:33,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bc40590270ad7b6d9de4 HTTP/1.1" 200 None 2026-03-28 04:55:33,571 - lava_callback - ERROR - Error processing callback for node 69c4bc40590270ad7b6d9de4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48574 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53848 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:56:13,582 - lava_callback - INFO - Received callback for node 69c4bc66590270ad7b6d9e01 INFO: 172.21.0.1:45602 - "POST /node/69c4bc66590270ad7b6d9e01 HTTP/1.0" 202 Accepted 2026-03-28 04:56:13,588 - lava_callback - INFO - Starting async job submit for node 69c4bc66590270ad7b6d9e01 2026-03-28 04:56:13,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:56:13,605 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc66590270ad7b6d9e01 HTTP/1.1" 200 None 2026-03-28 04:56:13,606 - lava_callback - DEBUG - Processing node 69c4bc66590270ad7b6d9e01: baseline-arm-clabbe 2026-03-28 04:56:13,606 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bc66590270ad7b6d9e01 2026-03-28 04:56:13,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:56:13,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bc66590270ad7b6d9e01 HTTP/1.1" 200 None 2026-03-28 04:56:13,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:56:13,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ac8944faaf0e7f5e408 HTTP/1.1" 200 None 2026-03-28 04:56:13,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:56:13,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bc66590270ad7b6d9e01 HTTP/1.1" 200 None 2026-03-28 04:56:13,687 - lava_callback - ERROR - Error processing callback for node 69c4bc66590270ad7b6d9e01: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45612 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49116 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45684 - "GET /node/6993f9bc3e9ba4930771d9a2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:44862 - "GET /node/69a0078262feda9f0694ec6f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52728 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:59:13,702 - lava_callback - INFO - Received callback for node 69c4bd1a590270ad7b6d9e27 INFO: 172.21.0.1:58382 - "POST /node/69c4bd1a590270ad7b6d9e27 HTTP/1.0" 202 Accepted 2026-03-28 04:59:13,708 - lava_callback - INFO - Starting async job submit for node 69c4bd1a590270ad7b6d9e27 2026-03-28 04:59:13,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:13,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd1a590270ad7b6d9e27 HTTP/1.1" 200 None 2026-03-28 04:59:13,724 - lava_callback - DEBUG - Processing node 69c4bd1a590270ad7b6d9e27: baseline-arm-clabbe 2026-03-28 04:59:13,724 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bd1a590270ad7b6d9e27 2026-03-28 04:59:13,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:13,738 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd1a590270ad7b6d9e27 HTTP/1.1" 200 None 2026-03-28 04:59:13,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:13,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:59:13,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:13,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bd1a590270ad7b6d9e27 HTTP/1.1" 200 None 2026-03-28 04:59:13,775 - lava_callback - ERROR - Error processing callback for node 69c4bd1a590270ad7b6d9e27: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34158 - "GET /node/69865c69177ca0f097bd452b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:34164 - "GET /node/6989388885773790cd34d199%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.21.0.1:34174 - "GET / HTTP/1.0" 200 OK 2026-03-28 04:59:53,675 - lava_callback - INFO - Received callback for node 69c4bd41590270ad7b6d9e30 INFO: 172.21.0.1:48526 - "POST /node/69c4bd41590270ad7b6d9e30 HTTP/1.0" 202 Accepted 2026-03-28 04:59:53,681 - lava_callback - INFO - Starting async job submit for node 69c4bd41590270ad7b6d9e30 2026-03-28 04:59:53,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:53,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd41590270ad7b6d9e30 HTTP/1.1" 200 None 2026-03-28 04:59:53,697 - lava_callback - DEBUG - Processing node 69c4bd41590270ad7b6d9e30: baseline-arm-clabbe 2026-03-28 04:59:53,697 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bd41590270ad7b6d9e30 2026-03-28 04:59:53,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:53,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd41590270ad7b6d9e30 HTTP/1.1" 200 None 2026-03-28 04:59:53,739 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:53,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 04:59:53,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 04:59:53,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bd41590270ad7b6d9e30 HTTP/1.1" 200 None 2026-03-28 04:59:53,780 - lava_callback - ERROR - Error processing callback for node 69c4bd41590270ad7b6d9e30: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45588 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:00:33,742 - lava_callback - INFO - Received callback for node 69c4bd66590270ad7b6d9e45 INFO: 172.21.0.1:42226 - "POST /node/69c4bd66590270ad7b6d9e45 HTTP/1.0" 202 Accepted 2026-03-28 05:00:33,748 - lava_callback - INFO - Starting async job submit for node 69c4bd66590270ad7b6d9e45 2026-03-28 05:00:33,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:00:33,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd66590270ad7b6d9e45 HTTP/1.1" 200 None 2026-03-28 05:00:33,766 - lava_callback - DEBUG - Processing node 69c4bd66590270ad7b6d9e45: baseline-arm-clabbe 2026-03-28 05:00:33,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bd66590270ad7b6d9e45 2026-03-28 05:00:33,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:00:33,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bd66590270ad7b6d9e45 HTTP/1.1" 200 None 2026-03-28 05:00:33,782 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:00:33,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:00:33,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:00:33,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bd66590270ad7b6d9e45 HTTP/1.1" 200 None 2026-03-28 05:00:33,851 - lava_callback - ERROR - Error processing callback for node 69c4bd66590270ad7b6d9e45: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43066 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36390 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60640 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41460 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58008 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49092 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44400 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37096 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57972 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45290 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:07:54,344 - lava_callback - INFO - Received callback for node 69c4bf24590270ad7b6d9e90 INFO: 172.21.0.1:43432 - "POST /node/69c4bf24590270ad7b6d9e90 HTTP/1.0" 202 Accepted 2026-03-28 05:07:54,350 - lava_callback - INFO - Starting async job submit for node 69c4bf24590270ad7b6d9e90 2026-03-28 05:07:54,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:07:54,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bf24590270ad7b6d9e90 HTTP/1.1" 200 None 2026-03-28 05:07:54,367 - lava_callback - DEBUG - Processing node 69c4bf24590270ad7b6d9e90: baseline-arm-clabbe 2026-03-28 05:07:54,367 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bf24590270ad7b6d9e90 2026-03-28 05:07:54,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:07:54,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bf24590270ad7b6d9e90 HTTP/1.1" 200 None 2026-03-28 05:07:54,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:07:54,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:07:54,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:07:54,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bf24590270ad7b6d9e90 HTTP/1.1" 200 None 2026-03-28 05:07:54,417 - lava_callback - ERROR - Error processing callback for node 69c4bf24590270ad7b6d9e90: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54590 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:08:34,129 - lava_callback - INFO - Received callback for node 69c4bf48590270ad7b6d9ea2 INFO: 172.21.0.1:38518 - "POST /node/69c4bf48590270ad7b6d9ea2 HTTP/1.0" 202 Accepted 2026-03-28 05:08:34,135 - lava_callback - INFO - Starting async job submit for node 69c4bf48590270ad7b6d9ea2 2026-03-28 05:08:34,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:08:34,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bf48590270ad7b6d9ea2 HTTP/1.1" 200 None 2026-03-28 05:08:34,151 - lava_callback - DEBUG - Processing node 69c4bf48590270ad7b6d9ea2: baseline-arm-clabbe 2026-03-28 05:08:34,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4bf48590270ad7b6d9ea2 2026-03-28 05:08:34,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:08:34,165 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4bf48590270ad7b6d9ea2 HTTP/1.1" 200 None 2026-03-28 05:08:34,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:08:34,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:08:34,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:08:34,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4bf48590270ad7b6d9ea2 HTTP/1.1" 200 None 2026-03-28 05:08:34,215 - lava_callback - ERROR - Error processing callback for node 69c4bf48590270ad7b6d9ea2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50938 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:12:20,495 - lava_callback - INFO - Received callback for node 69c4c02d590270ad7b6d9ec7 INFO: 172.21.0.1:55194 - "POST /node/69c4c02d590270ad7b6d9ec7 HTTP/1.0" 202 Accepted 2026-03-28 05:12:20,501 - lava_callback - INFO - Starting async job submit for node 69c4c02d590270ad7b6d9ec7 2026-03-28 05:12:20,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:12:20,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c02d590270ad7b6d9ec7 HTTP/1.1" 200 None 2026-03-28 05:12:20,516 - lava_callback - DEBUG - Processing node 69c4c02d590270ad7b6d9ec7: baseline-arm64-baylibre 2026-03-28 05:12:20,516 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c02d590270ad7b6d9ec7 2026-03-28 05:12:20,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:12:20,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c02d590270ad7b6d9ec7 HTTP/1.1" 200 None 2026-03-28 05:12:20,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:12:20,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:12:20,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:12:20,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c02d590270ad7b6d9ec7 HTTP/1.1" 200 None 2026-03-28 05:12:20,575 - lava_callback - ERROR - Error processing callback for node 69c4c02d590270ad7b6d9ec7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43878 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43894 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:13:40,610 - lava_callback - INFO - Received callback for node 69c4c07d590270ad7b6d9ed2 INFO: 172.21.0.1:47938 - "POST /node/69c4c07d590270ad7b6d9ed2 HTTP/1.0" 202 Accepted 2026-03-28 05:13:40,615 - lava_callback - INFO - Starting async job submit for node 69c4c07d590270ad7b6d9ed2 2026-03-28 05:13:40,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:13:40,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c07d590270ad7b6d9ed2 HTTP/1.1" 200 None 2026-03-28 05:13:40,631 - lava_callback - DEBUG - Processing node 69c4c07d590270ad7b6d9ed2: baseline-arm64-baylibre 2026-03-28 05:13:40,631 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c07d590270ad7b6d9ed2 2026-03-28 05:13:40,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:13:40,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c07d590270ad7b6d9ed2 HTTP/1.1" 200 None 2026-03-28 05:13:40,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:13:40,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:13:40,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:13:40,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c07d590270ad7b6d9ed2 HTTP/1.1" 200 None 2026-03-28 05:13:40,695 - lava_callback - ERROR - Error processing callback for node 69c4c07d590270ad7b6d9ed2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41046 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:15:00,674 - lava_callback - INFO - Received callback for node 69c4c0cd590270ad7b6d9edf INFO: 172.21.0.1:43636 - "POST /node/69c4c0cd590270ad7b6d9edf HTTP/1.0" 202 Accepted 2026-03-28 05:15:00,679 - lava_callback - INFO - Starting async job submit for node 69c4c0cd590270ad7b6d9edf 2026-03-28 05:15:00,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:15:00,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c0cd590270ad7b6d9edf HTTP/1.1" 200 None 2026-03-28 05:15:00,696 - lava_callback - DEBUG - Processing node 69c4c0cd590270ad7b6d9edf: baseline-arm64-baylibre 2026-03-28 05:15:00,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c0cd590270ad7b6d9edf 2026-03-28 05:15:00,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:15:00,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c0cd590270ad7b6d9edf HTTP/1.1" 200 None 2026-03-28 05:15:00,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:15:00,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:15:00,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:15:00,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c0cd590270ad7b6d9edf HTTP/1.1" 200 None 2026-03-28 05:15:00,766 - lava_callback - ERROR - Error processing callback for node 69c4c0cd590270ad7b6d9edf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49966 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34484 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34494 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:16:20,721 - lava_callback - INFO - Received callback for node 69c4c11f590270ad7b6d9eec INFO: 172.21.0.1:37080 - "POST /node/69c4c11f590270ad7b6d9eec HTTP/1.0" 202 Accepted 2026-03-28 05:16:20,727 - lava_callback - INFO - Starting async job submit for node 69c4c11f590270ad7b6d9eec 2026-03-28 05:16:20,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:16:20,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c11f590270ad7b6d9eec HTTP/1.1" 200 None 2026-03-28 05:16:20,743 - lava_callback - DEBUG - Processing node 69c4c11f590270ad7b6d9eec: baseline-arm64-baylibre 2026-03-28 05:16:20,743 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c11f590270ad7b6d9eec 2026-03-28 05:16:20,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:16:20,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c11f590270ad7b6d9eec HTTP/1.1" 200 None 2026-03-28 05:16:20,770 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:16:20,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:16:20,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:16:20,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c11f590270ad7b6d9eec HTTP/1.1" 200 None 2026-03-28 05:16:20,807 - lava_callback - ERROR - Error processing callback for node 69c4c11f590270ad7b6d9eec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54370 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48800 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:18:14,575 - lava_callback - INFO - Received callback for node 69c4c190590270ad7b6d9efc INFO: 172.21.0.1:56016 - "POST /node/69c4c190590270ad7b6d9efc HTTP/1.0" 202 Accepted 2026-03-28 05:18:14,581 - lava_callback - INFO - Starting async job submit for node 69c4c190590270ad7b6d9efc 2026-03-28 05:18:14,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:18:14,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c190590270ad7b6d9efc HTTP/1.1" 200 None 2026-03-28 05:18:14,596 - lava_callback - DEBUG - Processing node 69c4c190590270ad7b6d9efc: baseline-arm-clabbe 2026-03-28 05:18:14,596 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c190590270ad7b6d9efc 2026-03-28 05:18:14,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:18:14,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c190590270ad7b6d9efc HTTP/1.1" 200 None 2026-03-28 05:18:14,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:18:14,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:18:14,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:18:14,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c190590270ad7b6d9efc HTTP/1.1" 200 None 2026-03-28 05:18:14,646 - lava_callback - ERROR - Error processing callback for node 69c4c190590270ad7b6d9efc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56042 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44488 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44502 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57682 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44396 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48818 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48828 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33144 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:22:34,767 - lava_callback - INFO - Received callback for node 69c4c28f590270ad7b6d9f17 INFO: 172.21.0.1:51614 - "POST /node/69c4c28f590270ad7b6d9f17 HTTP/1.0" 202 Accepted 2026-03-28 05:22:34,772 - lava_callback - INFO - Starting async job submit for node 69c4c28f590270ad7b6d9f17 2026-03-28 05:22:34,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:22:34,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c28f590270ad7b6d9f17 HTTP/1.1" 200 None 2026-03-28 05:22:34,788 - lava_callback - DEBUG - Processing node 69c4c28f590270ad7b6d9f17: baseline-arm64-clabbe 2026-03-28 05:22:34,788 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c28f590270ad7b6d9f17 2026-03-28 05:22:34,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:22:34,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c28f590270ad7b6d9f17 HTTP/1.1" 200 None 2026-03-28 05:22:34,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:22:34,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:22:34,822 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:22:34,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c28f590270ad7b6d9f17 HTTP/1.1" 200 None 2026-03-28 05:22:34,853 - lava_callback - ERROR - Error processing callback for node 69c4c28f590270ad7b6d9f17: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57156 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52918 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56488 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:23:54,750 - lava_callback - INFO - Received callback for node 69c4c2de590270ad7b6d9f26 INFO: 172.21.0.1:51298 - "POST /node/69c4c2de590270ad7b6d9f26 HTTP/1.0" 202 Accepted 2026-03-28 05:23:54,756 - lava_callback - INFO - Starting async job submit for node 69c4c2de590270ad7b6d9f26 2026-03-28 05:23:54,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:23:54,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c2de590270ad7b6d9f26 HTTP/1.1" 200 None 2026-03-28 05:23:54,777 - lava_callback - DEBUG - Processing node 69c4c2de590270ad7b6d9f26: baseline-arm64-clabbe 2026-03-28 05:23:54,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c2de590270ad7b6d9f26 2026-03-28 05:23:54,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:23:54,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c2de590270ad7b6d9f26 HTTP/1.1" 200 None 2026-03-28 05:23:54,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:23:54,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:23:54,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:23:54,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c2de590270ad7b6d9f26 HTTP/1.1" 200 None 2026-03-28 05:23:54,848 - lava_callback - ERROR - Error processing callback for node 69c4c2de590270ad7b6d9f26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34500 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:25:34,843 - lava_callback - INFO - Received callback for node 69c4c343590270ad7b6d9f2f INFO: 172.21.0.1:56940 - "POST /node/69c4c343590270ad7b6d9f2f HTTP/1.0" 202 Accepted 2026-03-28 05:25:34,850 - lava_callback - INFO - Starting async job submit for node 69c4c343590270ad7b6d9f2f 2026-03-28 05:25:34,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:25:34,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c343590270ad7b6d9f2f HTTP/1.1" 200 None 2026-03-28 05:25:34,866 - lava_callback - DEBUG - Processing node 69c4c343590270ad7b6d9f2f: baseline-arm64-clabbe 2026-03-28 05:25:34,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c343590270ad7b6d9f2f 2026-03-28 05:25:34,866 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:25:34,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c343590270ad7b6d9f2f HTTP/1.1" 200 None 2026-03-28 05:25:34,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:25:34,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:25:34,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:25:34,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c343590270ad7b6d9f2f HTTP/1.1" 200 None 2026-03-28 05:25:34,917 - lava_callback - ERROR - Error processing callback for node 69c4c343590270ad7b6d9f2f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49594 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:26:54,880 - lava_callback - INFO - Received callback for node 69c4c394590270ad7b6d9f3e INFO: 172.21.0.1:54428 - "POST /node/69c4c394590270ad7b6d9f3e HTTP/1.0" 202 Accepted 2026-03-28 05:26:54,885 - lava_callback - INFO - Starting async job submit for node 69c4c394590270ad7b6d9f3e 2026-03-28 05:26:54,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:26:54,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c394590270ad7b6d9f3e HTTP/1.1" 200 None 2026-03-28 05:26:54,901 - lava_callback - DEBUG - Processing node 69c4c394590270ad7b6d9f3e: baseline-arm64-clabbe 2026-03-28 05:26:54,901 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c394590270ad7b6d9f3e 2026-03-28 05:26:54,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:26:54,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c394590270ad7b6d9f3e HTTP/1.1" 200 None 2026-03-28 05:26:54,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:26:54,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:26:54,931 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:26:54,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c394590270ad7b6d9f3e HTTP/1.1" 200 None 2026-03-28 05:26:54,957 - lava_callback - ERROR - Error processing callback for node 69c4c394590270ad7b6d9f3e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:54440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56148 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45190 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:28:14,957 - lava_callback - INFO - Received callback for node 69c4c3e5590270ad7b6d9f4a INFO: 172.21.0.1:44944 - "POST /node/69c4c3e5590270ad7b6d9f4a HTTP/1.0" 202 Accepted 2026-03-28 05:28:14,963 - lava_callback - INFO - Starting async job submit for node 69c4c3e5590270ad7b6d9f4a 2026-03-28 05:28:14,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:28:14,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c3e5590270ad7b6d9f4a HTTP/1.1" 200 None 2026-03-28 05:28:14,984 - lava_callback - DEBUG - Processing node 69c4c3e5590270ad7b6d9f4a: baseline-arm64-clabbe 2026-03-28 05:28:14,985 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c3e5590270ad7b6d9f4a 2026-03-28 05:28:14,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:28:14,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c3e5590270ad7b6d9f4a HTTP/1.1" 200 None 2026-03-28 05:28:15,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:28:15,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:28:15,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:28:15,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c3e5590270ad7b6d9f4a HTTP/1.1" 200 None 2026-03-28 05:28:15,066 - lava_callback - ERROR - Error processing callback for node 69c4c3e5590270ad7b6d9f4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44204 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47798 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56732 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:29:34,950 - lava_callback - INFO - Received callback for node 69c4c435590270ad7b6d9f53 INFO: 172.21.0.1:60812 - "POST /node/69c4c435590270ad7b6d9f53 HTTP/1.0" 202 Accepted 2026-03-28 05:29:34,955 - lava_callback - INFO - Starting async job submit for node 69c4c435590270ad7b6d9f53 2026-03-28 05:29:34,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:29:34,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c435590270ad7b6d9f53 HTTP/1.1" 200 None 2026-03-28 05:29:34,971 - lava_callback - DEBUG - Processing node 69c4c435590270ad7b6d9f53: baseline-arm64-clabbe 2026-03-28 05:29:34,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c435590270ad7b6d9f53 2026-03-28 05:29:34,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:29:34,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c435590270ad7b6d9f53 HTTP/1.1" 200 None 2026-03-28 05:29:34,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:29:34,998 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c40ad9944faaf0e7f5e41d HTTP/1.1" 200 None 2026-03-28 05:29:35,000 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:29:35,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c435590270ad7b6d9f53 HTTP/1.1" 200 None 2026-03-28 05:29:35,028 - lava_callback - ERROR - Error processing callback for node 69c4c435590270ad7b6d9f53: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44050 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:30:55,113 - lava_callback - INFO - Received callback for node 69c4c478590270ad7b6d9f5e INFO: 172.21.0.1:48794 - "POST /node/69c4c478590270ad7b6d9f5e HTTP/1.0" 202 Accepted 2026-03-28 05:30:55,119 - lava_callback - INFO - Starting async job submit for node 69c4c478590270ad7b6d9f5e 2026-03-28 05:30:55,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:30:55,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c478590270ad7b6d9f5e HTTP/1.1" 200 None 2026-03-28 05:30:55,140 - lava_callback - DEBUG - Processing node 69c4c478590270ad7b6d9f5e: baseline-arm-clabbe 2026-03-28 05:30:55,140 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c478590270ad7b6d9f5e 2026-03-28 05:30:55,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:30:55,153 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c478590270ad7b6d9f5e HTTP/1.1" 200 None 2026-03-28 05:30:55,155 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:30:55,167 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:30:55,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:30:55,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c478590270ad7b6d9f5e HTTP/1.1" 200 None 2026-03-28 05:30:55,195 - lava_callback - ERROR - Error processing callback for node 69c4c478590270ad7b6d9f5e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57964 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:31:15,005 - lava_callback - INFO - Received callback for node 69c4c49f590270ad7b6d9f75 INFO: 172.21.0.1:39198 - "POST /node/69c4c49f590270ad7b6d9f75 HTTP/1.0" 202 Accepted 2026-03-28 05:31:15,011 - lava_callback - INFO - Starting async job submit for node 69c4c49f590270ad7b6d9f75 2026-03-28 05:31:15,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:15,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c49f590270ad7b6d9f75 HTTP/1.1" 200 None 2026-03-28 05:31:15,027 - lava_callback - DEBUG - Processing node 69c4c49f590270ad7b6d9f75: baseline-arm-clabbe 2026-03-28 05:31:15,028 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c49f590270ad7b6d9f75 2026-03-28 05:31:15,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:15,041 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c49f590270ad7b6d9f75 HTTP/1.1" 200 None 2026-03-28 05:31:15,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:15,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:31:15,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:15,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c49f590270ad7b6d9f75 HTTP/1.1" 200 None 2026-03-28 05:31:15,082 - lava_callback - ERROR - Error processing callback for node 69c4c49f590270ad7b6d9f75: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39206 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49372 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:31:55,068 - lava_callback - INFO - Received callback for node 69c4c4c4590270ad7b6d9f84 INFO: 172.21.0.1:39086 - "POST /node/69c4c4c4590270ad7b6d9f84 HTTP/1.0" 202 Accepted 2026-03-28 05:31:55,074 - lava_callback - INFO - Starting async job submit for node 69c4c4c4590270ad7b6d9f84 2026-03-28 05:31:55,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:55,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c4c4590270ad7b6d9f84 HTTP/1.1" 200 None 2026-03-28 05:31:55,090 - lava_callback - DEBUG - Processing node 69c4c4c4590270ad7b6d9f84: baseline-arm-clabbe 2026-03-28 05:31:55,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c4c4590270ad7b6d9f84 2026-03-28 05:31:55,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:55,114 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c4c4590270ad7b6d9f84 HTTP/1.1" 200 None 2026-03-28 05:31:55,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:55,128 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:31:55,130 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:31:55,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c4c4590270ad7b6d9f84 HTTP/1.1" 200 None 2026-03-28 05:31:55,151 - lava_callback - ERROR - Error processing callback for node 69c4c4c4590270ad7b6d9f84: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58770 - "GET / HTTP/1.0" 200 OK 2026-03-28 05:32:35,083 - lava_callback - INFO - Received callback for node 69c4c4ea590270ad7b6d9f95 INFO: 172.21.0.1:52262 - "POST /node/69c4c4ea590270ad7b6d9f95 HTTP/1.0" 202 Accepted 2026-03-28 05:32:35,088 - lava_callback - INFO - Starting async job submit for node 69c4c4ea590270ad7b6d9f95 2026-03-28 05:32:35,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:32:35,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c4ea590270ad7b6d9f95 HTTP/1.1" 200 None 2026-03-28 05:32:35,105 - lava_callback - DEBUG - Processing node 69c4c4ea590270ad7b6d9f95: baseline-arm-clabbe 2026-03-28 05:32:35,105 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4c4ea590270ad7b6d9f95 2026-03-28 05:32:35,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:32:35,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4c4ea590270ad7b6d9f95 HTTP/1.1" 200 None 2026-03-28 05:32:35,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:32:35,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c167842b2b51e9ad977943 HTTP/1.1" 200 None 2026-03-28 05:32:35,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 05:32:35,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4c4ea590270ad7b6d9f95 HTTP/1.1" 200 None 2026-03-28 05:32:35,162 - lava_callback - ERROR - Error processing callback for node 69c4c4ea590270ad7b6d9f95: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53590 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57898 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56300 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60830 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:51412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52422 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49838 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45812 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33344 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47212 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42536 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37694 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60504 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60700 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41170 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56050 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56062 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36598 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40616 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42666 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60656 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44808 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45056 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46960 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58186 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52366 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52368 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41610 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48394 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48408 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50064 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40110 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60328 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57266 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53496 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41328 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41340 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33778 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34384 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36062 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32854 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32866 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34322 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49100 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53994 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49892 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37436 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41470 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60950 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44748 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51670 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51686 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33050 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43154 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54192 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58940 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48688 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42220 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52600 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54020 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59832 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49242 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47040 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41922 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:16:44,080 - lava_callback - INFO - Received callback for node 69c4cf49590270ad7b6d9ffc INFO: 172.21.0.1:41926 - "POST /node/69c4cf49590270ad7b6d9ffc HTTP/1.0" 202 Accepted 2026-03-28 06:16:44,086 - lava_callback - INFO - Starting async job submit for node 69c4cf49590270ad7b6d9ffc 2026-03-28 06:16:44,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:16:44,124 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cf49590270ad7b6d9ffc HTTP/1.1" 200 None 2026-03-28 06:16:44,124 - lava_callback - DEBUG - Processing node 69c4cf49590270ad7b6d9ffc: baseline-arm64-baylibre 2026-03-28 06:16:44,125 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4cf49590270ad7b6d9ffc 2026-03-28 06:16:44,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:16:44,142 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cf49590270ad7b6d9ffc HTTP/1.1" 200 None 2026-03-28 06:16:44,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:16:44,157 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:16:44,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:16:44,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4cf49590270ad7b6d9ffc HTTP/1.1" 200 None 2026-03-28 06:16:44,182 - lava_callback - ERROR - Error processing callback for node 69c4cf49590270ad7b6d9ffc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41934 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43812 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48580 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44892 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:18:24,283 - lava_callback - INFO - Received callback for node 69c4cf9d590270ad7b6da00c INFO: 172.21.0.1:40484 - "POST /node/69c4cf9d590270ad7b6da00c HTTP/1.0" 202 Accepted 2026-03-28 06:18:24,288 - lava_callback - INFO - Starting async job submit for node 69c4cf9d590270ad7b6da00c 2026-03-28 06:18:24,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:18:24,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cf9d590270ad7b6da00c HTTP/1.1" 200 None 2026-03-28 06:18:24,311 - lava_callback - DEBUG - Processing node 69c4cf9d590270ad7b6da00c: baseline-arm64-baylibre 2026-03-28 06:18:24,312 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4cf9d590270ad7b6da00c 2026-03-28 06:18:24,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:18:24,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cf9d590270ad7b6da00c HTTP/1.1" 200 None 2026-03-28 06:18:24,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:18:24,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:18:24,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:18:24,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4cf9d590270ad7b6da00c HTTP/1.1" 200 None 2026-03-28 06:18:24,379 - lava_callback - ERROR - Error processing callback for node 69c4cf9d590270ad7b6da00c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47848 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50232 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:19:44,182 - lava_callback - INFO - Received callback for node 69c4cff1590270ad7b6da018 INFO: 172.21.0.1:50236 - "POST /node/69c4cff1590270ad7b6da018 HTTP/1.0" 202 Accepted 2026-03-28 06:19:44,188 - lava_callback - INFO - Starting async job submit for node 69c4cff1590270ad7b6da018 2026-03-28 06:19:44,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:19:44,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cff1590270ad7b6da018 HTTP/1.1" 200 None 2026-03-28 06:19:44,212 - lava_callback - DEBUG - Processing node 69c4cff1590270ad7b6da018: baseline-arm64-baylibre 2026-03-28 06:19:44,212 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4cff1590270ad7b6da018 2026-03-28 06:19:44,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:19:44,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4cff1590270ad7b6da018 HTTP/1.1" 200 None 2026-03-28 06:19:44,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:19:44,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:19:44,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:19:44,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4cff1590270ad7b6da018 HTTP/1.1" 200 None 2026-03-28 06:19:44,298 - lava_callback - ERROR - Error processing callback for node 69c4cff1590270ad7b6da018: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53378 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35480 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:21:04,306 - lava_callback - INFO - Received callback for node 69c4d043590270ad7b6da02a INFO: 172.21.0.1:39812 - "POST /node/69c4d043590270ad7b6da02a HTTP/1.0" 202 Accepted 2026-03-28 06:21:04,312 - lava_callback - INFO - Starting async job submit for node 69c4d043590270ad7b6da02a 2026-03-28 06:21:04,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:21:04,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d043590270ad7b6da02a HTTP/1.1" 200 None 2026-03-28 06:21:04,328 - lava_callback - DEBUG - Processing node 69c4d043590270ad7b6da02a: baseline-arm64-baylibre 2026-03-28 06:21:04,328 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d043590270ad7b6da02a 2026-03-28 06:21:04,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:21:04,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d043590270ad7b6da02a HTTP/1.1" 200 None 2026-03-28 06:21:04,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:21:04,357 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:21:04,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:21:04,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d043590270ad7b6da02a HTTP/1.1" 200 None 2026-03-28 06:21:04,388 - lava_callback - ERROR - Error processing callback for node 69c4d043590270ad7b6da02a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54734 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41302 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41304 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:22:24,451 - lava_callback - INFO - Received callback for node 69c4d093590270ad7b6da038 INFO: 172.21.0.1:49332 - "POST /node/69c4d093590270ad7b6da038 HTTP/1.0" 202 Accepted 2026-03-28 06:22:24,456 - lava_callback - INFO - Starting async job submit for node 69c4d093590270ad7b6da038 2026-03-28 06:22:24,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:22:24,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d093590270ad7b6da038 HTTP/1.1" 200 None 2026-03-28 06:22:24,490 - lava_callback - DEBUG - Processing node 69c4d093590270ad7b6da038: baseline-arm64-baylibre 2026-03-28 06:22:24,490 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d093590270ad7b6da038 2026-03-28 06:22:24,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:22:24,504 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d093590270ad7b6da038 HTTP/1.1" 200 None 2026-03-28 06:22:24,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:22:24,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:22:24,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:22:24,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d093590270ad7b6da038 HTTP/1.1" 200 None 2026-03-28 06:22:24,543 - lava_callback - ERROR - Error processing callback for node 69c4d093590270ad7b6da038: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43636 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47638 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:23:44,439 - lava_callback - INFO - Received callback for node 69c4d0e6590270ad7b6da041 INFO: 172.21.0.1:47648 - "POST /node/69c4d0e6590270ad7b6da041 HTTP/1.0" 202 Accepted 2026-03-28 06:23:44,445 - lava_callback - INFO - Starting async job submit for node 69c4d0e6590270ad7b6da041 2026-03-28 06:23:44,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:23:44,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d0e6590270ad7b6da041 HTTP/1.1" 200 None 2026-03-28 06:23:44,461 - lava_callback - DEBUG - Processing node 69c4d0e6590270ad7b6da041: baseline-arm64-baylibre 2026-03-28 06:23:44,462 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d0e6590270ad7b6da041 2026-03-28 06:23:44,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:23:44,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d0e6590270ad7b6da041 HTTP/1.1" 200 None 2026-03-28 06:23:44,495 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:23:44,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:23:44,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:23:44,552 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d0e6590270ad7b6da041 HTTP/1.1" 200 None 2026-03-28 06:23:44,553 - lava_callback - ERROR - Error processing callback for node 69c4d0e6590270ad7b6da041: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47652 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42528 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:24:21,898 - lava_callback - INFO - Received callback for node 69c72d0229015d0f95992382 INFO: 172.21.0.1:41078 - "POST /node/69c72d0229015d0f95992382 HTTP/1.0" 202 Accepted 2026-03-28 06:24:21,905 - lava_callback - INFO - Starting async job submit for node 69c72d0229015d0f95992382 2026-03-28 06:24:21,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:24:21,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0229015d0f95992382 HTTP/1.1" 200 None 2026-03-28 06:24:21,959 - lava_callback - DEBUG - Processing node 69c72d0229015d0f95992382: baseline-arm64-broonie 2026-03-28 06:24:22,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:24:22,328 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:24:22,330 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0229015d0f95992382/log.txt.gz 2026-03-28 06:24:22,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:24:22,691 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:24:22,693 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0229015d0f95992382/lava-logs.yaml 2026-03-28 06:24:22,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:24:22,854 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:24:22,856 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0229015d0f95992382/lava_callback.json.gz 2026-03-28 06:24:22,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:24:22,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0229015d0f95992382 HTTP/1.1" 200 None 2026-03-28 06:24:22,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:24:22,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 06:24:22,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:24:22,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0229015d0f95992382 HTTP/1.1" 200 None 2026-03-28 06:24:22,949 - lava_callback - INFO - Completed processing callback for node 69c72d0229015d0f95992382 INFO: 172.21.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48648 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:24:36,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:24:36,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:54644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40662 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:25:04,520 - lava_callback - INFO - Received callback for node 69c4d136590270ad7b6da04f INFO: 172.21.0.1:57682 - "POST /node/69c4d136590270ad7b6da04f HTTP/1.0" 202 Accepted 2026-03-28 06:25:04,525 - lava_callback - INFO - Starting async job submit for node 69c4d136590270ad7b6da04f 2026-03-28 06:25:04,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:25:04,541 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d136590270ad7b6da04f HTTP/1.1" 200 None 2026-03-28 06:25:04,541 - lava_callback - DEBUG - Processing node 69c4d136590270ad7b6da04f: baseline-arm64-baylibre 2026-03-28 06:25:04,542 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d136590270ad7b6da04f 2026-03-28 06:25:04,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:25:04,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d136590270ad7b6da04f HTTP/1.1" 200 None 2026-03-28 06:25:04,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:25:04,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:25:04,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:25:04,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d136590270ad7b6da04f HTTP/1.1" 200 None 2026-03-28 06:25:04,594 - lava_callback - ERROR - Error processing callback for node 69c4d136590270ad7b6da04f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57690 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42392 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60676 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60692 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:26:24,533 - lava_callback - INFO - Received callback for node 69c4d183590270ad7b6da05c INFO: 172.21.0.1:53540 - "POST /node/69c4d183590270ad7b6da05c HTTP/1.0" 202 Accepted 2026-03-28 06:26:24,539 - lava_callback - INFO - Starting async job submit for node 69c4d183590270ad7b6da05c 2026-03-28 06:26:24,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:26:24,554 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d183590270ad7b6da05c HTTP/1.1" 200 None 2026-03-28 06:26:24,555 - lava_callback - DEBUG - Processing node 69c4d183590270ad7b6da05c: baseline-arm64-baylibre 2026-03-28 06:26:24,556 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d183590270ad7b6da05c 2026-03-28 06:26:24,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:26:24,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d183590270ad7b6da05c HTTP/1.1" 200 None 2026-03-28 06:26:24,571 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:26:24,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:26:24,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:26:24,611 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d183590270ad7b6da05c HTTP/1.1" 200 None 2026-03-28 06:26:24,612 - lava_callback - ERROR - Error processing callback for node 69c4d183590270ad7b6da05c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53546 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39602 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46674 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59376 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33458 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56382 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:27:44,628 - lava_callback - INFO - Received callback for node 69c4d1d1590270ad7b6da065 INFO: 172.21.0.1:56390 - "POST /node/69c4d1d1590270ad7b6da065 HTTP/1.0" 202 Accepted 2026-03-28 06:27:44,634 - lava_callback - INFO - Starting async job submit for node 69c4d1d1590270ad7b6da065 2026-03-28 06:27:44,635 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:27:44,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d1d1590270ad7b6da065 HTTP/1.1" 200 None 2026-03-28 06:27:44,651 - lava_callback - DEBUG - Processing node 69c4d1d1590270ad7b6da065: baseline-arm64-baylibre 2026-03-28 06:27:44,651 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d1d1590270ad7b6da065 2026-03-28 06:27:44,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:27:44,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d1d1590270ad7b6da065 HTTP/1.1" 200 None 2026-03-28 06:27:44,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:27:44,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:27:44,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:27:44,710 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d1d1590270ad7b6da065 HTTP/1.1" 200 None 2026-03-28 06:27:44,716 - lava_callback - ERROR - Error processing callback for node 69c4d1d1590270ad7b6da065: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56402 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36852 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54424 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60248 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:29:04,712 - lava_callback - INFO - Received callback for node 69c4d220590270ad7b6da06f INFO: 172.21.0.1:48692 - "POST /node/69c4d220590270ad7b6da06f HTTP/1.0" 202 Accepted 2026-03-28 06:29:04,721 - lava_callback - INFO - Starting async job submit for node 69c4d220590270ad7b6da06f 2026-03-28 06:29:04,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:29:04,759 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d220590270ad7b6da06f HTTP/1.1" 200 None 2026-03-28 06:29:04,760 - lava_callback - DEBUG - Processing node 69c4d220590270ad7b6da06f: baseline-arm64-baylibre 2026-03-28 06:29:04,760 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d220590270ad7b6da06f 2026-03-28 06:29:04,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:29:04,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d220590270ad7b6da06f HTTP/1.1" 200 None 2026-03-28 06:29:04,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:29:04,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:29:04,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:29:04,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d220590270ad7b6da06f HTTP/1.1" 200 None 2026-03-28 06:29:04,849 - lava_callback - ERROR - Error processing callback for node 69c4d220590270ad7b6da06f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45738 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45750 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36030 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53308 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:30:24,742 - lava_callback - INFO - Received callback for node 69c4d26e590270ad7b6da07c INFO: 172.21.0.1:40508 - "POST /node/69c4d26e590270ad7b6da07c HTTP/1.0" 202 Accepted 2026-03-28 06:30:24,748 - lava_callback - INFO - Starting async job submit for node 69c4d26e590270ad7b6da07c 2026-03-28 06:30:24,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:30:24,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d26e590270ad7b6da07c HTTP/1.1" 200 None 2026-03-28 06:30:24,764 - lava_callback - DEBUG - Processing node 69c4d26e590270ad7b6da07c: baseline-arm64-baylibre 2026-03-28 06:30:24,764 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d26e590270ad7b6da07c 2026-03-28 06:30:24,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:30:24,824 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d26e590270ad7b6da07c HTTP/1.1" 200 None 2026-03-28 06:30:24,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:30:24,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:30:24,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:30:24,861 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d26e590270ad7b6da07c HTTP/1.1" 200 None 2026-03-28 06:30:24,862 - lava_callback - ERROR - Error processing callback for node 69c4d26e590270ad7b6da07c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40516 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47724 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56210 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44874 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:31:44,880 - lava_callback - INFO - Received callback for node 69c4d2bd590270ad7b6da086 INFO: 172.21.0.1:44876 - "POST /node/69c4d2bd590270ad7b6da086 HTTP/1.0" 202 Accepted 2026-03-28 06:31:44,885 - lava_callback - INFO - Starting async job submit for node 69c4d2bd590270ad7b6da086 2026-03-28 06:31:44,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:31:44,911 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d2bd590270ad7b6da086 HTTP/1.1" 200 None 2026-03-28 06:31:44,912 - lava_callback - DEBUG - Processing node 69c4d2bd590270ad7b6da086: baseline-arm64-baylibre 2026-03-28 06:31:44,912 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d2bd590270ad7b6da086 2026-03-28 06:31:44,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:31:44,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d2bd590270ad7b6da086 HTTP/1.1" 200 None 2026-03-28 06:31:44,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:31:44,952 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:31:44,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:31:44,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d2bd590270ad7b6da086 HTTP/1.1" 200 None 2026-03-28 06:31:44,977 - lava_callback - ERROR - Error processing callback for node 69c4d2bd590270ad7b6da086: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:60886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57102 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45532 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50776 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54612 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:32:57,612 - lava_callback - INFO - Received callback for node 69c4d312590270ad7b6da091 INFO: 172.21.0.1:54616 - "POST /node/69c4d312590270ad7b6da091 HTTP/1.0" 202 Accepted 2026-03-28 06:32:57,617 - lava_callback - INFO - Starting async job submit for node 69c4d312590270ad7b6da091 2026-03-28 06:32:57,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:32:57,640 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d312590270ad7b6da091 HTTP/1.1" 200 None 2026-03-28 06:32:57,641 - lava_callback - DEBUG - Processing node 69c4d312590270ad7b6da091: baseline-arm64-clabbe 2026-03-28 06:32:57,641 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d312590270ad7b6da091 2026-03-28 06:32:57,641 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:32:57,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d312590270ad7b6da091 HTTP/1.1" 200 None 2026-03-28 06:32:57,656 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:32:57,669 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:32:57,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:32:57,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d312590270ad7b6da091 HTTP/1.1" 200 None 2026-03-28 06:32:57,701 - lava_callback - ERROR - Error processing callback for node 69c4d312590270ad7b6da091: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37300 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41734 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45068 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54310 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42418 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:34:17,657 - lava_callback - INFO - Received callback for node 69c4d360590270ad7b6da09d INFO: 172.21.0.1:42428 - "POST /node/69c4d360590270ad7b6da09d HTTP/1.0" 202 Accepted 2026-03-28 06:34:17,662 - lava_callback - INFO - Starting async job submit for node 69c4d360590270ad7b6da09d 2026-03-28 06:34:17,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:34:17,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d360590270ad7b6da09d HTTP/1.1" 200 None 2026-03-28 06:34:17,688 - lava_callback - DEBUG - Processing node 69c4d360590270ad7b6da09d: baseline-arm64-clabbe 2026-03-28 06:34:17,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d360590270ad7b6da09d 2026-03-28 06:34:17,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:34:17,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d360590270ad7b6da09d HTTP/1.1" 200 None 2026-03-28 06:34:17,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:34:17,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:34:17,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:34:17,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d360590270ad7b6da09d HTTP/1.1" 200 None 2026-03-28 06:34:17,801 - lava_callback - ERROR - Error processing callback for node 69c4d360590270ad7b6da09d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57342 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36768 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:35:37,704 - lava_callback - INFO - Received callback for node 69c4d3b0590270ad7b6da0a3 INFO: 172.21.0.1:36774 - "POST /node/69c4d3b0590270ad7b6da0a3 HTTP/1.0" 202 Accepted 2026-03-28 06:35:37,710 - lava_callback - INFO - Starting async job submit for node 69c4d3b0590270ad7b6da0a3 2026-03-28 06:35:37,711 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:35:37,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d3b0590270ad7b6da0a3 HTTP/1.1" 200 None 2026-03-28 06:35:37,749 - lava_callback - DEBUG - Processing node 69c4d3b0590270ad7b6da0a3: baseline-arm64-clabbe 2026-03-28 06:35:37,749 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d3b0590270ad7b6da0a3 2026-03-28 06:35:37,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:35:37,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d3b0590270ad7b6da0a3 HTTP/1.1" 200 None 2026-03-28 06:35:37,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:35:37,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:35:37,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:35:37,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d3b0590270ad7b6da0a3 HTTP/1.1" 200 None 2026-03-28 06:35:37,849 - lava_callback - ERROR - Error processing callback for node 69c4d3b0590270ad7b6da0a3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42280 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59006 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:36:57,793 - lava_callback - INFO - Received callback for node 69c4d3ff590270ad7b6da0ad INFO: 172.21.0.1:59020 - "POST /node/69c4d3ff590270ad7b6da0ad HTTP/1.0" 202 Accepted 2026-03-28 06:36:57,798 - lava_callback - INFO - Starting async job submit for node 69c4d3ff590270ad7b6da0ad 2026-03-28 06:36:57,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:36:57,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d3ff590270ad7b6da0ad HTTP/1.1" 200 None 2026-03-28 06:36:57,816 - lava_callback - DEBUG - Processing node 69c4d3ff590270ad7b6da0ad: baseline-arm64-clabbe 2026-03-28 06:36:57,816 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d3ff590270ad7b6da0ad 2026-03-28 06:36:57,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:36:57,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d3ff590270ad7b6da0ad HTTP/1.1" 200 None 2026-03-28 06:36:57,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:36:57,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:36:57,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:36:57,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d3ff590270ad7b6da0ad HTTP/1.1" 200 None 2026-03-28 06:36:57,874 - lava_callback - ERROR - Error processing callback for node 69c4d3ff590270ad7b6da0ad: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43912 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41522 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:38:17,842 - lava_callback - INFO - Received callback for node 69c4d44f590270ad7b6da0ba INFO: 172.21.0.1:41524 - "POST /node/69c4d44f590270ad7b6da0ba HTTP/1.0" 202 Accepted 2026-03-28 06:38:17,848 - lava_callback - INFO - Starting async job submit for node 69c4d44f590270ad7b6da0ba 2026-03-28 06:38:17,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:38:17,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d44f590270ad7b6da0ba HTTP/1.1" 200 None 2026-03-28 06:38:17,864 - lava_callback - DEBUG - Processing node 69c4d44f590270ad7b6da0ba: baseline-arm64-clabbe 2026-03-28 06:38:17,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d44f590270ad7b6da0ba 2026-03-28 06:38:17,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:38:17,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d44f590270ad7b6da0ba HTTP/1.1" 200 None 2026-03-28 06:38:17,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:38:17,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:38:17,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:38:17,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d44f590270ad7b6da0ba HTTP/1.1" 200 None 2026-03-28 06:38:17,936 - lava_callback - ERROR - Error processing callback for node 69c4d44f590270ad7b6da0ba: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53416 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58462 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:39:37,861 - lava_callback - INFO - Received callback for node 69c4d49e590270ad7b6da0c7 INFO: 172.21.0.1:58466 - "POST /node/69c4d49e590270ad7b6da0c7 HTTP/1.0" 202 Accepted 2026-03-28 06:39:37,866 - lava_callback - INFO - Starting async job submit for node 69c4d49e590270ad7b6da0c7 2026-03-28 06:39:37,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:39:37,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d49e590270ad7b6da0c7 HTTP/1.1" 200 None 2026-03-28 06:39:37,882 - lava_callback - DEBUG - Processing node 69c4d49e590270ad7b6da0c7: baseline-arm64-clabbe 2026-03-28 06:39:37,883 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d49e590270ad7b6da0c7 2026-03-28 06:39:37,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:39:37,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d49e590270ad7b6da0c7 HTTP/1.1" 200 None 2026-03-28 06:39:37,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:39:37,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1679b2b2b51e9ad977955 HTTP/1.1" 200 None 2026-03-28 06:39:37,914 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:39:37,936 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d49e590270ad7b6da0c7 HTTP/1.1" 200 None 2026-03-28 06:39:37,937 - lava_callback - ERROR - Error processing callback for node 69c4d49e590270ad7b6da0c7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57572 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37504 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35388 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40794 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:40:45,308 - lava_callback - INFO - Received callback for node 69c4d4da590270ad7b6da0d4 INFO: 172.21.0.1:40800 - "POST /node/69c4d4da590270ad7b6da0d4 HTTP/1.0" 202 Accepted 2026-03-28 06:40:45,313 - lava_callback - INFO - Starting async job submit for node 69c4d4da590270ad7b6da0d4 2026-03-28 06:40:45,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:40:45,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d4da590270ad7b6da0d4 HTTP/1.1" 200 None 2026-03-28 06:40:45,339 - lava_callback - DEBUG - Processing node 69c4d4da590270ad7b6da0d4: baseline-arm-baylibre 2026-03-28 06:40:45,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d4da590270ad7b6da0d4 2026-03-28 06:40:45,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:40:45,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d4da590270ad7b6da0d4 HTTP/1.1" 200 None 2026-03-28 06:40:45,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:40:45,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:40:45,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:40:45,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d4da590270ad7b6da0d4 HTTP/1.1" 200 None 2026-03-28 06:40:45,396 - lava_callback - ERROR - Error processing callback for node 69c4d4da590270ad7b6da0d4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51628 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:41:05,291 - lava_callback - INFO - Received callback for node 69c4d4ff590270ad7b6da0e2 INFO: 172.21.0.1:52162 - "POST /node/69c4d4ff590270ad7b6da0e2 HTTP/1.0" 202 Accepted 2026-03-28 06:41:05,296 - lava_callback - INFO - Starting async job submit for node 69c4d4ff590270ad7b6da0e2 2026-03-28 06:41:05,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:05,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d4ff590270ad7b6da0e2 HTTP/1.1" 200 None 2026-03-28 06:41:05,335 - lava_callback - DEBUG - Processing node 69c4d4ff590270ad7b6da0e2: baseline-arm-baylibre 2026-03-28 06:41:05,335 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d4ff590270ad7b6da0e2 2026-03-28 06:41:05,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:05,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d4ff590270ad7b6da0e2 HTTP/1.1" 200 None 2026-03-28 06:41:05,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:05,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:41:05,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:05,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d4ff590270ad7b6da0e2 HTTP/1.1" 200 None 2026-03-28 06:41:05,389 - lava_callback - ERROR - Error processing callback for node 69c4d4ff590270ad7b6da0e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34430 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:41:45,320 - lava_callback - INFO - Received callback for node 69c4d524590270ad7b6da122 INFO: 172.21.0.1:34446 - "POST /node/69c4d524590270ad7b6da122 HTTP/1.0" 202 Accepted 2026-03-28 06:41:45,325 - lava_callback - INFO - Starting async job submit for node 69c4d524590270ad7b6da122 2026-03-28 06:41:45,326 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:45,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d524590270ad7b6da122 HTTP/1.1" 200 None 2026-03-28 06:41:45,340 - lava_callback - DEBUG - Processing node 69c4d524590270ad7b6da122: baseline-arm-baylibre 2026-03-28 06:41:45,341 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d524590270ad7b6da122 2026-03-28 06:41:45,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:45,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d524590270ad7b6da122 HTTP/1.1" 200 None 2026-03-28 06:41:45,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:45,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:41:45,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:41:45,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d524590270ad7b6da122 HTTP/1.1" 200 None 2026-03-28 06:41:45,393 - lava_callback - ERROR - Error processing callback for node 69c4d524590270ad7b6da122: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34450 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47970 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57792 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60252 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36672 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34136 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44662 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45274 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34534 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58560 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35910 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56330 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45146 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45922 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55896 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:46:25,620 - lava_callback - INFO - Received callback for node 69c4d63c590270ad7b6da186 INFO: 172.21.0.1:57854 - "POST /node/69c4d63c590270ad7b6da186 HTTP/1.0" 202 Accepted 2026-03-28 06:46:25,626 - lava_callback - INFO - Starting async job submit for node 69c4d63c590270ad7b6da186 2026-03-28 06:46:25,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:46:25,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d63c590270ad7b6da186 HTTP/1.1" 200 None 2026-03-28 06:46:25,642 - lava_callback - DEBUG - Processing node 69c4d63c590270ad7b6da186: baseline-arm-baylibre 2026-03-28 06:46:25,642 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d63c590270ad7b6da186 2026-03-28 06:46:25,643 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:46:25,656 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d63c590270ad7b6da186 HTTP/1.1" 200 None 2026-03-28 06:46:25,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:46:25,671 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:46:25,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:46:25,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d63c590270ad7b6da186 HTTP/1.1" 200 None 2026-03-28 06:46:25,708 - lava_callback - ERROR - Error processing callback for node 69c4d63c590270ad7b6da186: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54782 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43130 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36158 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38440 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57692 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57698 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32992 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36052 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43786 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58292 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35886 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35900 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41666 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41452 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36170 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38896 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37820 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37834 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:51:18,350 - lava_callback - INFO - Received callback for node 69c4d762590270ad7b6da1f7 INFO: 172.21.0.1:37842 - "POST /node/69c4d762590270ad7b6da1f7 HTTP/1.0" 202 Accepted 2026-03-28 06:51:18,356 - lava_callback - INFO - Starting async job submit for node 69c4d762590270ad7b6da1f7 2026-03-28 06:51:18,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:18,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d762590270ad7b6da1f7 HTTP/1.1" 200 None 2026-03-28 06:51:18,371 - lava_callback - DEBUG - Processing node 69c4d762590270ad7b6da1f7: baseline-arm-clabbe 2026-03-28 06:51:18,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d762590270ad7b6da1f7 2026-03-28 06:51:18,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:18,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d762590270ad7b6da1f7 HTTP/1.1" 200 None 2026-03-28 06:51:18,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:18,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:51:18,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:18,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d762590270ad7b6da1f7 HTTP/1.1" 200 None 2026-03-28 06:51:18,435 - lava_callback - ERROR - Error processing callback for node 69c4d762590270ad7b6da1f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53714 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53720 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39918 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:51:58,371 - lava_callback - INFO - Received callback for node 69c4d788590270ad7b6da206 INFO: 172.21.0.1:39934 - "POST /node/69c4d788590270ad7b6da206 HTTP/1.0" 202 Accepted 2026-03-28 06:51:58,376 - lava_callback - INFO - Starting async job submit for node 69c4d788590270ad7b6da206 2026-03-28 06:51:58,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:58,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d788590270ad7b6da206 HTTP/1.1" 200 None 2026-03-28 06:51:58,392 - lava_callback - DEBUG - Processing node 69c4d788590270ad7b6da206: baseline-arm-clabbe 2026-03-28 06:51:58,392 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d788590270ad7b6da206 2026-03-28 06:51:58,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:58,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d788590270ad7b6da206 HTTP/1.1" 200 None 2026-03-28 06:51:58,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:58,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:51:58,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:51:58,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d788590270ad7b6da206 HTTP/1.1" 200 None 2026-03-28 06:51:58,467 - lava_callback - ERROR - Error processing callback for node 69c4d788590270ad7b6da206: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54044 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36590 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:52:38,468 - lava_callback - INFO - Received callback for node 69c4d7ad590270ad7b6da216 INFO: 172.21.0.1:36592 - "POST /node/69c4d7ad590270ad7b6da216 HTTP/1.0" 202 Accepted 2026-03-28 06:52:38,473 - lava_callback - INFO - Starting async job submit for node 69c4d7ad590270ad7b6da216 2026-03-28 06:52:38,474 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:52:38,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7ad590270ad7b6da216 HTTP/1.1" 200 None 2026-03-28 06:52:38,489 - lava_callback - DEBUG - Processing node 69c4d7ad590270ad7b6da216: baseline-arm-clabbe 2026-03-28 06:52:38,489 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d7ad590270ad7b6da216 2026-03-28 06:52:38,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:52:38,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7ad590270ad7b6da216 HTTP/1.1" 200 None 2026-03-28 06:52:38,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:52:38,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:52:38,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:52:38,559 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d7ad590270ad7b6da216 HTTP/1.1" 200 None 2026-03-28 06:52:38,561 - lava_callback - ERROR - Error processing callback for node 69c4d7ad590270ad7b6da216: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41216 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47424 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:53:18,414 - lava_callback - INFO - Received callback for node 69c4d7d3590270ad7b6da22f INFO: 172.21.0.1:47430 - "POST /node/69c4d7d3590270ad7b6da22f HTTP/1.0" 202 Accepted 2026-03-28 06:53:18,420 - lava_callback - INFO - Starting async job submit for node 69c4d7d3590270ad7b6da22f 2026-03-28 06:53:18,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:18,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7d3590270ad7b6da22f HTTP/1.1" 200 None 2026-03-28 06:53:18,436 - lava_callback - DEBUG - Processing node 69c4d7d3590270ad7b6da22f: baseline-arm-clabbe 2026-03-28 06:53:18,436 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d7d3590270ad7b6da22f 2026-03-28 06:53:18,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:18,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7d3590270ad7b6da22f HTTP/1.1" 200 None 2026-03-28 06:53:18,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:18,464 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:53:18,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:18,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d7d3590270ad7b6da22f HTTP/1.1" 200 None 2026-03-28 06:53:18,488 - lava_callback - ERROR - Error processing callback for node 69c4d7d3590270ad7b6da22f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43566 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:53:58,473 - lava_callback - INFO - Received callback for node 69c4d7f8590270ad7b6da239 INFO: 172.21.0.1:43580 - "POST /node/69c4d7f8590270ad7b6da239 HTTP/1.0" 202 Accepted 2026-03-28 06:53:58,479 - lava_callback - INFO - Starting async job submit for node 69c4d7f8590270ad7b6da239 2026-03-28 06:53:58,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:58,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7f8590270ad7b6da239 HTTP/1.1" 200 None 2026-03-28 06:53:58,500 - lava_callback - DEBUG - Processing node 69c4d7f8590270ad7b6da239: baseline-arm-clabbe 2026-03-28 06:53:58,500 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d7f8590270ad7b6da239 2026-03-28 06:53:58,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:58,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d7f8590270ad7b6da239 HTTP/1.1" 200 None 2026-03-28 06:53:58,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:58,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:53:58,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:53:58,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d7f8590270ad7b6da239 HTTP/1.1" 200 None 2026-03-28 06:53:58,560 - lava_callback - ERROR - Error processing callback for node 69c4d7f8590270ad7b6da239: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44790 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:54:38,522 - lava_callback - INFO - Received callback for node 69c4d81f590270ad7b6da249 INFO: 172.21.0.1:44800 - "POST /node/69c4d81f590270ad7b6da249 HTTP/1.0" 202 Accepted 2026-03-28 06:54:38,528 - lava_callback - INFO - Starting async job submit for node 69c4d81f590270ad7b6da249 2026-03-28 06:54:38,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:54:38,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d81f590270ad7b6da249 HTTP/1.1" 200 None 2026-03-28 06:54:38,544 - lava_callback - DEBUG - Processing node 69c4d81f590270ad7b6da249: baseline-arm-clabbe 2026-03-28 06:54:38,544 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d81f590270ad7b6da249 2026-03-28 06:54:38,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:54:38,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d81f590270ad7b6da249 HTTP/1.1" 200 None 2026-03-28 06:54:38,559 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:54:38,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b3b590270ad7b6d698b HTTP/1.1" 200 None 2026-03-28 06:54:38,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:54:38,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d81f590270ad7b6da249 HTTP/1.1" 200 None 2026-03-28 06:54:38,596 - lava_callback - ERROR - Error processing callback for node 69c4d81f590270ad7b6da249: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40626 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45138 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37108 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:55:58,593 - lava_callback - INFO - Received callback for node 69c4d86a590270ad7b6da270 INFO: 172.21.0.1:37116 - "POST /node/69c4d86a590270ad7b6da270 HTTP/1.0" 202 Accepted 2026-03-28 06:55:58,599 - lava_callback - INFO - Starting async job submit for node 69c4d86a590270ad7b6da270 2026-03-28 06:55:58,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:55:58,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d86a590270ad7b6da270 HTTP/1.1" 200 None 2026-03-28 06:55:58,615 - lava_callback - DEBUG - Processing node 69c4d86a590270ad7b6da270: baseline-arm-clabbe 2026-03-28 06:55:58,616 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d86a590270ad7b6da270 2026-03-28 06:55:58,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:55:58,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d86a590270ad7b6da270 HTTP/1.1" 200 None 2026-03-28 06:55:58,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:55:58,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:55:58,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:55:58,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d86a590270ad7b6da270 HTTP/1.1" 200 None 2026-03-28 06:55:58,670 - lava_callback - ERROR - Error processing callback for node 69c4d86a590270ad7b6da270: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49558 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37554 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:56:18,551 - lava_callback - INFO - Received callback for node 69c4d890590270ad7b6da27b INFO: 172.21.0.1:37560 - "POST /node/69c4d890590270ad7b6da27b HTTP/1.0" 202 Accepted 2026-03-28 06:56:18,557 - lava_callback - INFO - Starting async job submit for node 69c4d890590270ad7b6da27b 2026-03-28 06:56:18,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:18,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d890590270ad7b6da27b HTTP/1.1" 200 None 2026-03-28 06:56:18,581 - lava_callback - DEBUG - Processing node 69c4d890590270ad7b6da27b: baseline-arm-clabbe 2026-03-28 06:56:18,582 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d890590270ad7b6da27b 2026-03-28 06:56:18,582 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:18,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d890590270ad7b6da27b HTTP/1.1" 200 None 2026-03-28 06:56:18,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:18,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:56:18,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:18,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d890590270ad7b6da27b HTTP/1.1" 200 None 2026-03-28 06:56:18,643 - lava_callback - ERROR - Error processing callback for node 69c4d890590270ad7b6da27b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:55648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52188 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52190 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56688 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:56:58,577 - lava_callback - INFO - Received callback for node 69c4d8b5590270ad7b6da28c INFO: 172.21.0.1:56702 - "POST /node/69c4d8b5590270ad7b6da28c HTTP/1.0" 202 Accepted 2026-03-28 06:56:58,583 - lava_callback - INFO - Starting async job submit for node 69c4d8b5590270ad7b6da28c 2026-03-28 06:56:58,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:58,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d8b5590270ad7b6da28c HTTP/1.1" 200 None 2026-03-28 06:56:58,599 - lava_callback - DEBUG - Processing node 69c4d8b5590270ad7b6da28c: baseline-arm-clabbe 2026-03-28 06:56:58,600 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d8b5590270ad7b6da28c 2026-03-28 06:56:58,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:58,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d8b5590270ad7b6da28c HTTP/1.1" 200 None 2026-03-28 06:56:58,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:58,629 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:56:58,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:56:58,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d8b5590270ad7b6da28c HTTP/1.1" 200 None 2026-03-28 06:56:58,660 - lava_callback - ERROR - Error processing callback for node 69c4d8b5590270ad7b6da28c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:51632 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56646 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35820 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:57:38,601 - lava_callback - INFO - Received callback for node 69c4d8dc590270ad7b6da29e INFO: 172.21.0.1:35830 - "POST /node/69c4d8dc590270ad7b6da29e HTTP/1.0" 202 Accepted 2026-03-28 06:57:38,606 - lava_callback - INFO - Starting async job submit for node 69c4d8dc590270ad7b6da29e 2026-03-28 06:57:38,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:57:38,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d8dc590270ad7b6da29e HTTP/1.1" 200 None 2026-03-28 06:57:38,622 - lava_callback - DEBUG - Processing node 69c4d8dc590270ad7b6da29e: baseline-arm-clabbe 2026-03-28 06:57:38,622 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d8dc590270ad7b6da29e 2026-03-28 06:57:38,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:57:38,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d8dc590270ad7b6da29e HTTP/1.1" 200 None 2026-03-28 06:57:38,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:57:38,650 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:57:38,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:57:38,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d8dc590270ad7b6da29e HTTP/1.1" 200 None 2026-03-28 06:57:38,674 - lava_callback - ERROR - Error processing callback for node 69c4d8dc590270ad7b6da29e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46244 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38920 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35466 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50718 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:58:18,711 - lava_callback - INFO - Received callback for node 69c4d901590270ad7b6da2b5 INFO: 172.21.0.1:50730 - "POST /node/69c4d901590270ad7b6da2b5 HTTP/1.0" 202 Accepted 2026-03-28 06:58:18,716 - lava_callback - INFO - Starting async job submit for node 69c4d901590270ad7b6da2b5 2026-03-28 06:58:18,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:18,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d901590270ad7b6da2b5 HTTP/1.1" 200 None 2026-03-28 06:58:18,732 - lava_callback - DEBUG - Processing node 69c4d901590270ad7b6da2b5: baseline-arm-clabbe 2026-03-28 06:58:18,733 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d901590270ad7b6da2b5 2026-03-28 06:58:18,733 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:18,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d901590270ad7b6da2b5 HTTP/1.1" 200 None 2026-03-28 06:58:18,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:18,761 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:58:18,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:18,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d901590270ad7b6da2b5 HTTP/1.1" 200 None 2026-03-28 06:58:18,792 - lava_callback - ERROR - Error processing callback for node 69c4d901590270ad7b6da2b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45256 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43272 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59620 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:58:58,643 - lava_callback - INFO - Received callback for node 69c4d926590270ad7b6da2c2 INFO: 172.21.0.1:59636 - "POST /node/69c4d926590270ad7b6da2c2 HTTP/1.0" 202 Accepted 2026-03-28 06:58:58,648 - lava_callback - INFO - Starting async job submit for node 69c4d926590270ad7b6da2c2 2026-03-28 06:58:58,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:58,663 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d926590270ad7b6da2c2 HTTP/1.1" 200 None 2026-03-28 06:58:58,664 - lava_callback - DEBUG - Processing node 69c4d926590270ad7b6da2c2: baseline-arm-clabbe 2026-03-28 06:58:58,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d926590270ad7b6da2c2 2026-03-28 06:58:58,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:58,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d926590270ad7b6da2c2 HTTP/1.1" 200 None 2026-03-28 06:58:58,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:58,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:58:58,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:58:58,717 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d926590270ad7b6da2c2 HTTP/1.1" 200 None 2026-03-28 06:58:58,718 - lava_callback - ERROR - Error processing callback for node 69c4d926590270ad7b6da2c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:43998 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44550 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:59:38,784 - lava_callback - INFO - Received callback for node 69c4d94c590270ad7b6da2d5 INFO: 172.21.0.1:44564 - "POST /node/69c4d94c590270ad7b6da2d5 HTTP/1.0" 202 Accepted 2026-03-28 06:59:38,790 - lava_callback - INFO - Starting async job submit for node 69c4d94c590270ad7b6da2d5 2026-03-28 06:59:38,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:38,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d94c590270ad7b6da2d5 HTTP/1.1" 200 None 2026-03-28 06:59:38,806 - lava_callback - DEBUG - Processing node 69c4d94c590270ad7b6da2d5: baseline-arm-clabbe 2026-03-28 06:59:38,806 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d94c590270ad7b6da2d5 2026-03-28 06:59:38,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:38,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d94c590270ad7b6da2d5 HTTP/1.1" 200 None 2026-03-28 06:59:38,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:38,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 06:59:38,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:38,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d94c590270ad7b6da2d5 HTTP/1.1" 200 None 2026-03-28 06:59:38,858 - lava_callback - ERROR - Error processing callback for node 69c4d94c590270ad7b6da2d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56780 - "GET / HTTP/1.0" 200 OK 2026-03-28 06:59:42,942 - lava_callback - INFO - Received callback for node 69c72d0a29015d0f959923b5 INFO: 172.21.0.1:56782 - "POST /node/69c72d0a29015d0f959923b5 HTTP/1.0" 202 Accepted 2026-03-28 06:59:42,948 - lava_callback - INFO - Starting async job submit for node 69c72d0a29015d0f959923b5 2026-03-28 06:59:42,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:42,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0a29015d0f959923b5 HTTP/1.1" 200 None 2026-03-28 06:59:42,986 - lava_callback - DEBUG - Processing node 69c72d0a29015d0f959923b5: baseline-arm64-broonie 2026-03-28 06:59:43,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:59:43,384 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:59:43,386 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0a29015d0f959923b5/log.txt.gz 2026-03-28 06:59:43,549 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:59:43,790 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:59:43,792 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0a29015d0f959923b5/lava-logs.yaml 2026-03-28 06:59:43,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 06:59:43,927 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 06:59:43,929 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0a29015d0f959923b5/lava_callback.json.gz 2026-03-28 06:59:43,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:43,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0a29015d0f959923b5 HTTP/1.1" 200 None 2026-03-28 06:59:43,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:43,982 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 06:59:43,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 06:59:44,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0a29015d0f959923b5 HTTP/1.1" 200 None 2026-03-28 06:59:44,020 - lava_callback - INFO - Completed processing callback for node 69c72d0a29015d0f959923b5 INFO: 172.21.0.1:56784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39426 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:00:06,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:06,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:33592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:00:18,851 - lava_callback - INFO - Received callback for node 69c4d972590270ad7b6da2e6 INFO: 172.21.0.1:33608 - "POST /node/69c4d972590270ad7b6da2e6 HTTP/1.0" 202 Accepted 2026-03-28 07:00:18,857 - lava_callback - INFO - Starting async job submit for node 69c4d972590270ad7b6da2e6 2026-03-28 07:00:18,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:18,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d972590270ad7b6da2e6 HTTP/1.1" 200 None 2026-03-28 07:00:18,873 - lava_callback - DEBUG - Processing node 69c4d972590270ad7b6da2e6: baseline-arm-clabbe 2026-03-28 07:00:18,873 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d972590270ad7b6da2e6 2026-03-28 07:00:18,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:18,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d972590270ad7b6da2e6 HTTP/1.1" 200 None 2026-03-28 07:00:18,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:18,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 07:00:18,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:18,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d972590270ad7b6da2e6 HTTP/1.1" 200 None 2026-03-28 07:00:18,934 - lava_callback - ERROR - Error processing callback for node 69c4d972590270ad7b6da2e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41386 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:00:58,837 - lava_callback - INFO - Received callback for node 69c4d998590270ad7b6da303 INFO: 172.21.0.1:41396 - "POST /node/69c4d998590270ad7b6da303 HTTP/1.0" 202 Accepted 2026-03-28 07:00:58,843 - lava_callback - INFO - Starting async job submit for node 69c4d998590270ad7b6da303 2026-03-28 07:00:58,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:58,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d998590270ad7b6da303 HTTP/1.1" 200 None 2026-03-28 07:00:58,858 - lava_callback - DEBUG - Processing node 69c4d998590270ad7b6da303: baseline-arm-clabbe 2026-03-28 07:00:58,858 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d998590270ad7b6da303 2026-03-28 07:00:58,859 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:58,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d998590270ad7b6da303 HTTP/1.1" 200 None 2026-03-28 07:00:58,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:58,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d82b0efaf21d93103543 HTTP/1.1" 200 None 2026-03-28 07:00:58,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:00:58,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d998590270ad7b6da303 HTTP/1.1" 200 None 2026-03-28 07:00:58,917 - lava_callback - ERROR - Error processing callback for node 69c4d998590270ad7b6da303: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:58134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54238 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54250 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38114 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:01:38,850 - lava_callback - INFO - Received callback for node 69c4d9bf590270ad7b6da30d INFO: 172.21.0.1:38124 - "POST /node/69c4d9bf590270ad7b6da30d HTTP/1.0" 202 Accepted 2026-03-28 07:01:38,856 - lava_callback - INFO - Starting async job submit for node 69c4d9bf590270ad7b6da30d 2026-03-28 07:01:38,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:01:38,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d9bf590270ad7b6da30d HTTP/1.1" 200 None 2026-03-28 07:01:38,892 - lava_callback - DEBUG - Processing node 69c4d9bf590270ad7b6da30d: baseline-arm-clabbe 2026-03-28 07:01:38,892 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d9bf590270ad7b6da30d 2026-03-28 07:01:38,893 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:01:38,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d9bf590270ad7b6da30d HTTP/1.1" 200 None 2026-03-28 07:01:38,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:01:38,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:01:38,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:01:38,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d9bf590270ad7b6da30d HTTP/1.1" 200 None 2026-03-28 07:01:38,981 - lava_callback - ERROR - Error processing callback for node 69c4d9bf590270ad7b6da30d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57794 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41564 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52362 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52372 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:02:18,876 - lava_callback - INFO - Received callback for node 69c4d9e5590270ad7b6da31c INFO: 172.21.0.1:52374 - "POST /node/69c4d9e5590270ad7b6da31c HTTP/1.0" 202 Accepted 2026-03-28 07:02:18,882 - lava_callback - INFO - Starting async job submit for node 69c4d9e5590270ad7b6da31c 2026-03-28 07:02:18,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:18,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d9e5590270ad7b6da31c HTTP/1.1" 200 None 2026-03-28 07:02:18,898 - lava_callback - DEBUG - Processing node 69c4d9e5590270ad7b6da31c: baseline-arm-clabbe 2026-03-28 07:02:18,898 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4d9e5590270ad7b6da31c 2026-03-28 07:02:18,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:18,912 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4d9e5590270ad7b6da31c HTTP/1.1" 200 None 2026-03-28 07:02:18,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:18,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:02:18,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:18,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4d9e5590270ad7b6da31c HTTP/1.1" 200 None 2026-03-28 07:02:18,952 - lava_callback - ERROR - Error processing callback for node 69c4d9e5590270ad7b6da31c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45874 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46748 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:02:38,864 - lava_callback - INFO - Received callback for node 69c4da0a590270ad7b6da336 INFO: 172.21.0.1:46764 - "POST /node/69c4da0a590270ad7b6da336 HTTP/1.0" 202 Accepted 2026-03-28 07:02:38,870 - lava_callback - INFO - Starting async job submit for node 69c4da0a590270ad7b6da336 2026-03-28 07:02:38,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:38,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da0a590270ad7b6da336 HTTP/1.1" 200 None 2026-03-28 07:02:38,886 - lava_callback - DEBUG - Processing node 69c4da0a590270ad7b6da336: baseline-arm-clabbe 2026-03-28 07:02:38,886 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4da0a590270ad7b6da336 2026-03-28 07:02:38,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:38,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da0a590270ad7b6da336 HTTP/1.1" 200 None 2026-03-28 07:02:38,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:38,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:02:38,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:02:38,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4da0a590270ad7b6da336 HTTP/1.1" 200 None 2026-03-28 07:02:38,939 - lava_callback - ERROR - Error processing callback for node 69c4da0a590270ad7b6da336: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33654 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50088 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37196 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:03:18,986 - lava_callback - INFO - Received callback for node 69c4da2f590270ad7b6da34d INFO: 172.21.0.1:37198 - "POST /node/69c4da2f590270ad7b6da34d HTTP/1.0" 202 Accepted 2026-03-28 07:03:18,992 - lava_callback - INFO - Starting async job submit for node 69c4da2f590270ad7b6da34d 2026-03-28 07:03:18,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:19,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da2f590270ad7b6da34d HTTP/1.1" 200 None 2026-03-28 07:03:19,007 - lava_callback - DEBUG - Processing node 69c4da2f590270ad7b6da34d: baseline-arm-clabbe 2026-03-28 07:03:19,008 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4da2f590270ad7b6da34d 2026-03-28 07:03:19,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:19,021 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da2f590270ad7b6da34d HTTP/1.1" 200 None 2026-03-28 07:03:19,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:19,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:03:19,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:19,058 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4da2f590270ad7b6da34d HTTP/1.1" 200 None 2026-03-28 07:03:19,059 - lava_callback - ERROR - Error processing callback for node 69c4da2f590270ad7b6da34d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50542 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57068 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:03:58,918 - lava_callback - INFO - Received callback for node 69c4da55590270ad7b6da35c INFO: 172.21.0.1:57072 - "POST /node/69c4da55590270ad7b6da35c HTTP/1.0" 202 Accepted 2026-03-28 07:03:58,924 - lava_callback - INFO - Starting async job submit for node 69c4da55590270ad7b6da35c 2026-03-28 07:03:58,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:58,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da55590270ad7b6da35c HTTP/1.1" 200 None 2026-03-28 07:03:58,940 - lava_callback - DEBUG - Processing node 69c4da55590270ad7b6da35c: baseline-arm-clabbe 2026-03-28 07:03:58,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4da55590270ad7b6da35c 2026-03-28 07:03:58,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:58,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da55590270ad7b6da35c HTTP/1.1" 200 None 2026-03-28 07:03:58,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:58,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:03:58,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:03:58,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4da55590270ad7b6da35c HTTP/1.1" 200 None 2026-03-28 07:03:58,992 - lava_callback - ERROR - Error processing callback for node 69c4da55590270ad7b6da35c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38234 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55672 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47514 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53990 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:04:39,013 - lava_callback - INFO - Received callback for node 69c4da7a590270ad7b6da36d INFO: 172.21.0.1:54002 - "POST /node/69c4da7a590270ad7b6da36d HTTP/1.0" 202 Accepted 2026-03-28 07:04:39,018 - lava_callback - INFO - Starting async job submit for node 69c4da7a590270ad7b6da36d 2026-03-28 07:04:39,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:04:39,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da7a590270ad7b6da36d HTTP/1.1" 200 None 2026-03-28 07:04:39,034 - lava_callback - DEBUG - Processing node 69c4da7a590270ad7b6da36d: baseline-arm-clabbe 2026-03-28 07:04:39,034 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4da7a590270ad7b6da36d 2026-03-28 07:04:39,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:04:39,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4da7a590270ad7b6da36d HTTP/1.1" 200 None 2026-03-28 07:04:39,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:04:39,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:04:39,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:04:39,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4da7a590270ad7b6da36d HTTP/1.1" 200 None 2026-03-28 07:04:39,121 - lava_callback - ERROR - Error processing callback for node 69c4da7a590270ad7b6da36d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48914 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44198 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54756 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:54758 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:05:19,002 - lava_callback - INFO - Received callback for node 69c4daa1590270ad7b6da37e INFO: 172.21.0.1:54764 - "POST /node/69c4daa1590270ad7b6da37e HTTP/1.0" 202 Accepted 2026-03-28 07:05:19,008 - lava_callback - INFO - Starting async job submit for node 69c4daa1590270ad7b6da37e 2026-03-28 07:05:19,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:19,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4daa1590270ad7b6da37e HTTP/1.1" 200 None 2026-03-28 07:05:19,025 - lava_callback - DEBUG - Processing node 69c4daa1590270ad7b6da37e: baseline-arm-clabbe 2026-03-28 07:05:19,025 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4daa1590270ad7b6da37e 2026-03-28 07:05:19,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:19,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4daa1590270ad7b6da37e HTTP/1.1" 200 None 2026-03-28 07:05:19,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:19,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:05:19,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:19,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4daa1590270ad7b6da37e HTTP/1.1" 200 None 2026-03-28 07:05:19,083 - lava_callback - ERROR - Error processing callback for node 69c4daa1590270ad7b6da37e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-03-28 07:05:20,103 - lava_callback - INFO - Received callback for node 69c72cf129015d0f95992350 INFO: 172.21.0.1:50738 - "POST /node/69c72cf129015d0f95992350 HTTP/1.0" 202 Accepted 2026-03-28 07:05:20,110 - lava_callback - INFO - Starting async job submit for node 69c72cf129015d0f95992350 2026-03-28 07:05:20,138 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:20,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf129015d0f95992350 HTTP/1.1" 200 None 2026-03-28 07:05:20,164 - lava_callback - DEBUG - Processing node 69c72cf129015d0f95992350: baseline-nfs-arm64 2026-03-28 07:05:20,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:05:20,969 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:05:20,971 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf129015d0f95992350/log.txt.gz 2026-03-28 07:05:21,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:05:21,464 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:05:21,467 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf129015d0f95992350/lava-logs.yaml 2026-03-28 07:05:21,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:05:21,654 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:05:21,656 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c72cf129015d0f95992350/lava_callback.json.gz 2026-03-28 07:05:21,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:21,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72cf129015d0f95992350 HTTP/1.1" 200 None 2026-03-28 07:05:21,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:21,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 07:05:21,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:21,735 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72cf129015d0f95992350 HTTP/1.1" 200 None 2026-03-28 07:05:21,824 - lava_callback - INFO - Completed processing callback for node 69c72cf129015d0f95992350 INFO: 172.21.0.1:50752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42048 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:05:36,769 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:36,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:52752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47464 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:05:59,018 - lava_callback - INFO - Received callback for node 69c4dac6590270ad7b6da38f INFO: 172.21.0.1:47480 - "POST /node/69c4dac6590270ad7b6da38f HTTP/1.0" 202 Accepted 2026-03-28 07:05:59,023 - lava_callback - INFO - Starting async job submit for node 69c4dac6590270ad7b6da38f 2026-03-28 07:05:59,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:59,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dac6590270ad7b6da38f HTTP/1.1" 200 None 2026-03-28 07:05:59,051 - lava_callback - DEBUG - Processing node 69c4dac6590270ad7b6da38f: baseline-arm-clabbe 2026-03-28 07:05:59,052 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dac6590270ad7b6da38f 2026-03-28 07:05:59,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:59,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dac6590270ad7b6da38f HTTP/1.1" 200 None 2026-03-28 07:05:59,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:59,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:05:59,082 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:05:59,103 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dac6590270ad7b6da38f HTTP/1.1" 200 None 2026-03-28 07:05:59,109 - lava_callback - ERROR - Error processing callback for node 69c4dac6590270ad7b6da38f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39768 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:06:39,039 - lava_callback - INFO - Received callback for node 69c4daeb590270ad7b6da39c INFO: 172.21.0.1:39776 - "POST /node/69c4daeb590270ad7b6da39c HTTP/1.0" 202 Accepted 2026-03-28 07:06:39,045 - lava_callback - INFO - Starting async job submit for node 69c4daeb590270ad7b6da39c 2026-03-28 07:06:39,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:06:39,064 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4daeb590270ad7b6da39c HTTP/1.1" 200 None 2026-03-28 07:06:39,065 - lava_callback - DEBUG - Processing node 69c4daeb590270ad7b6da39c: baseline-arm-clabbe 2026-03-28 07:06:39,065 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4daeb590270ad7b6da39c 2026-03-28 07:06:39,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:06:39,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4daeb590270ad7b6da39c HTTP/1.1" 200 None 2026-03-28 07:06:39,085 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:06:39,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:06:39,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:06:39,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4daeb590270ad7b6da39c HTTP/1.1" 200 None 2026-03-28 07:06:39,129 - lava_callback - ERROR - Error processing callback for node 69c4daeb590270ad7b6da39c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:60222 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59524 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34172 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:07:19,059 - lava_callback - INFO - Received callback for node 69c4db11590270ad7b6da3ab INFO: 172.21.0.1:34180 - "POST /node/69c4db11590270ad7b6da3ab HTTP/1.0" 202 Accepted 2026-03-28 07:07:19,064 - lava_callback - INFO - Starting async job submit for node 69c4db11590270ad7b6da3ab 2026-03-28 07:07:19,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:19,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db11590270ad7b6da3ab HTTP/1.1" 200 None 2026-03-28 07:07:19,087 - lava_callback - DEBUG - Processing node 69c4db11590270ad7b6da3ab: baseline-arm-clabbe 2026-03-28 07:07:19,088 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4db11590270ad7b6da3ab 2026-03-28 07:07:19,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:19,102 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db11590270ad7b6da3ab HTTP/1.1" 200 None 2026-03-28 07:07:19,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:19,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:07:19,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:19,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4db11590270ad7b6da3ab HTTP/1.1" 200 None 2026-03-28 07:07:19,144 - lava_callback - ERROR - Error processing callback for node 69c4db11590270ad7b6da3ab: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48054 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:07:39,069 - lava_callback - INFO - Received callback for node 69c4db37590270ad7b6da3bd INFO: 172.21.0.1:48070 - "POST /node/69c4db37590270ad7b6da3bd HTTP/1.0" 202 Accepted 2026-03-28 07:07:39,074 - lava_callback - INFO - Starting async job submit for node 69c4db37590270ad7b6da3bd 2026-03-28 07:07:39,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:39,089 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db37590270ad7b6da3bd HTTP/1.1" 200 None 2026-03-28 07:07:39,090 - lava_callback - DEBUG - Processing node 69c4db37590270ad7b6da3bd: baseline-arm-clabbe 2026-03-28 07:07:39,090 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4db37590270ad7b6da3bd 2026-03-28 07:07:39,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:39,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db37590270ad7b6da3bd HTTP/1.1" 200 None 2026-03-28 07:07:39,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:39,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:07:39,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:07:39,147 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4db37590270ad7b6da3bd HTTP/1.1" 200 None 2026-03-28 07:07:39,148 - lava_callback - ERROR - Error processing callback for node 69c4db37590270ad7b6da3bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45082 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53134 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:08:19,098 - lava_callback - INFO - Received callback for node 69c4db5c590270ad7b6da3ce INFO: 172.21.0.1:53146 - "POST /node/69c4db5c590270ad7b6da3ce HTTP/1.0" 202 Accepted 2026-03-28 07:08:19,104 - lava_callback - INFO - Starting async job submit for node 69c4db5c590270ad7b6da3ce 2026-03-28 07:08:19,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:19,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db5c590270ad7b6da3ce HTTP/1.1" 200 None 2026-03-28 07:08:19,120 - lava_callback - DEBUG - Processing node 69c4db5c590270ad7b6da3ce: baseline-arm-clabbe 2026-03-28 07:08:19,120 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4db5c590270ad7b6da3ce 2026-03-28 07:08:19,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:19,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db5c590270ad7b6da3ce HTTP/1.1" 200 None 2026-03-28 07:08:19,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:19,176 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:08:19,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:19,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4db5c590270ad7b6da3ce HTTP/1.1" 200 None 2026-03-28 07:08:19,204 - lava_callback - ERROR - Error processing callback for node 69c4db5c590270ad7b6da3ce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38834 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40980 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44146 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60478 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:08:59,158 - lava_callback - INFO - Received callback for node 69c4db81590270ad7b6da3db INFO: 172.21.0.1:60480 - "POST /node/69c4db81590270ad7b6da3db HTTP/1.0" 202 Accepted 2026-03-28 07:08:59,164 - lava_callback - INFO - Starting async job submit for node 69c4db81590270ad7b6da3db 2026-03-28 07:08:59,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:59,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db81590270ad7b6da3db HTTP/1.1" 200 None 2026-03-28 07:08:59,179 - lava_callback - DEBUG - Processing node 69c4db81590270ad7b6da3db: baseline-arm-clabbe 2026-03-28 07:08:59,180 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4db81590270ad7b6da3db 2026-03-28 07:08:59,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:59,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4db81590270ad7b6da3db HTTP/1.1" 200 None 2026-03-28 07:08:59,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:59,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:08:59,209 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:08:59,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4db81590270ad7b6da3db HTTP/1.1" 200 None 2026-03-28 07:08:59,241 - lava_callback - ERROR - Error processing callback for node 69c4db81590270ad7b6da3db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36990 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44904 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49378 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:09:39,267 - lava_callback - INFO - Received callback for node 69c4dba6590270ad7b6da3ec INFO: 172.21.0.1:49382 - "POST /node/69c4dba6590270ad7b6da3ec HTTP/1.0" 202 Accepted 2026-03-28 07:09:39,273 - lava_callback - INFO - Starting async job submit for node 69c4dba6590270ad7b6da3ec 2026-03-28 07:09:39,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:09:39,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dba6590270ad7b6da3ec HTTP/1.1" 200 None 2026-03-28 07:09:39,289 - lava_callback - DEBUG - Processing node 69c4dba6590270ad7b6da3ec: baseline-arm-clabbe 2026-03-28 07:09:39,289 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dba6590270ad7b6da3ec 2026-03-28 07:09:39,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:09:39,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dba6590270ad7b6da3ec HTTP/1.1" 200 None 2026-03-28 07:09:39,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:09:39,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:09:39,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:09:39,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dba6590270ad7b6da3ec HTTP/1.1" 200 None 2026-03-28 07:09:39,348 - lava_callback - ERROR - Error processing callback for node 69c4dba6590270ad7b6da3ec: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36386 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:10:19,169 - lava_callback - INFO - Received callback for node 69c4dbcb590270ad7b6da3fc INFO: 172.21.0.1:36394 - "POST /node/69c4dbcb590270ad7b6da3fc HTTP/1.0" 202 Accepted 2026-03-28 07:10:19,175 - lava_callback - INFO - Starting async job submit for node 69c4dbcb590270ad7b6da3fc 2026-03-28 07:10:19,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:10:19,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dbcb590270ad7b6da3fc HTTP/1.1" 200 None 2026-03-28 07:10:19,191 - lava_callback - DEBUG - Processing node 69c4dbcb590270ad7b6da3fc: baseline-arm-clabbe 2026-03-28 07:10:19,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dbcb590270ad7b6da3fc 2026-03-28 07:10:19,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:10:19,205 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dbcb590270ad7b6da3fc HTTP/1.1" 200 None 2026-03-28 07:10:19,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:10:19,222 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf353af1004e82292e0db7 HTTP/1.1" 200 None 2026-03-28 07:10:19,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:10:19,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dbcb590270ad7b6da3fc HTTP/1.1" 200 None 2026-03-28 07:10:19,249 - lava_callback - ERROR - Error processing callback for node 69c4dbcb590270ad7b6da3fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:56236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50362 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:11:07,273 - lava_callback - INFO - Received callback for node 69c4dbf5590270ad7b6da417 INFO: 172.21.0.1:50378 - "POST /node/69c4dbf5590270ad7b6da417 HTTP/1.0" 202 Accepted 2026-03-28 07:11:07,279 - lava_callback - INFO - Starting async job submit for node 69c4dbf5590270ad7b6da417 2026-03-28 07:11:07,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:11:07,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dbf5590270ad7b6da417 HTTP/1.1" 200 None 2026-03-28 07:11:07,295 - lava_callback - DEBUG - Processing node 69c4dbf5590270ad7b6da417: baseline-arm64-baylibre 2026-03-28 07:11:07,295 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dbf5590270ad7b6da417 2026-03-28 07:11:07,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:11:07,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dbf5590270ad7b6da417 HTTP/1.1" 200 None 2026-03-28 07:11:07,310 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:11:07,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:11:07,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:11:07,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dbf5590270ad7b6da417 HTTP/1.1" 200 None 2026-03-28 07:11:07,353 - lava_callback - ERROR - Error processing callback for node 69c4dbf5590270ad7b6da417: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52820 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34608 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56038 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36814 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56842 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:12:06,985 - lava_callback - INFO - Received callback for node 69c4dc45590270ad7b6da41d INFO: 172.21.0.1:56848 - "POST /node/69c4dc45590270ad7b6da41d HTTP/1.0" 202 Accepted 2026-03-28 07:12:06,995 - lava_callback - INFO - Starting async job submit for node 69c4dc45590270ad7b6da41d 2026-03-28 07:12:06,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:12:07,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dc45590270ad7b6da41d HTTP/1.1" 200 None 2026-03-28 07:12:07,013 - lava_callback - DEBUG - Processing node 69c4dc45590270ad7b6da41d: baseline-arm64-baylibre 2026-03-28 07:12:07,013 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dc45590270ad7b6da41d 2026-03-28 07:12:07,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:12:07,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dc45590270ad7b6da41d HTTP/1.1" 200 None 2026-03-28 07:12:07,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:12:07,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:12:07,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:12:07,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dc45590270ad7b6da41d HTTP/1.1" 200 None 2026-03-28 07:12:07,106 - lava_callback - ERROR - Error processing callback for node 69c4dc45590270ad7b6da41d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53498 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55678 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42152 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48292 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:13:27,251 - lava_callback - INFO - Received callback for node 69c4dc94590270ad7b6da42b INFO: 172.21.0.1:48308 - "POST /node/69c4dc94590270ad7b6da42b HTTP/1.0" 202 Accepted 2026-03-28 07:13:27,256 - lava_callback - INFO - Starting async job submit for node 69c4dc94590270ad7b6da42b 2026-03-28 07:13:27,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:13:27,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dc94590270ad7b6da42b HTTP/1.1" 200 None 2026-03-28 07:13:27,272 - lava_callback - DEBUG - Processing node 69c4dc94590270ad7b6da42b: baseline-arm64-baylibre 2026-03-28 07:13:27,273 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dc94590270ad7b6da42b 2026-03-28 07:13:27,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:13:27,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dc94590270ad7b6da42b HTTP/1.1" 200 None 2026-03-28 07:13:27,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:13:27,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:13:27,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:13:27,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dc94590270ad7b6da42b HTTP/1.1" 200 None 2026-03-28 07:13:27,329 - lava_callback - ERROR - Error processing callback for node 69c4dc94590270ad7b6da42b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:35932 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39988 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48360 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50020 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36596 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36612 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46262 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:15:07,188 - lava_callback - INFO - Received callback for node 69c4dce9590270ad7b6da449 INFO: 172.21.0.1:46266 - "POST /node/69c4dce9590270ad7b6da449 HTTP/1.0" 202 Accepted 2026-03-28 07:15:07,194 - lava_callback - INFO - Starting async job submit for node 69c4dce9590270ad7b6da449 2026-03-28 07:15:07,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:15:07,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dce9590270ad7b6da449 HTTP/1.1" 200 None 2026-03-28 07:15:07,209 - lava_callback - DEBUG - Processing node 69c4dce9590270ad7b6da449: baseline-arm64-baylibre 2026-03-28 07:15:07,209 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dce9590270ad7b6da449 2026-03-28 07:15:07,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:15:07,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dce9590270ad7b6da449 HTTP/1.1" 200 None 2026-03-28 07:15:07,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:15:07,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:15:07,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:15:07,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dce9590270ad7b6da449 HTTP/1.1" 200 None 2026-03-28 07:15:07,264 - lava_callback - ERROR - Error processing callback for node 69c4dce9590270ad7b6da449: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33746 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42230 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42238 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38738 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:16:27,353 - lava_callback - INFO - Received callback for node 69c4dd39590270ad7b6da46d INFO: 172.21.0.1:38750 - "POST /node/69c4dd39590270ad7b6da46d HTTP/1.0" 202 Accepted 2026-03-28 07:16:27,358 - lava_callback - INFO - Starting async job submit for node 69c4dd39590270ad7b6da46d 2026-03-28 07:16:27,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:16:27,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dd39590270ad7b6da46d HTTP/1.1" 200 None 2026-03-28 07:16:27,373 - lava_callback - DEBUG - Processing node 69c4dd39590270ad7b6da46d: baseline-arm64-baylibre 2026-03-28 07:16:27,374 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dd39590270ad7b6da46d 2026-03-28 07:16:27,374 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:16:27,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dd39590270ad7b6da46d HTTP/1.1" 200 None 2026-03-28 07:16:27,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:16:27,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:16:27,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:16:27,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dd39590270ad7b6da46d HTTP/1.1" 200 None 2026-03-28 07:16:27,437 - lava_callback - ERROR - Error processing callback for node 69c4dd39590270ad7b6da46d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43824 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47976 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38822 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33694 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37902 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39082 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57236 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49108 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34222 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40012 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40196 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39076 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46278 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45708 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57162 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51086 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49916 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33568 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45074 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54594 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48294 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39080 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32836 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54160 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36268 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36282 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56404 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:24:27,937 - lava_callback - INFO - Received callback for node 69c4df1d590270ad7b6da4b7 INFO: 172.21.0.1:56414 - "POST /node/69c4df1d590270ad7b6da4b7 HTTP/1.0" 202 Accepted 2026-03-28 07:24:27,943 - lava_callback - INFO - Starting async job submit for node 69c4df1d590270ad7b6da4b7 2026-03-28 07:24:27,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:24:27,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4df1d590270ad7b6da4b7 HTTP/1.1" 200 None 2026-03-28 07:24:27,959 - lava_callback - DEBUG - Processing node 69c4df1d590270ad7b6da4b7: baseline-arm64-baylibre 2026-03-28 07:24:27,959 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4df1d590270ad7b6da4b7 2026-03-28 07:24:27,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:24:27,973 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4df1d590270ad7b6da4b7 HTTP/1.1" 200 None 2026-03-28 07:24:27,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:24:28,159 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:24:28,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:24:28,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4df1d590270ad7b6da4b7 HTTP/1.1" 200 None 2026-03-28 07:24:28,191 - lava_callback - ERROR - Error processing callback for node 69c4df1d590270ad7b6da4b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48674 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48682 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55172 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44554 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44558 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:25:47,729 - lava_callback - INFO - Received callback for node 69c4df68590270ad7b6da4bc INFO: 172.21.0.1:44572 - "POST /node/69c4df68590270ad7b6da4bc HTTP/1.0" 202 Accepted 2026-03-28 07:25:47,735 - lava_callback - INFO - Starting async job submit for node 69c4df68590270ad7b6da4bc 2026-03-28 07:25:47,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:25:47,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4df68590270ad7b6da4bc HTTP/1.1" 200 None 2026-03-28 07:25:47,752 - lava_callback - DEBUG - Processing node 69c4df68590270ad7b6da4bc: baseline-arm64-baylibre 2026-03-28 07:25:47,752 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4df68590270ad7b6da4bc 2026-03-28 07:25:47,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:25:47,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4df68590270ad7b6da4bc HTTP/1.1" 200 None 2026-03-28 07:25:47,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:25:47,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:25:47,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:25:47,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4df68590270ad7b6da4bc HTTP/1.1" 200 None 2026-03-28 07:25:47,810 - lava_callback - ERROR - Error processing callback for node 69c4df68590270ad7b6da4bc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39958 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38438 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60614 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:27:07,845 - lava_callback - INFO - Received callback for node 69c4dfbb590270ad7b6da4c5 INFO: 172.21.0.1:60616 - "POST /node/69c4dfbb590270ad7b6da4c5 HTTP/1.0" 202 Accepted 2026-03-28 07:27:07,851 - lava_callback - INFO - Starting async job submit for node 69c4dfbb590270ad7b6da4c5 2026-03-28 07:27:07,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:27:07,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dfbb590270ad7b6da4c5 HTTP/1.1" 200 None 2026-03-28 07:27:07,866 - lava_callback - DEBUG - Processing node 69c4dfbb590270ad7b6da4c5: baseline-arm64-baylibre 2026-03-28 07:27:07,866 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4dfbb590270ad7b6da4c5 2026-03-28 07:27:07,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:27:07,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4dfbb590270ad7b6da4c5 HTTP/1.1" 200 None 2026-03-28 07:27:07,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:27:07,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:27:07,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:27:07,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4dfbb590270ad7b6da4c5 HTTP/1.1" 200 None 2026-03-28 07:27:07,918 - lava_callback - ERROR - Error processing callback for node 69c4dfbb590270ad7b6da4c5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:48628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42844 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35816 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41296 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41308 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46038 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46046 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55298 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38676 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40264 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56786 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33518 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58176 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:29:59,968 - lava_callback - INFO - Received callback for node 69c4e065590270ad7b6da4db INFO: 172.21.0.1:42354 - "POST /node/69c4e065590270ad7b6da4db HTTP/1.0" 202 Accepted 2026-03-28 07:29:59,974 - lava_callback - INFO - Starting async job submit for node 69c4e065590270ad7b6da4db 2026-03-28 07:29:59,975 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:29:59,989 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e065590270ad7b6da4db HTTP/1.1" 200 None 2026-03-28 07:29:59,989 - lava_callback - DEBUG - Processing node 69c4e065590270ad7b6da4db: baseline-arm64-clabbe 2026-03-28 07:29:59,990 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e065590270ad7b6da4db 2026-03-28 07:29:59,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:30:00,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e065590270ad7b6da4db HTTP/1.1" 200 None 2026-03-28 07:30:00,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:30:00,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:30:00,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:30:00,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e065590270ad7b6da4db HTTP/1.1" 200 None 2026-03-28 07:30:00,075 - lava_callback - ERROR - Error processing callback for node 69c4e065590270ad7b6da4db: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:42362 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43540 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45924 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35084 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59064 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59080 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:31:20,026 - lava_callback - INFO - Received callback for node 69c4e0b7590270ad7b6da4e2 INFO: 172.21.0.1:39178 - "POST /node/69c4e0b7590270ad7b6da4e2 HTTP/1.0" 202 Accepted 2026-03-28 07:31:20,032 - lava_callback - INFO - Starting async job submit for node 69c4e0b7590270ad7b6da4e2 2026-03-28 07:31:20,032 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:31:20,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e0b7590270ad7b6da4e2 HTTP/1.1" 200 None 2026-03-28 07:31:20,048 - lava_callback - DEBUG - Processing node 69c4e0b7590270ad7b6da4e2: baseline-arm64-clabbe 2026-03-28 07:31:20,048 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e0b7590270ad7b6da4e2 2026-03-28 07:31:20,048 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:31:20,062 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e0b7590270ad7b6da4e2 HTTP/1.1" 200 None 2026-03-28 07:31:20,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:31:20,079 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:31:20,081 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:31:20,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e0b7590270ad7b6da4e2 HTTP/1.1" 200 None 2026-03-28 07:31:20,106 - lava_callback - ERROR - Error processing callback for node 69c4e0b7590270ad7b6da4e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47114 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36500 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60772 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53478 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44182 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32934 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:32:40,100 - lava_callback - INFO - Received callback for node 69c4e107590270ad7b6da4ee INFO: 172.21.0.1:47778 - "POST /node/69c4e107590270ad7b6da4ee HTTP/1.0" 202 Accepted 2026-03-28 07:32:40,106 - lava_callback - INFO - Starting async job submit for node 69c4e107590270ad7b6da4ee 2026-03-28 07:32:40,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:32:40,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e107590270ad7b6da4ee HTTP/1.1" 200 None 2026-03-28 07:32:40,122 - lava_callback - DEBUG - Processing node 69c4e107590270ad7b6da4ee: baseline-arm64-clabbe 2026-03-28 07:32:40,122 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e107590270ad7b6da4ee 2026-03-28 07:32:40,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:32:40,136 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e107590270ad7b6da4ee HTTP/1.1" 200 None 2026-03-28 07:32:40,137 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:32:40,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:32:40,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:32:40,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e107590270ad7b6da4ee HTTP/1.1" 200 None 2026-03-28 07:32:40,187 - lava_callback - ERROR - Error processing callback for node 69c4e107590270ad7b6da4ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47796 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34120 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40246 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42090 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42094 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36362 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:34:00,176 - lava_callback - INFO - Received callback for node 69c4e158590270ad7b6da4fc INFO: 172.21.0.1:44942 - "POST /node/69c4e158590270ad7b6da4fc HTTP/1.0" 202 Accepted 2026-03-28 07:34:00,181 - lava_callback - INFO - Starting async job submit for node 69c4e158590270ad7b6da4fc 2026-03-28 07:34:00,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:34:00,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e158590270ad7b6da4fc HTTP/1.1" 200 None 2026-03-28 07:34:00,198 - lava_callback - DEBUG - Processing node 69c4e158590270ad7b6da4fc: baseline-arm64-clabbe 2026-03-28 07:34:00,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e158590270ad7b6da4fc 2026-03-28 07:34:00,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:34:00,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e158590270ad7b6da4fc HTTP/1.1" 200 None 2026-03-28 07:34:00,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:34:00,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:34:00,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:34:00,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e158590270ad7b6da4fc HTTP/1.1" 200 None 2026-03-28 07:34:00,257 - lava_callback - ERROR - Error processing callback for node 69c4e158590270ad7b6da4fc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39620 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59410 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54178 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35758 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54646 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52710 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41784 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:35:20,209 - lava_callback - INFO - Received callback for node 69c4e1a8590270ad7b6da503 INFO: 172.21.0.1:37904 - "POST /node/69c4e1a8590270ad7b6da503 HTTP/1.0" 202 Accepted 2026-03-28 07:35:20,215 - lava_callback - INFO - Starting async job submit for node 69c4e1a8590270ad7b6da503 2026-03-28 07:35:20,216 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:35:20,237 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e1a8590270ad7b6da503 HTTP/1.1" 200 None 2026-03-28 07:35:20,237 - lava_callback - DEBUG - Processing node 69c4e1a8590270ad7b6da503: baseline-arm64-clabbe 2026-03-28 07:35:20,238 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e1a8590270ad7b6da503 2026-03-28 07:35:20,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:35:20,252 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e1a8590270ad7b6da503 HTTP/1.1" 200 None 2026-03-28 07:35:20,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:35:20,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:35:20,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:35:20,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e1a8590270ad7b6da503 HTTP/1.1" 200 None 2026-03-28 07:35:20,290 - lava_callback - ERROR - Error processing callback for node 69c4e1a8590270ad7b6da503: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37916 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43002 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33130 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35954 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:48352 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37742 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52094 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:36:40,242 - lava_callback - INFO - Received callback for node 69c4e1fa590270ad7b6da50f INFO: 172.21.0.1:34124 - "POST /node/69c4e1fa590270ad7b6da50f HTTP/1.0" 202 Accepted 2026-03-28 07:36:40,248 - lava_callback - INFO - Starting async job submit for node 69c4e1fa590270ad7b6da50f 2026-03-28 07:36:40,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:36:40,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e1fa590270ad7b6da50f HTTP/1.1" 200 None 2026-03-28 07:36:40,275 - lava_callback - DEBUG - Processing node 69c4e1fa590270ad7b6da50f: baseline-arm64-clabbe 2026-03-28 07:36:40,276 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e1fa590270ad7b6da50f 2026-03-28 07:36:40,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:36:40,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e1fa590270ad7b6da50f HTTP/1.1" 200 None 2026-03-28 07:36:40,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:36:40,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c47b4a590270ad7b6d699d HTTP/1.1" 200 None 2026-03-28 07:36:40,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:36:40,325 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e1fa590270ad7b6da50f HTTP/1.1" 200 None 2026-03-28 07:36:40,330 - lava_callback - ERROR - Error processing callback for node 69c4e1fa590270ad7b6da50f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:34132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55286 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55302 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51992 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:52000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52016 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57520 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57536 - "GET /.well-known/security.txt HTTP/1.0" 404 Not Found INFO: 172.21.0.1:37288 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46456 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46466 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:37:48,499 - lava_callback - INFO - Received callback for node 69c4e242590270ad7b6da51b INFO: 172.21.0.1:46482 - "POST /node/69c4e242590270ad7b6da51b HTTP/1.0" 202 Accepted 2026-03-28 07:37:48,505 - lava_callback - INFO - Starting async job submit for node 69c4e242590270ad7b6da51b 2026-03-28 07:37:48,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:37:48,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e242590270ad7b6da51b HTTP/1.1" 200 None 2026-03-28 07:37:48,520 - lava_callback - DEBUG - Processing node 69c4e242590270ad7b6da51b: baseline-arm64-baylibre 2026-03-28 07:37:48,521 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e242590270ad7b6da51b 2026-03-28 07:37:48,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:37:48,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e242590270ad7b6da51b HTTP/1.1" 200 None 2026-03-28 07:37:48,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:37:48,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:37:48,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:37:48,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e242590270ad7b6da51b HTTP/1.1" 200 None 2026-03-28 07:37:48,589 - lava_callback - ERROR - Error processing callback for node 69c4e242590270ad7b6da51b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36530 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50684 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:35356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48890 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40122 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51862 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50522 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46996 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33798 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:39:20,376 - lava_callback - INFO - Received callback for node 69c4e29d590270ad7b6da528 INFO: 172.21.0.1:45866 - "POST /node/69c4e29d590270ad7b6da528 HTTP/1.0" 202 Accepted 2026-03-28 07:39:20,381 - lava_callback - INFO - Starting async job submit for node 69c4e29d590270ad7b6da528 2026-03-28 07:39:20,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:39:20,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e29d590270ad7b6da528 HTTP/1.1" 200 None 2026-03-28 07:39:20,399 - lava_callback - DEBUG - Processing node 69c4e29d590270ad7b6da528: baseline-arm64-clabbe 2026-03-28 07:39:20,399 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e29d590270ad7b6da528 2026-03-28 07:39:20,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:39:20,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e29d590270ad7b6da528 HTTP/1.1" 200 None 2026-03-28 07:39:20,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:39:20,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:39:20,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:39:20,488 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e29d590270ad7b6da528 HTTP/1.1" 200 None 2026-03-28 07:39:20,489 - lava_callback - ERROR - Error processing callback for node 69c4e29d590270ad7b6da528: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:45880 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45316 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43112 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39808 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56254 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:45762 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39284 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:40:28,546 - lava_callback - INFO - Received callback for node 69c4e2e5590270ad7b6da530 INFO: 172.21.0.1:39300 - "POST /node/69c4e2e5590270ad7b6da530 HTTP/1.0" 202 Accepted 2026-03-28 07:40:28,551 - lava_callback - INFO - Starting async job submit for node 69c4e2e5590270ad7b6da530 2026-03-28 07:40:28,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:40:28,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e2e5590270ad7b6da530 HTTP/1.1" 200 None 2026-03-28 07:40:28,567 - lava_callback - DEBUG - Processing node 69c4e2e5590270ad7b6da530: baseline-arm64-baylibre 2026-03-28 07:40:28,568 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e2e5590270ad7b6da530 2026-03-28 07:40:28,568 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:40:28,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e2e5590270ad7b6da530 HTTP/1.1" 200 None 2026-03-28 07:40:28,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:40:28,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:40:28,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:40:28,641 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e2e5590270ad7b6da530 HTTP/1.1" 200 None 2026-03-28 07:40:28,648 - lava_callback - ERROR - Error processing callback for node 69c4e2e5590270ad7b6da530: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:44314 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44098 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44106 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33704 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41010 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47950 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:47966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44228 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57908 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36964 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36968 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45090 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:42:00,476 - lava_callback - INFO - Received callback for node 69c4e33e590270ad7b6da540 INFO: 172.21.0.1:57778 - "POST /node/69c4e33e590270ad7b6da540 HTTP/1.0" 202 Accepted 2026-03-28 07:42:00,481 - lava_callback - INFO - Starting async job submit for node 69c4e33e590270ad7b6da540 2026-03-28 07:42:00,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:00,496 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e33e590270ad7b6da540 HTTP/1.1" 200 None 2026-03-28 07:42:00,497 - lava_callback - DEBUG - Processing node 69c4e33e590270ad7b6da540: baseline-arm64-clabbe 2026-03-28 07:42:00,497 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e33e590270ad7b6da540 2026-03-28 07:42:00,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:00,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e33e590270ad7b6da540 HTTP/1.1" 200 None 2026-03-28 07:42:00,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:00,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:42:00,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:00,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e33e590270ad7b6da540 HTTP/1.1" 200 None 2026-03-28 07:42:00,549 - lava_callback - ERROR - Error processing callback for node 69c4e33e590270ad7b6da540: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:57790 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36130 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:42:20,833 - lava_callback - INFO - Received callback for node 69c7802e29015d0f959933bd INFO: 172.21.0.1:47500 - "POST /node/69c7802e29015d0f959933bd HTTP/1.0" 202 Accepted 2026-03-28 07:42:20,840 - lava_callback - INFO - Starting async job submit for node 69c7802e29015d0f959933bd 2026-03-28 07:42:20,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:20,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7802e29015d0f959933bd HTTP/1.1" 200 None 2026-03-28 07:42:20,904 - lava_callback - DEBUG - Processing node 69c7802e29015d0f959933bd: baseline-nfs-arm64 2026-03-28 07:42:21,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:42:21,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:42:21,614 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7802e29015d0f959933bd/log.txt.gz 2026-03-28 07:42:21,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:42:22,099 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:42:22,101 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7802e29015d0f959933bd/lava-logs.yaml 2026-03-28 07:42:22,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 07:42:22,294 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 07:42:22,296 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69c7802e29015d0f959933bd/lava_callback.json.gz 2026-03-28 07:42:22,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:22,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7802e29015d0f959933bd HTTP/1.1" 200 None 2026-03-28 07:42:22,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:22,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 07:42:22,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:22,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7802e29015d0f959933bd HTTP/1.1" 200 None 2026-03-28 07:42:22,436 - lava_callback - INFO - Completed processing callback for node 69c7802e29015d0f959933bd INFO: 172.21.0.1:47510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33976 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:42:36,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:42:36,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:43150 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53184 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43320 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43330 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:43:20,533 - lava_callback - INFO - Received callback for node 69c4e38f590270ad7b6da54a INFO: 172.21.0.1:52006 - "POST /node/69c4e38f590270ad7b6da54a HTTP/1.0" 202 Accepted 2026-03-28 07:43:20,538 - lava_callback - INFO - Starting async job submit for node 69c4e38f590270ad7b6da54a 2026-03-28 07:43:20,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:43:20,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e38f590270ad7b6da54a HTTP/1.1" 200 None 2026-03-28 07:43:20,554 - lava_callback - DEBUG - Processing node 69c4e38f590270ad7b6da54a: baseline-arm64-clabbe 2026-03-28 07:43:20,554 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e38f590270ad7b6da54a 2026-03-28 07:43:20,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:43:20,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e38f590270ad7b6da54a HTTP/1.1" 200 None 2026-03-28 07:43:20,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:43:20,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:43:20,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:43:20,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e38f590270ad7b6da54a HTTP/1.1" 200 None 2026-03-28 07:43:20,611 - lava_callback - ERROR - Error processing callback for node 69c4e38f590270ad7b6da54a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52022 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42680 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60206 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48336 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56588 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48056 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:44:28,820 - lava_callback - INFO - Received callback for node 69c4e3d9590270ad7b6da553 INFO: 172.21.0.1:48072 - "POST /node/69c4e3d9590270ad7b6da553 HTTP/1.0" 202 Accepted 2026-03-28 07:44:28,826 - lava_callback - INFO - Starting async job submit for node 69c4e3d9590270ad7b6da553 2026-03-28 07:44:28,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:44:28,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e3d9590270ad7b6da553 HTTP/1.1" 200 None 2026-03-28 07:44:28,841 - lava_callback - DEBUG - Processing node 69c4e3d9590270ad7b6da553: baseline-arm64-baylibre 2026-03-28 07:44:28,841 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e3d9590270ad7b6da553 2026-03-28 07:44:28,842 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:44:28,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e3d9590270ad7b6da553 HTTP/1.1" 200 None 2026-03-28 07:44:28,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:44:28,873 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:44:28,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:44:28,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e3d9590270ad7b6da553 HTTP/1.1" 200 None 2026-03-28 07:44:28,903 - lava_callback - ERROR - Error processing callback for node 69c4e3d9590270ad7b6da553: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60768 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60780 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49078 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37332 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42136 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:42144 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55240 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33586 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54650 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54664 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:45:48,846 - lava_callback - INFO - Received callback for node 69c4e429590270ad7b6da560 INFO: 172.21.0.1:54670 - "POST /node/69c4e429590270ad7b6da560 HTTP/1.0" 202 Accepted 2026-03-28 07:45:48,851 - lava_callback - INFO - Starting async job submit for node 69c4e429590270ad7b6da560 2026-03-28 07:45:48,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:45:48,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e429590270ad7b6da560 HTTP/1.1" 200 None 2026-03-28 07:45:48,867 - lava_callback - DEBUG - Processing node 69c4e429590270ad7b6da560: baseline-arm64-baylibre 2026-03-28 07:45:48,867 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e429590270ad7b6da560 2026-03-28 07:45:48,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:45:48,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e429590270ad7b6da560 HTTP/1.1" 200 None 2026-03-28 07:45:48,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:45:48,895 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:45:48,897 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:45:48,929 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e429590270ad7b6da560 HTTP/1.1" 200 None 2026-03-28 07:45:48,935 - lava_callback - ERROR - Error processing callback for node 69c4e429590270ad7b6da560: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44868 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41156 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34356 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40218 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43134 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59140 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41822 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:41826 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:47:20,686 - lava_callback - INFO - Received callback for node 69c4e47f590270ad7b6da56b INFO: 172.21.0.1:36864 - "POST /node/69c4e47f590270ad7b6da56b HTTP/1.0" 202 Accepted 2026-03-28 07:47:20,692 - lava_callback - INFO - Starting async job submit for node 69c4e47f590270ad7b6da56b 2026-03-28 07:47:20,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:47:20,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e47f590270ad7b6da56b HTTP/1.1" 200 None 2026-03-28 07:47:20,707 - lava_callback - DEBUG - Processing node 69c4e47f590270ad7b6da56b: baseline-arm64-clabbe 2026-03-28 07:47:20,708 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e47f590270ad7b6da56b 2026-03-28 07:47:20,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:47:20,733 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e47f590270ad7b6da56b HTTP/1.1" 200 None 2026-03-28 07:47:20,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:47:20,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:47:20,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:47:20,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e47f590270ad7b6da56b HTTP/1.1" 200 None 2026-03-28 07:47:20,818 - lava_callback - ERROR - Error processing callback for node 69c4e47f590270ad7b6da56b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55668 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51622 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59420 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53180 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60716 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:48:40,770 - lava_callback - INFO - Received callback for node 69c4e4cf590270ad7b6da575 INFO: 172.21.0.1:38476 - "POST /node/69c4e4cf590270ad7b6da575 HTTP/1.0" 202 Accepted 2026-03-28 07:48:40,777 - lava_callback - INFO - Starting async job submit for node 69c4e4cf590270ad7b6da575 2026-03-28 07:48:40,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:48:40,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e4cf590270ad7b6da575 HTTP/1.1" 200 None 2026-03-28 07:48:40,795 - lava_callback - DEBUG - Processing node 69c4e4cf590270ad7b6da575: baseline-arm64-clabbe 2026-03-28 07:48:40,795 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e4cf590270ad7b6da575 2026-03-28 07:48:40,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:48:40,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e4cf590270ad7b6da575 HTTP/1.1" 200 None 2026-03-28 07:48:40,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:48:40,839 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:48:40,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:48:40,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e4cf590270ad7b6da575 HTTP/1.1" 200 None 2026-03-28 07:48:40,864 - lava_callback - ERROR - Error processing callback for node 69c4e4cf590270ad7b6da575: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59770 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59774 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59476 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49982 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33604 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33606 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:49:49,235 - lava_callback - INFO - Received callback for node 69c4e519590270ad7b6da581 INFO: 172.21.0.1:33612 - "POST /node/69c4e519590270ad7b6da581 HTTP/1.0" 202 Accepted 2026-03-28 07:49:49,241 - lava_callback - INFO - Starting async job submit for node 69c4e519590270ad7b6da581 2026-03-28 07:49:49,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:49:49,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e519590270ad7b6da581 HTTP/1.1" 200 None 2026-03-28 07:49:49,257 - lava_callback - DEBUG - Processing node 69c4e519590270ad7b6da581: baseline-arm64-baylibre 2026-03-28 07:49:49,258 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e519590270ad7b6da581 2026-03-28 07:49:49,258 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:49:49,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e519590270ad7b6da581 HTTP/1.1" 200 None 2026-03-28 07:49:49,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:49:49,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:49:49,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:49:49,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e519590270ad7b6da581 HTTP/1.1" 200 None 2026-03-28 07:49:49,314 - lava_callback - ERROR - Error processing callback for node 69c4e519590270ad7b6da581: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58492 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59752 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:59764 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40944 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54552 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39142 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:39154 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:51:20,909 - lava_callback - INFO - Received callback for node 69c4e571590270ad7b6da58d INFO: 172.21.0.1:37472 - "POST /node/69c4e571590270ad7b6da58d HTTP/1.0" 202 Accepted 2026-03-28 07:51:20,915 - lava_callback - INFO - Starting async job submit for node 69c4e571590270ad7b6da58d 2026-03-28 07:51:20,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:51:20,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e571590270ad7b6da58d HTTP/1.1" 200 None 2026-03-28 07:51:20,933 - lava_callback - DEBUG - Processing node 69c4e571590270ad7b6da58d: baseline-arm64-clabbe 2026-03-28 07:51:20,933 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e571590270ad7b6da58d 2026-03-28 07:51:20,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:51:20,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e571590270ad7b6da58d HTTP/1.1" 200 None 2026-03-28 07:51:20,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:51:20,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:51:20,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:51:20,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e571590270ad7b6da58d HTTP/1.1" 200 None 2026-03-28 07:51:20,991 - lava_callback - ERROR - Error processing callback for node 69c4e571590270ad7b6da58d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37482 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33462 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33472 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53706 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49246 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49262 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39952 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:52:40,994 - lava_callback - INFO - Received callback for node 69c4e5c2590270ad7b6da59a INFO: 172.21.0.1:52426 - "POST /node/69c4e5c2590270ad7b6da59a HTTP/1.0" 202 Accepted 2026-03-28 07:52:41,000 - lava_callback - INFO - Starting async job submit for node 69c4e5c2590270ad7b6da59a 2026-03-28 07:52:41,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:52:41,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e5c2590270ad7b6da59a HTTP/1.1" 200 None 2026-03-28 07:52:41,017 - lava_callback - DEBUG - Processing node 69c4e5c2590270ad7b6da59a: baseline-arm64-clabbe 2026-03-28 07:52:41,018 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e5c2590270ad7b6da59a 2026-03-28 07:52:41,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:52:41,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e5c2590270ad7b6da59a HTTP/1.1" 200 None 2026-03-28 07:52:41,051 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:52:41,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:52:41,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:52:41,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e5c2590270ad7b6da59a HTTP/1.1" 200 None 2026-03-28 07:52:41,121 - lava_callback - ERROR - Error processing callback for node 69c4e5c2590270ad7b6da59a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:52442 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58870 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:44008 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48660 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57658 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32842 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49574 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:54:01,084 - lava_callback - INFO - Received callback for node 69c4e611590270ad7b6da5a6 INFO: 172.21.0.1:39922 - "POST /node/69c4e611590270ad7b6da5a6 HTTP/1.0" 202 Accepted 2026-03-28 07:54:01,089 - lava_callback - INFO - Starting async job submit for node 69c4e611590270ad7b6da5a6 2026-03-28 07:54:01,090 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:54:01,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e611590270ad7b6da5a6 HTTP/1.1" 200 None 2026-03-28 07:54:01,105 - lava_callback - DEBUG - Processing node 69c4e611590270ad7b6da5a6: baseline-arm64-clabbe 2026-03-28 07:54:01,106 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e611590270ad7b6da5a6 2026-03-28 07:54:01,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:54:01,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e611590270ad7b6da5a6 HTTP/1.1" 200 None 2026-03-28 07:54:01,120 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:54:01,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:54:01,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:54:01,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e611590270ad7b6da5a6 HTTP/1.1" 200 None 2026-03-28 07:54:01,157 - lava_callback - ERROR - Error processing callback for node 69c4e611590270ad7b6da5a6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:39926 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40784 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41248 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48284 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39616 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39630 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37018 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48036 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:36798 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:55:21,053 - lava_callback - INFO - Received callback for node 69c4e662590270ad7b6da5b1 INFO: 172.21.0.1:46578 - "POST /node/69c4e662590270ad7b6da5b1 HTTP/1.0" 202 Accepted 2026-03-28 07:55:21,059 - lava_callback - INFO - Starting async job submit for node 69c4e662590270ad7b6da5b1 2026-03-28 07:55:21,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:55:21,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e662590270ad7b6da5b1 HTTP/1.1" 200 None 2026-03-28 07:55:21,075 - lava_callback - DEBUG - Processing node 69c4e662590270ad7b6da5b1: baseline-arm64-clabbe 2026-03-28 07:55:21,076 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e662590270ad7b6da5b1 2026-03-28 07:55:21,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:55:21,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e662590270ad7b6da5b1 HTTP/1.1" 200 None 2026-03-28 07:55:21,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:55:21,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:55:21,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:55:21,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e662590270ad7b6da5b1 HTTP/1.1" 200 None 2026-03-28 07:55:21,141 - lava_callback - ERROR - Error processing callback for node 69c4e662590270ad7b6da5b1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:46584 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44490 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44494 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38430 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56480 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37306 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38956 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42162 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:56:41,170 - lava_callback - INFO - Received callback for node 69c4e6b5590270ad7b6da5b8 INFO: 172.21.0.1:53544 - "POST /node/69c4e6b5590270ad7b6da5b8 HTTP/1.0" 202 Accepted 2026-03-28 07:56:41,176 - lava_callback - INFO - Starting async job submit for node 69c4e6b5590270ad7b6da5b8 2026-03-28 07:56:41,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:56:41,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e6b5590270ad7b6da5b8 HTTP/1.1" 200 None 2026-03-28 07:56:41,213 - lava_callback - DEBUG - Processing node 69c4e6b5590270ad7b6da5b8: baseline-arm64-clabbe 2026-03-28 07:56:41,214 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e6b5590270ad7b6da5b8 2026-03-28 07:56:41,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:56:41,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e6b5590270ad7b6da5b8 HTTP/1.1" 200 None 2026-03-28 07:56:41,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:56:41,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:56:41,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:56:41,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e6b5590270ad7b6da5b8 HTTP/1.1" 200 None 2026-03-28 07:56:41,293 - lava_callback - ERROR - Error processing callback for node 69c4e6b5590270ad7b6da5b8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53548 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53556 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50058 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42810 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49858 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42458 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57802 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48432 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:57:49,486 - lava_callback - INFO - Received callback for node 69c4e6fb590270ad7b6da5c4 INFO: 172.21.0.1:48440 - "POST /node/69c4e6fb590270ad7b6da5c4 HTTP/1.0" 202 Accepted 2026-03-28 07:57:49,492 - lava_callback - INFO - Starting async job submit for node 69c4e6fb590270ad7b6da5c4 2026-03-28 07:57:49,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:57:49,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e6fb590270ad7b6da5c4 HTTP/1.1" 200 None 2026-03-28 07:57:49,511 - lava_callback - DEBUG - Processing node 69c4e6fb590270ad7b6da5c4: baseline-arm64-baylibre 2026-03-28 07:57:49,511 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e6fb590270ad7b6da5c4 2026-03-28 07:57:49,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:57:49,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e6fb590270ad7b6da5c4 HTTP/1.1" 200 None 2026-03-28 07:57:49,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:57:49,539 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 07:57:49,540 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:57:49,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e6fb590270ad7b6da5c4 HTTP/1.1" 200 None 2026-03-28 07:57:49,567 - lava_callback - ERROR - Error processing callback for node 69c4e6fb590270ad7b6da5c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:47006 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46194 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50128 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39004 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55142 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52428 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35618 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50452 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:50460 - "GET / HTTP/1.0" 200 OK 2026-03-28 07:59:21,278 - lava_callback - INFO - Received callback for node 69c4e755590270ad7b6da5cf INFO: 172.21.0.1:50416 - "POST /node/69c4e755590270ad7b6da5cf HTTP/1.0" 202 Accepted 2026-03-28 07:59:21,283 - lava_callback - INFO - Starting async job submit for node 69c4e755590270ad7b6da5cf 2026-03-28 07:59:21,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:59:21,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e755590270ad7b6da5cf HTTP/1.1" 200 None 2026-03-28 07:59:21,299 - lava_callback - DEBUG - Processing node 69c4e755590270ad7b6da5cf: baseline-arm64-clabbe 2026-03-28 07:59:21,299 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e755590270ad7b6da5cf 2026-03-28 07:59:21,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:59:21,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e755590270ad7b6da5cf HTTP/1.1" 200 None 2026-03-28 07:59:21,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:59:21,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 07:59:21,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 07:59:21,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e755590270ad7b6da5cf HTTP/1.1" 200 None 2026-03-28 07:59:21,357 - lava_callback - ERROR - Error processing callback for node 69c4e755590270ad7b6da5cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:48508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49638 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49644 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:49132 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45642 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36484 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41042 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:00:45,177 - lava_callback - INFO - Received callback for node 69c72d0329015d0f95992389 INFO: 172.21.0.1:41052 - "POST /node/69c72d0329015d0f95992389 HTTP/1.0" 202 Accepted 2026-03-28 08:00:45,183 - lava_callback - INFO - Starting async job submit for node 69c72d0329015d0f95992389 2026-03-28 08:00:45,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:00:45,244 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0329015d0f95992389 HTTP/1.1" 200 None 2026-03-28 08:00:45,245 - lava_callback - DEBUG - Processing node 69c72d0329015d0f95992389: baseline-arm64-broonie 2026-03-28 08:00:45,473 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:00:45,626 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:00:45,628 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0329015d0f95992389/log.txt.gz 2026-03-28 08:00:45,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:00:45,973 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:00:45,975 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0329015d0f95992389/lava-logs.yaml 2026-03-28 08:00:45,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:00:46,306 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:00:46,308 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c72d0329015d0f95992389/lava_callback.json.gz 2026-03-28 08:00:46,331 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:00:46,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c72d0329015d0f95992389 HTTP/1.1" 200 None 2026-03-28 08:00:46,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:00:46,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 08:00:46,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:00:46,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c72d0329015d0f95992389 HTTP/1.1" 200 None 2026-03-28 08:00:46,394 - lava_callback - INFO - Completed processing callback for node 69c72d0329015d0f95992389 INFO: 172.21.0.1:41060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:35034 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:01:01,399 - lava_callback - INFO - Received callback for node 69c4e7a7590270ad7b6da5d9 INFO: 172.21.0.1:33790 - "POST /node/69c4e7a7590270ad7b6da5d9 HTTP/1.0" 202 Accepted 2026-03-28 08:01:01,405 - lava_callback - INFO - Starting async job submit for node 69c4e7a7590270ad7b6da5d9 2026-03-28 08:01:01,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:01:01,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e7a7590270ad7b6da5d9 HTTP/1.1" 200 None 2026-03-28 08:01:01,420 - lava_callback - DEBUG - Processing node 69c4e7a7590270ad7b6da5d9: baseline-arm64-clabbe 2026-03-28 08:01:01,421 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e7a7590270ad7b6da5d9 2026-03-28 08:01:01,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:01:01,434 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e7a7590270ad7b6da5d9 HTTP/1.1" 200 None 2026-03-28 08:01:01,435 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:01:01,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c1d8400efaf21d93103552 HTTP/1.1" 200 None 2026-03-28 08:01:01,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:01:01,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e7a7590270ad7b6da5d9 HTTP/1.1" 200 None 2026-03-28 08:01:01,475 - lava_callback - ERROR - Error processing callback for node 69c4e7a7590270ad7b6da5d9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:33802 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:01:06,820 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:01:06,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:53098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:53104 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33176 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41268 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46060 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46072 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49974 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55608 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.21.0.1:55620 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:02:09,722 - lava_callback - INFO - Received callback for node 69c4e7f3590270ad7b6da5e0 INFO: 172.21.0.1:59924 - "POST /node/69c4e7f3590270ad7b6da5e0 HTTP/1.0" 202 Accepted 2026-03-28 08:02:09,728 - lava_callback - INFO - Starting async job submit for node 69c4e7f3590270ad7b6da5e0 2026-03-28 08:02:09,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:02:09,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e7f3590270ad7b6da5e0 HTTP/1.1" 200 None 2026-03-28 08:02:09,745 - lava_callback - DEBUG - Processing node 69c4e7f3590270ad7b6da5e0: baseline-arm64-baylibre 2026-03-28 08:02:09,745 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e7f3590270ad7b6da5e0 2026-03-28 08:02:09,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:02:09,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e7f3590270ad7b6da5e0 HTTP/1.1" 200 None 2026-03-28 08:02:09,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:02:09,776 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:02:09,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:02:09,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e7f3590270ad7b6da5e0 HTTP/1.1" 200 None 2026-03-28 08:02:09,804 - lava_callback - ERROR - Error processing callback for node 69c4e7f3590270ad7b6da5e0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:59936 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:59952 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52398 - "GET /wiki HTTP/1.0" 404 Not Found INFO: 172.21.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37214 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56872 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56888 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54670 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:03:05,346 - lava_callback - INFO - Received callback for node 69c7854e29015d0f959934a9 INFO: 172.21.0.1:33768 - "POST /node/69c7854e29015d0f959934a9 HTTP/1.0" 202 Accepted 2026-03-28 08:03:05,352 - lava_callback - INFO - Starting async job submit for node 69c7854e29015d0f959934a9 2026-03-28 08:03:05,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:05,389 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7854e29015d0f959934a9 HTTP/1.1" 200 None 2026-03-28 08:03:05,390 - lava_callback - DEBUG - Processing node 69c7854e29015d0f959934a9: baseline-arm64-broonie 2026-03-28 08:03:05,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:03:05,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:03:05,925 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7854e29015d0f959934a9/log.txt.gz 2026-03-28 08:03:06,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.21.0.1:33772 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:03:10,855 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:03:10,857 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7854e29015d0f959934a9/lava-logs.yaml 2026-03-28 08:03:10,863 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-03-28 08:03:11,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-03-28 08:03:11,022 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69c7854e29015d0f959934a9/lava_callback.json.gz 2026-03-28 08:03:11,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:11,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c7854e29015d0f959934a9 HTTP/1.1" 200 None 2026-03-28 08:03:11,062 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:11,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c71e4e29015d0f9598f750 HTTP/1.1" 200 None 2026-03-28 08:03:11,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:11,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c7854e29015d0f959934a9 HTTP/1.1" 200 None 2026-03-28 08:03:11,125 - lava_callback - INFO - Completed processing callback for node 69c7854e29015d0f959934a9 INFO: 172.21.0.1:43702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:43718 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52240 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:03:29,830 - lava_callback - INFO - Received callback for node 69c4e840590270ad7b6da5eb INFO: 172.21.0.1:40932 - "POST /node/69c4e840590270ad7b6da5eb HTTP/1.0" 202 Accepted 2026-03-28 08:03:29,835 - lava_callback - INFO - Starting async job submit for node 69c4e840590270ad7b6da5eb 2026-03-28 08:03:29,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:29,851 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e840590270ad7b6da5eb HTTP/1.1" 200 None 2026-03-28 08:03:29,852 - lava_callback - DEBUG - Processing node 69c4e840590270ad7b6da5eb: baseline-arm64-baylibre 2026-03-28 08:03:29,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e840590270ad7b6da5eb 2026-03-28 08:03:29,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:29,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e840590270ad7b6da5eb HTTP/1.1" 200 None 2026-03-28 08:03:29,868 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:29,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:03:29,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:29,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e840590270ad7b6da5eb HTTP/1.1" 200 None 2026-03-28 08:03:29,907 - lava_callback - ERROR - Error processing callback for node 69c4e840590270ad7b6da5eb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:40938 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:03:36,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:03:36,853 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.21.0.1:45936 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45942 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40938 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38648 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46658 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:46664 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39728 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39978 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39788 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39796 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:04:49,803 - lava_callback - INFO - Received callback for node 69c4e88f590270ad7b6da5f1 INFO: 172.21.0.1:41882 - "POST /node/69c4e88f590270ad7b6da5f1 HTTP/1.0" 202 Accepted 2026-03-28 08:04:49,808 - lava_callback - INFO - Starting async job submit for node 69c4e88f590270ad7b6da5f1 2026-03-28 08:04:49,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:04:49,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e88f590270ad7b6da5f1 HTTP/1.1" 200 None 2026-03-28 08:04:49,824 - lava_callback - DEBUG - Processing node 69c4e88f590270ad7b6da5f1: baseline-arm64-baylibre 2026-03-28 08:04:49,824 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e88f590270ad7b6da5f1 2026-03-28 08:04:49,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:04:49,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e88f590270ad7b6da5f1 HTTP/1.1" 200 None 2026-03-28 08:04:49,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:04:49,852 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:04:49,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:04:49,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e88f590270ad7b6da5f1 HTTP/1.1" 200 None 2026-03-28 08:04:49,879 - lava_callback - ERROR - Error processing callback for node 69c4e88f590270ad7b6da5f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:41884 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54474 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:58576 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46722 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50350 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:41818 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:43450 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:06:09,883 - lava_callback - INFO - Received callback for node 69c4e8e2590270ad7b6da5fb INFO: 172.21.0.1:38884 - "POST /node/69c4e8e2590270ad7b6da5fb HTTP/1.0" 202 Accepted 2026-03-28 08:06:09,888 - lava_callback - INFO - Starting async job submit for node 69c4e8e2590270ad7b6da5fb 2026-03-28 08:06:09,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:06:09,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e8e2590270ad7b6da5fb HTTP/1.1" 200 None 2026-03-28 08:06:09,904 - lava_callback - DEBUG - Processing node 69c4e8e2590270ad7b6da5fb: baseline-arm64-baylibre 2026-03-28 08:06:09,905 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e8e2590270ad7b6da5fb 2026-03-28 08:06:09,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:06:09,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e8e2590270ad7b6da5fb HTTP/1.1" 200 None 2026-03-28 08:06:09,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:06:09,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:06:09,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:06:09,957 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e8e2590270ad7b6da5fb HTTP/1.1" 200 None 2026-03-28 08:06:09,965 - lava_callback - ERROR - Error processing callback for node 69c4e8e2590270ad7b6da5fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:38898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:38906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57000 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52858 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52860 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:34562 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54164 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:40894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:45036 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:07:29,983 - lava_callback - INFO - Received callback for node 69c4e932590270ad7b6da607 INFO: 172.21.0.1:53694 - "POST /node/69c4e932590270ad7b6da607 HTTP/1.0" 202 Accepted 2026-03-28 08:07:29,988 - lava_callback - INFO - Starting async job submit for node 69c4e932590270ad7b6da607 2026-03-28 08:07:29,989 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:07:30,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e932590270ad7b6da607 HTTP/1.1" 200 None 2026-03-28 08:07:30,004 - lava_callback - DEBUG - Processing node 69c4e932590270ad7b6da607: baseline-arm64-baylibre 2026-03-28 08:07:30,005 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e932590270ad7b6da607 2026-03-28 08:07:30,005 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:07:30,019 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e932590270ad7b6da607 HTTP/1.1" 200 None 2026-03-28 08:07:30,020 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:07:30,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:07:30,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:07:30,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e932590270ad7b6da607 HTTP/1.1" 200 None 2026-03-28 08:07:30,056 - lava_callback - ERROR - Error processing callback for node 69c4e932590270ad7b6da607: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:53696 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54506 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54508 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46984 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52374 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:33054 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:33070 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50966 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39732 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39742 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:08:50,159 - lava_callback - INFO - Received callback for node 69c4e980590270ad7b6da60d INFO: 172.21.0.1:36710 - "POST /node/69c4e980590270ad7b6da60d HTTP/1.0" 202 Accepted 2026-03-28 08:08:50,165 - lava_callback - INFO - Starting async job submit for node 69c4e980590270ad7b6da60d 2026-03-28 08:08:50,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:08:50,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e980590270ad7b6da60d HTTP/1.1" 200 None 2026-03-28 08:08:50,181 - lava_callback - DEBUG - Processing node 69c4e980590270ad7b6da60d: baseline-arm64-baylibre 2026-03-28 08:08:50,182 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e980590270ad7b6da60d 2026-03-28 08:08:50,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:08:50,202 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e980590270ad7b6da60d HTTP/1.1" 200 None 2026-03-28 08:08:50,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:08:50,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:08:50,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:08:50,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e980590270ad7b6da60d HTTP/1.1" 200 None 2026-03-28 08:08:50,245 - lava_callback - ERROR - Error processing callback for node 69c4e980590270ad7b6da60d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:36712 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:53770 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56872 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:56876 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:39634 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:38640 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55510 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:55512 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:44682 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:10:01,905 - lava_callback - INFO - Received callback for node 69c4e9d6590270ad7b6da617 INFO: 172.21.0.1:37390 - "POST /node/69c4e9d6590270ad7b6da617 HTTP/1.0" 202 Accepted 2026-03-28 08:10:01,911 - lava_callback - INFO - Starting async job submit for node 69c4e9d6590270ad7b6da617 2026-03-28 08:10:01,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:10:01,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e9d6590270ad7b6da617 HTTP/1.1" 200 None 2026-03-28 08:10:01,935 - lava_callback - DEBUG - Processing node 69c4e9d6590270ad7b6da617: baseline-arm64-clabbe 2026-03-28 08:10:01,936 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4e9d6590270ad7b6da617 2026-03-28 08:10:01,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:10:01,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4e9d6590270ad7b6da617 HTTP/1.1" 200 None 2026-03-28 08:10:01,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:10:01,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:10:01,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:10:02,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4e9d6590270ad7b6da617 HTTP/1.1" 200 None 2026-03-28 08:10:02,015 - lava_callback - ERROR - Error processing callback for node 69c4e9d6590270ad7b6da617: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:37404 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52478 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52486 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:49592 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:50028 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58840 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58850 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:36318 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:37790 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:37800 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52352 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:11:30,239 - lava_callback - INFO - Received callback for node 69c4ea21590270ad7b6da622 INFO: 172.21.0.1:49122 - "POST /node/69c4ea21590270ad7b6da622 HTTP/1.0" 202 Accepted 2026-03-28 08:11:30,245 - lava_callback - INFO - Starting async job submit for node 69c4ea21590270ad7b6da622 2026-03-28 08:11:30,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:11:30,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ea21590270ad7b6da622 HTTP/1.1" 200 None 2026-03-28 08:11:30,261 - lava_callback - DEBUG - Processing node 69c4ea21590270ad7b6da622: baseline-arm64-baylibre 2026-03-28 08:11:30,261 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4ea21590270ad7b6da622 2026-03-28 08:11:30,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:11:30,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ea21590270ad7b6da622 HTTP/1.1" 200 None 2026-03-28 08:11:30,277 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:11:30,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:11:30,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:11:30,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ea21590270ad7b6da622 HTTP/1.1" 200 None 2026-03-28 08:11:30,330 - lava_callback - ERROR - Error processing callback for node 69c4ea21590270ad7b6da622: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:49126 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58432 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:58446 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:47756 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:60804 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:57538 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:54566 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:46024 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57406 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:42032 - "GET / HTTP/1.0" 200 OK 2026-03-28 08:13:01,913 - lava_callback - INFO - Received callback for node 69c4ea77590270ad7b6da62b INFO: 172.21.0.1:50122 - "POST /node/69c4ea77590270ad7b6da62b HTTP/1.0" 202 Accepted 2026-03-28 08:13:01,922 - lava_callback - INFO - Starting async job submit for node 69c4ea77590270ad7b6da62b 2026-03-28 08:13:01,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:13:01,943 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ea77590270ad7b6da62b HTTP/1.1" 200 None 2026-03-28 08:13:01,944 - lava_callback - DEBUG - Processing node 69c4ea77590270ad7b6da62b: baseline-arm64-clabbe 2026-03-28 08:13:01,944 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69c4ea77590270ad7b6da62b 2026-03-28 08:13:01,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:13:01,968 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c4ea77590270ad7b6da62b HTTP/1.1" 200 None 2026-03-28 08:13:01,970 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:13:01,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69bf3545f1004e82292e0dc6 HTTP/1.1" 200 None 2026-03-28 08:13:01,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-03-28 08:13:02,025 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69c4ea77590270ad7b6da62b HTTP/1.1" 200 None 2026-03-28 08:13:02,026 - lava_callback - ERROR - Error processing callback for node 69c4ea77590270ad7b6da62b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.21.0.1:50124 - "GET / HTTP/1.0" 200 OK INFO: 172.21.0.1:52202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.21.0.1:52208 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]